tch-prob 1

Chap 3. Random VariablesThe outcome of a random experiment need not be a number.

However, we are usually interested in some measurement or numeric attribute of the outcome.

For example, toss a coin n times, total number of heads = ?

What is the prob. of the resulting numerical values ?

A random variable X is a function that assigns a real number, , to each outcome .

X

X

real line

s domain

Sx Range

( )X x

tch-prob 2

Ex.3.1 : HHH HHT HTH THH HTT THT TTH TTT

: 3 2 2 2 1 1 1 0

Let B be some subset of Sx

Event B in Sx occurs whenever

event A in S occurs.

Events A and B are equivalent events.

X

:A X in B A

B

S

real line

:P B P A P X in B

0,1,2,3XS

tch-prob 3

X

:A X in B

A

B

S

real line

HHH

HHTHTHTHH

HTTTHTTTH

TTT

0 1 2 3

tch-prob 4

3.2 Cumulative Distribution Function

Cumulative distribution function (cdf) of a random variable X is defined as

a convenient way of specifying the probability of all semi-infinite intervals of the real line of the form .

:

XF x P X x for x

P X x

( , ]x

tch-prob 5

Cdf has the following properties:

i. cdf is a prob., axiom 1 & corollary 2.

ii. is the entire sample space & Axiom II.

iii. is an empty set. Corollary 3.

iv. nondecreasing function, corollary 7.

v. continuous from the right. h>0

0 1X

F x

lim 1X

F xx x

lim 0X

F xx

,X X

if a b then F a F b

lim0X X X

F b F b h F bh

x

tch-prob 6

vi.

vii.

if cdf is continuous at b.

viii. Corollary 1.

X X

P a X b F b F a

X a a X b X b

vi, 0...

0

X X

X X

P X b F b F b

Let a b in P b X b F b F b as

P X b

1X

P X x F x

( ) ( ) ( ) ( )X X X X X XP a X b F a F a F b F a F b F a

a X b X a a X b

tch-prob 7

Ex.3.4. Fig 3.3 toss a coin 3 times. Count Heads.

X

7/8

1/2

1

1/8

Fx(x)

1 3 3 11 2 38 8 8 80 0

1 0

F x u x u x u x u xXx

u xx

X

1/8

fx(x)

0 1 32

1/8

3/83/8 probability mass function (pmf)

For discrete random variable, cdf is

right-continuous, staircase function of x.

1

2

3

4

1/ 8 0

3/ 8 1( )

3 / 8 2

1/ 8 3

X k

x

xp x

x

x

tch-prob 8

Ex.3.5. The transmission time X of messages obeys the exponential probability law with parameter .

is continuous for all x, its derivative exists everywhere except at x=0.

pdf.

0 0

1 0

0

find 1X

x

x

e x

xP X x e x

cdf F x P X x P X x

Fx(x) 1

x

'0 0

0X

xF x

xe x

F'x(x)

x

1 xe xe

( )XF x

tch-prob 9

Continuous random variable

cdf is continuous everywhere, and smooth enough.

can be written as an integral of some

nonnegative function f(x)

property (vii)

( )xX

f t dtF x

0 for allP X x x

tch-prob 10

Random variable of mixed type

Ex. 3.6 The waiting time X of a customer in a queueing system is zero if he finds the system idle (p), and an exponentially distributed random length of time if he finds the system busy (prob. 1-p).

0 0

(1 )(1 ) 0X x

x

p p e xF x

[ ] [ | ] [ | ](1 )

XF x P X x

P X x idle p P X x busy p

tch-prob 11

3.3 Probability Density Function (pdf)

The pdf of X, if it exists, is defined as

( ) ( )

( ) ( )

0

XX

X X

X X

X

dF xf x

dx

P x X x h F x h F x

F x h F xh

h

as h P x X x h f x h

density

tch-prob 12

i.

ii.

iii.

iv.

0X

f x

XbP a X b f x dxa

X XxF x f t dt

1X

f t dt

since is nondecreasing( )XF x

pdf completely specifies the behavior of continuous random variables.

tch-prob 13

Ex. 3.7 uniform random variable

1( )

0 and X

a x bf x b a

x a x b

a b

1/(b-a)

tch-prob 14

The derivative of the cdf does not exist at points where the cdf is not

continuous.

To generalize pdf for discrete random variable.

Define delta function

xu x t dt

0 x

1

0 x

0 0

1 0u x

x

x

( )x

tch-prob 15

Ex

X k k

X k k

p u

p

F x x x xxk

f x x x xxk

X k k

p x P X x

X

1/8

fx(x)

0 1 32

1/8

3/83/8

X

7/8

1/2

1

1/8

Fx(x)

tch-prob 16

Conditional cdf’s and pdf’s

The conditional cdf of X given A is

satisfies all the properties of a cdf.

The conditional pdf. of X given A is

0

X

X

P X x AF x A if P A

P A

F x A

X Xdf x A F x Adx

tch-prob 17

Ex.3.10 The lifetime X of a machine has a continuous cdf .

Find the conditional cdf and pdf given A={X>t}.

0

1

1

X

X XX

X

XX

X

F x X t P X x X t

P X x X t

P X t

x t

F x F tF x X t

x tF t

f xx X t x t

F tf

( )XF x

tch-prob 18

3.4 Some important random variables

- Discrete Random Variables

1. Bernoulli r.v.

2. Binomial Random variable

X: number of times a certain event occurs in n independent trials.

0( )

1A

notif in AI

if in A

0 1

1I

I

p p

p p

1, 2, ,

1 0, ,

A A n A

n kk

X I I In

P X k p p for k nk

Indicator function for A

is a r.v. with pmfAI

tch-prob 19

3. Geometric r.v.

M indep. Bernoulli trials until the first success

or M’=M-1 , number of failures before a success

the only discrete r.v. that satisfies the memoryless property:

11 1,2,

kP M k p p k

' 1 1 0,1,2,kP M k P M k p p k

1

for all , 1

1: 1 1

[ , ] [ ] (1 )

[ ] [ ]k

P M k j M j P M k j k

j k jPf P M j p P M k j p

P M k j M jP M k j M j P M k j p

P M j P M j

tch-prob 20

4. Poisson r.v.

counting the number of occurrences of an event in a time period.

average number of event occurrences in a time interval t.

0,1,2,...!

:

kP N k e k

k

1!0!0

ke e e

kk

ke

kk

t

Figure 3.10.

tch-prob 21

Binomial prob. Poisson prob. As

, 0,n p np

11

1

1

. . 1!

1! !1

1 ! 1 ! 11

1

1 1 1 1

1

0,1,2,...1

kn kk

k

n kk

k

n kkk

k k

ni e p p p e

k k

np p

k n k pkpnp k n k p

p pk

kn k p nk p k n

as nk

p p for kk

tch-prob 22

00

1 0

2

2 1

10

1

2 2!

0,1,2,...!

1 , as n

n

k

k

np p p

p p e

p p e

k

e n

pk

n

e

tch-prob 23

- Continuous r.v.

1. uniform r.v.

2. Exponential r.v.

model the time between event occurrences.

1

0 and

0

1

a x bb a

x a x b

x a

x aa x b

b ax b

f xx

F xx

a b X

1---b-a

a b X

1

0 0

0

0 0

1 0

X

X

xf x

xe x

xF x

xe X

: rate at which events occur

fx(x)

x

tch-prob 24

Exponential r.v. is limiting form of the geometric r.v.

- An interval of duration T is divided into subintervals of length

- Perform a Bernoulli trial on each subinterval with prob. of success

- The number of subintervals until the occurrence of a successful event is a geometric r.v. M.

- Thus, the time until the occurrence of the first successful event is X=M (T/n)

0 T 2 3T T T

n n n

Tn

pn

1

1

tT

nP X t P M tT

ntp T

tn Tn

tTe e as n

tch-prob 25

For a Poisson r.v., the time between events is an exponentially

distributed r.v. with parameter events/sec.

Exponential r.v. also has the memoryless propertyT

0

( )

P X t h X tP X t h X t for h

P X t

t hP X t h etP X t e

he P X h

The probability of having to wait at least h additional seconds given that one has already been waiting t seconds = The probability of waiting at least h seconds when one first begin to wait.

tch-prob 26

3. Gaussian (Normal) r.v.

Sum of a large number of small r.v.s

p.d.f

cdf.

change of variable

where

2' 22

21 22

2

1 '2

X

x mx

xx

f x e

P X x e dx

m

2

21

2

tx m

X e dt

x m

F x

'x mt

X2 2m m m m m

x

2

212

tx

e dtx

tch-prob 27

Ex. 3.14. Show that Gaussian pdf integrates to one.

2 2

2

2 22 21 12 2 2

22

12

x y

yx xe dx e dx e dy

e dxdy

cos , sin ,let x r y r 2

2

2

22

0 0

2

0

2

12

10

r

r

r

e rdrd

re dr

e

tch-prob 28

Ex.3.15.

4.753, 9.506 /

6[ 0] 102 2

2v

v m v vP Y P N v Q

v

-6

Output voltage , where is input voltage and is

Gaussian noise with m=0, =2. Find such that P[ <0]=10 .

Y V N V N

V Y

tail of the pdf

21 2Q 12

Q 1 Q

tx x e dtx

x x x

Q-function Table 3.3

It is sometimes convenient to work with Q(x).

tch-prob 29

Q(x)

x Q(x)

0 0.500

1.0 0.159

2.0 2.28E-2

3.0 1.35E-3

4.0 3.17E-5

5.0 2.87E-7

6.0 9.87E-10

k

1 1.2815

2 2.3263

3 3.0902

4 3.7190

5 4.2649

6 4.7535

7 5.1993

)10(1 kQx

tch-prob 30

4. Gamma r.v.

Pdf

where is the gamma function

10 , 0, 0X

xx ef x x

z

0

1 0

12

1 0

1 !

z

z

z xx e dx z

z z z

m m

m non negative integer

11

,1 !X

m xx em f x

m

m-Erlang r.v.

exponential r.v.

Figure 3.14

tch-prob 31

※ : the time until the occurrence of the mth event

Assume the times between events are exponential r.v., (Poisson r.v. limiting case)

Let N(t) be the Poisson r.v. for the number of events in t seconds.

※ iff

m th event occurs before t m or more events occur in t second.

, , ,1 2X X Xm

1 2S X X Xm m

( )S t N t mm

1

0

11

1

1

1!

1 ! !

1 !

m m

kmt

k

k kmt t t

mk

m

t

Fs t P S t P N t m

te

k

t tfs t e e e

k k

te

k

m-Erlang

mS

tch-prob 32

3.5 Functions of a Random Variable

X: r. v. g(x): real-valued function

Y=g(X) is also a r.v.

Event C in Y <=> equivalent event B in X

[ in ] [ ( ) in ] [ in ]P Y C P g X C P X B

( )x g x

{ } { ( ) }Y y g X y

[{ }] [{ ( ) }]P Y y P g X y

tch-prob 33

Ex 3.21. X: # of active speakers in a group of N indep. speakers

p: Prob. that a speaker is active

A Voice transmission system can transmit up to M voice signals at

a time

Y: # of Signals discarded

in {0,1,2,...,M}

0

0 10

1 0

N jj

N M kM k

X

x MY X M

X M M x NM N

P Y P pjj

NP Y k P X M k p p k N M

M k

p

M N

N-M

Y

X

tch-prob 34

Y=aX+b , where a is nonzero.

Suppose X is continuous and has cdf , Find .

0

1 0

X

X

y b y bP X F if aa a

y b y bP X F if aa a

Y

x

y

Y=ax+b

a>0 YF y P Y y P aX b y

pdf.

1 0

1 0

1

X

X

X

y bf adF y a aYf yY dy y bf a

a a

y bfa a

( )YF y( )XF x

tch-prob 35

Ex.3.24 X: Gaussian

2

2

0 0

0

02 2

2 2

YX X

X XY

X X

Y X

P Y y P X y P y X y

yF y

F y F y y

f y f yf y y

y y

f y f yy y

If has n solutions, , then will have n terms.1 2, , ,

nx x x

X

y

0( )

Yf y0 ( )y g x

tch-prob 36

Consider a nonlinear function Y=g(X)

event

Its equivalent event

yC y Y y dy

1 1 1

2 2 2

3 3 3

1 1 2 2 3 3

y

Y

y X X X

B x X x dx

x dx X x

x X x dx

P C f y dyy

P B f x dx f x dx f x dx

yy+dy

x1 x2 x3x3+dx3

dx2 is negative

XY

k

X

k

f xf y

dykx xdx

dxf x dy x xk

Ex.3.27.

tch-prob 37

Ex. 3.28 Y=cos(X) , X: uniformly distributed in

for –1< y <1 , y has two solutions

(0,2 ]

10

1 0

cos

2

x y

x x

10

0

0

2 2

20

2

cos

sin( ) sin cos

si

1

1

n

1,

d

y x

z

yx y

x

y z y

d

y

z

xx

y

0 1cos y

2

tch-prob 38

2 2

2

1

1

21 1

2 1 2 1

11 1

1

0 1

1 sin1 1

21 1

X

Y

Y

f x

f yy y

for yy

y

yF y y

y

tch-prob 39

3.6 The Expected Value of R.V.s

c.d.f. or p.d.f provides complete description of a r.v.

Sometimes interested in a few parameters that summarize the information

Expected value of X or mean of X is defined by

for discrete r.v.

X

k x kk

E X t f t dt

or E X x p x

X

k x kk

E X t f t dt

or E X x p x

The expected value is defined if

tch-prob 40

When the pdf is symmetric about a point m, i.e.,

If

Ex. .Gasussian

uniform

2

E X m

a bE X

( ) ( ), X Xf m x f m x then E X m

0 X

x

t m f t dt

tf t dt m

tch-prob 41

00 0

0

0 0

1

X X X

X

Xtf t dt tF t F t dt t F t d

F dt

t

t

0

0

1 x

k

E X F t dt

or E X P X k

X continuous

X integer-valued

When X is a non-negative r.v.

1

tch-prob 42

Ex. Exponential r.v.

0 0

0

( ) ( ) 1

11

t tx x

tx

t

f t e F t e

E X F t dt e dt

or t e dt

Expected value of Y=g(X)

Y

X

E Y yf y dy

g x f x dx

tch-prob 43

Ex.3.33 constant

uniform r.v. in

cos ,Y a t

(0,2 )

2

0

2 2 2

cos

1cos

221

sin 002

cos

E Y E a t

a t d

a t

E Y E a t

2 2

2 2 22

0

cos(2 2 )2 2

1cos(2 2 )

2 2 2 2

a aE t

a a at d

, ,a t

tch-prob 44

If c is a constant

If

1

1

20 1 2

20 1 2

20 1 2

( )

: ( )

[ ] [ ] [ ] [ ] [ ]

[ ] [ ] [ ]

n

kk

n

kk

nn

nn

nn

Y g X

E Y E g X

Ex Y g X a a X a X a X

E Y E a E a X E a X E a X

a a E X a E X a E X

E X c E X c

[ ] ( ) ( )

[ ] ( ) ( ) [ ]

E c cf x dx c f x dx c

E cX cxf x dx c xf x dx cE X

tch-prob 45

2

22

22

22

2

2

Var X E X E X

E X E X X E X

E X E X E X E X

E X E X

2

2

2

2

2

2

2

2

2 2

1

2

1

2

1

2

x m

x m

x m

e

x m e dx

e

Variance of X

Expected value provides limited info., also want to know variation magnitude

Ex. 3.38. Var. of Gaussian

(x-m)

dx

tch-prob 46

Var [c]=0

Var [X+c]= Var [X]

Var [cX]= 2 [ ]c Var X

2 2

2 2

22 2

0E c E c E c c

E X c E X c E X E X

E cX E cX E c X E X

nth moment of the random variable X

( )n nXE X x f x dx

tch-prob 47

3.7 Markov and Chebyshev Inequalities

Suppose X is a non-negative r.v.,

0[ ] ( ) ( )

( )

( )

a

x xa

xa

xa

E X tf t dt tf t dt

tf t dt

af t dt aP X a

E XP X a

a

Markov inequality

1

a

tch-prob 48

Suppose are known.

Let

and use Markov inequality, we obtain

2 2( )D X m

2[ ] , [ ]E X m Var X

22

2 22 2

22 2

2

2

2

E X mP D a

a a

P X m aa

P X m aa

Chebyshev inequality

tch-prob 49

Ex. 3.42 Suppose

Then the Chebeshev inequality for gives

2[ ] , [ ]E X m Var X

2

2 2 2

1 (=0.25 for 2)P X m k k

k k

Now suppose that we know that X is a Gaussian r.v., then for k=2

a k

2 0.0456P X m

tch-prob 50

3.9 Transform Methods

useful computational aids in the solution of equations that involves derivatives and integrals of functions.

A. Characteristic Function

Expected value of

Fourier Transform of

1.

2

j XX

j

X

xX X

j x

f x

E e

f x e d

e d inve

x

rse FT

j X

X

e

f x

Ex.3.47. Exponential r.v.

check p.101.

0 0

j xx j xX e e dx e dx

j

tch-prob 51

If X is a discrete r.v. .

kj xX X k

k

p x e If X is a discrete integer-valued r.v.,

j kX X

k

p k e

A periodic function of with period 2

2

0

10, 1, 2,...

2j k

X X

X

p k e d k

Fourier series coefficients of

tch-prob 52

If f(x) is a periodic function of period , then f(x) can be represented as

, Fourier series of ( )

1( ) , 0, 1, 2,

2

jkxk

k

jkxk

f x c e f x

c f x e dx k

2

tch-prob 53

Moments of X can be obtained from by X

0

2

2 2

0

0

1

( ). 1 ...

2!

1 ... ...2! !

nn

Xn n

X X

n n

X

nn n

Xn

dE X

j d

j xpf f x j x dx

j E X j E Xj E X

nd

jE Xd

dj E X

d

tch-prob 54

Ex.3.49 exponential

2

3

22 2

222 2 2

'

' 0 1

2''

'' 0 2

2 1 1

X

X

X

X

X

wj

jw

j

E Xj

j

E Xj

Var X E X E X

Check p.101

tch-prob 55

B. Probability Generating Function

For nonnegative r.v.

a. if N is nonnegative integer-valued r.v.

prob. Generating Function of N

0

0

1|

!

NN

kN

k

k

jN

N N

N

zk

G z E z

p k z

dp

G e

k G zk dz

tch-prob 56

11 1

0 0

22

1 120 0

2

2" ' '

|

| 1 1

1

1 1 1

kN z N z N

k k

kN z N z N

k k

N N N

dG z p k kz kp k E N

dz

dG z p k k z k k p k

dz

E N N E N E N

Var N G G G

tch-prob 57

b. if X is a non-negative continuous r.v.

Laplace transform of the pdf

0

01

sx sXX

nnn

sn

X s f x e dx E e

dE X X s

ds

Ex. 3.51. Laplace transform of the gamma pdf

1

0

1

0

1

0

1

xsx

s x

y

x eX s e dx

x e dx

y e dys

s

tch-prob 58

01

0

22

022 2

0

222

1 1

s

s

s

s

dE X

ds s s

dE X

ds s s

Var X E X E X