lecture 14 spring13

7/30/2019 Lecture 14 Spring13

1/39

Elec2600 Lecture 14 1

Elec2600: Lecture 14

Pairs of continuous random variables

Review of 2D functions, differentiation and integration

Joint cumulative distribution function

Joint density function


2/39


Two Random Variables

One random variable can be considered as a mapping from the

sample space to the real line.

Two random variables can be considered as a mapping from the

sample space to the plane.

Note that given the outcome of the experiment, bothXand Y

are determined simultaneously.


3/39




Review of 2D functions, differentiation and integration (MATH)




4/39

Review: 2D Functions

A real-valued 2D function,z =f(x,y), assigns a real value to each point on the

(x,y) plane.

Think ofz as being the height of the ground and regardx andy as your

east/west and north/south coordinates. Example:


x

y

contour plot

-5 0 5-5

0

5

2 2z x y lines of constant height


5/39



6/39

Differentiating 2D functions

The derivative of a 2D function with respect tox indicates how the function

changes as you move from west to east. It is calculated using the normal

rules of differentiation, assuming thaty is constant.

The derivative of a 2D function with respect toy indicates how the functionchanges as you move from south to north. It is calculated using the normal

rules of differentiation, assuming thatx is constant.


x

y

derivative with respect to y

-5 0 5-5

0

5

x

y

derivative with respect to x

-5 0 5-5

0

5

x

y

contour plot

-5 0 5-5

0

5

2 2z x y 2ddx z x 2

d

dyz y

0ddx

z 0ddx

z


7/39

Multiple Differentiations

Multiple differentiations are performed in sequence.

The sequence of the differentiation does not matter.

Example:


2 2 4 2( ) 2z x y x x y y

4 22

0 2 2

4

d d d d z x x y y

dx dy dx dy

d

x ydx

x

4 2

3

2

4 4 0

4

d d d d z x x y y

dy dx dy dx

d

x xydy

x


8/39


And, of course.


9/39

2D Integration

Recall that the integral of a 1D function gives us the area under the curve

between two endpoints.

The integral of a 2D function gives us the volume under the surface within a

region. In order to evaluate a 2D integral numerically, we need two things:

The function to be integratedf(x,y).

A region in the 2D plane over which the function will be integrated. The region

determines the limits of the integration.

A 2D integral is evaluated as a sequence of two 1D integrals.

For each 1D integral (e.g. overx), the other variable (e.g.y) is assumed to be

constant.

The order of integration can usually be switched. However, the limits of the

integral may change.



10/39

Example

Integrate the functionx2y over the regionR.

Solution:

x first, theny: y first, thenx:


x

y

2

1

R

2 1 2 1

2 2

0 0 0 0

12 3

0 0

2

0

22 2

0 0

3

1 03

1 1 2

3 3 2 3

x ydxdy y x dx dy

xy dy

y dy

yydy

1 2 1 2

2 2

0 0 0 021 2

0 0

1

2

0

11 32

0 0

2

4 02

22 2

3 3

x ydydx x ydy dx

yy dx

x dx

xx dx


11/39

Example

Integrate the function c (a constant)

over the shaded region.


y

h

b x

2

2

hb

b

h

y x

x y

2

2

( )hb

bh

y b x

x b y


12/39

Example (x first, then y)



Solution:


y

h

b x

2

2

hb

b

h

y x

x y

2

2

( )hb

bh

y b x

x b y

2 2

2 2

2

2

0 0

0 0

2

0

21

2

1

2

2

b bh h

b bh h

bh

b

h

h b y h b y

y y

h hb y b

hy

h

cdxdy c dx dy

c x dy c b y dy

b yc by

h

b hc bh c bh

h

Note:

The 2D integral

of a constant overa region is equal to

the constant times the

area of the region.


13/39

Example (y first, then x)



Solution:


y

h

b x

2

2

hb

b

h

y x

x y

2

2

( )hb

bh

y b x

x b y

2 2 2 22 2

2 2

2 22

2

2

2

2 22

2

2

( ) ( )

0 0 0 0 0 0

( )

0 00

2 2

0

2 22 20

2

8

1 1

( )

b h h b h hb b b b

b b

b h hb b

b

b

b

b

b

x b b x x b b x

bx b x

bh h

b b

b

h x h xb b

h b

b

cdydx cdydx c dy dx c dy dx

c y dx c y dx

c xdx c b x dx

c c bx

c

2 2 222

2 2 8

1

2

h b b b

bc b

c bh


14/39








15/39


J oint Cumulative Distribution Function

Thejoint cumulative distribution function of two random variablesXand Y,

F(x,y) is defined as

The subscript indicates the variables and the order they are referred to in the

function.

yxyYxXPyxF YX ,where),(,

X

Y

(x,y)

x

y


16/39


Properties of the J oint CDF

The joint cdf is continuous from the north and east.

, 1 1 , 2 2 1 2 1 2, , if andX Y X YF x y F x y x x y y

X

Y

(x1,y1)

(x2,y2)

,

,

,

,

( , ) 0

( , ) 0

( , ) 0

( , ) 1

X Y

X Y

X Y

X Y

F

F y

F x

F

, ,

, ,

lim ( , ) ( , )

lim ( , ) ( , )

X Y X Y

x a

X Y X Yy b

F x y F a y

F x y F x b


17/39



1 2 , 2 , 1

1 2 , 2 , 1

{ } { } , ,

{ } { } , ,

X Y X Y

X Y X Y

P x X x Y y F x y F x y

P X x y Y y F x y F x y

1 2 1 2 , 2 2

, 2 1 , 1 2

, 1 1

{ } { } ,

, ,

,

X Y

X Y X Y

X Y

P x X x y Y y F x y

F x y F x y

F x y


18/39



Thus, the marginal statistics can be derived from the joint

(but not in general vice versa).

,

,

( ) [ ] [ , ] ,

( ) ,

X X Y

Y X Y

F x P X x P X x Y F x

F y F y


19/39



20/39


Product Form Events

Aproduct form eventis an event that is the intersection of one-dimensional

events (events that involve only one random variable).

Product form events have a rectangular shape, and therefore can be computed

easily using the cumulative distribution function

Examples:

1 2{ } { }X A Y A

x

y(x1,y2) (x2,y2)

x

y

x1 x2

y2

y1

}{}{ 221 yYxXx }{}{ 2121 yYyxXx


21/39

Example

Compute the probability of the following product form event in terms of the

cumulative distribution function. Assume that the cdf is continuous.

SolutionWe can compute the probability by finding

the probability of the event {y1 < Yy2} and

subtracting the probability of the event

{-x1 Xx1}{y1 < Yy2}


x

y

-x1 x1

y2

y1

1 1 2{| | } { }X x y Y y

1 1 2 1 2

1 1 1 2

, 2 , 1

, 1 2 , 1 1 , 1 2 , 1 1

{| | } { } { }

{ } { }

( , ) ( , )

( , ) ( , ) ( , ) ( , )

X Y X Y

X Y X Y X Y X Y

P X x y Y y P y Y y

P x X x y Y y

F y F y

F x y F x y F x y F x y


22/39

Example 5.11

The joint cdf for a pair of

random variables is given by

Since FX,Y(1,1)=1,

X and Y assume values

less than or equal to 1.


,

0 0 or 0

0 1,0 1( , ) 0 1, 1

1,0 1

1 0, 0

X Y

x y

xy x yF x y x x y

y x y

x y

1 1


23/39


The joint cdf for the vector random variable X=(X, Y) is given by

Find the marginal cdfs.

otherwise0

0,0)1)(1(),(,

yxeeyxF

yx

YX

01),()( ,

xexFxF xYXX

Solution We let one argument go to infinity

01),()(,

yeyFyF yYXY

Example 5.12

Thus, Xand Yindividually have exponential distributions with

parameters and .


24/39


For the cdf in Example 5.12, find the probability of the eventsA = {X< 1, Y< 1},

B = {X>x, Y>y} wherex > 0 andy > 0, and

D = {1


25/39


Approximations by Product Form Events

Many events cannotbe expressed as the union of a finite number of product

form events .

However, they can be approximated arbitrarily wellby rectangles of

infinitesimal width.

If the cdf is sufficiently smooth, probabilities can be expressed in terms of adensity function

x

y(0,10)

(10,0)

y

x

(10,0)

(0,10) 10 YXA 2 2 10B X Y

,j kB,j kA

, , ,[ ] [ ] ( , ) ( , )j k X Y j k X Yj k j k y x

P A P A f x y x y f x y dxdy


26/39








27/39

J ointly continuous random variables

We say that two random variables arejointly continuous if the joint

cumulative distribution function is continuous and differentiable.

We define thejoint probability density function to be the second

derivative of the joint distribution:

The joint probability density function is also referred to as thejoint pdforthejoint density.

Note that the probability density function is not necessarily continuous.

Just as in the case of a single random variable, the probability that a pair ofcontinuous random variables achieves any particular combination of values is

zero:


2

,

,

( , )( , )

X Y

X Y

F x yf x y

x y

0P X x Y y


28/39


Properties of the joint probability density function

The joint pdf is always non-negative:

The joint cdf can be computed by integrating the joint pdf:

The total area under the joint pdf is 1:

The probability of any eventA is the integral of the joint pdf over thecorresponding region in theX-Yplane:

For example,

,[( , ) ] ( , )X YA

P X Y A f x y dxdy

, ,( , ) ( , )y x

X Y X YF x y f d d ,

( , ) 1X Y

f x y dxdy

, ( , ) 0 for all ,X Yf x y x y

2 2

1 1

1 2 1 2 ,[{ } { }] ( , )

y x

X Y

y x

P x X x y Y y f x y dxdy


29/39


Note that the value of the density function is not a probability.

However, it is large in regions of the plane with high probability

and small in regions of the plane with low probability.

It is true thatfX,Y(x,y) 0 for allx andy.

However, it is not necessarily true thatfX,Y(x,y) 1.

The joint pdf is NOT a probability

,

,

( , )

,

y y x x

X Y

y x

X Y

P x X x x y Y y y f d d

f x y x y

X

Y

x

y

x

y


30/39


Marginal densities

The marginal densitiesfX(x) orfY(y) can always be recovered from the joint

densityf(x,y):

However, it is not generally possible to determine the joint density from the

marginal densities.

dxf

ddfdx

d

ddfdx

ddx

xdF

dx

xdFxf

YX

x

YX

x

YX

YXXX

),(

),(

),(

),()()(

,

,

,

,

dyfyf YXY ),()( ,

Y

Xx

y

Integrate

along this

vertical line

forfX(x)

Integrate along

this horizontal line

forfY(y)


31/39

Example 5.15

Find the joint pdf of the pair of random variables with joint cdf given in

Example 5.11

These RVs are said to have a uniform joint pdf in the unit square.


, ,

0 0 or 0 0 0 or 0

0 1,0 1 1 0 1,0 1( , ) ( , )0 1, 1 0 0 1, 1

1,0 1 0 1,0 1

1 0, 0 0 0, 0

d ddx dy

X Y X Y

x y x y

xy x y x yF x y f x yx x y x y

y x y x y

x y x y

1 1 1 1


32/39


Find the constant c and the marginal densities ofXand Y.

elsewhere.0

0),(,

xyeceyxf

yx

YX

Solution: x

y

Example 5.16

Suppose that random variablesXand Yare jointly distributed according to

Integrate alongy first in yellow region, the alongx.

0 0 0 0

0 00

2

0

1

1

112 2

x xx y x y

xx y x x

x x

ce e dydx c e e dy dx

c e e dx c e e dx

cc e e dx c

2c

1y ye dy e

0

1xe dx


33/39


,

2

( ) ( , )

2

2

2 ( ) 2

Y X Y

x y

y

y x

y

y y y

f y f x y dx

e e dx

e e dx

e e e

,

0

0

( ) ( , )

2

2

2 1

X X Y

xx y

xx y

x x

f x f x y dy

e e dy

e e dy

e e

Example 5.16 (cont.)

If 0 ,y

Otherwise, ( ) 0.Yf y

If 0 ,x

Otherwise, ( ) 0.Xf x

x

y y x

integrate along

this line

x

y y xintegrate along

this line


34/39


Find P[X+Y< 1] in Example 5.16

Solution:

The probability is computed by integrating over the triangular region which isthe intersection of the area where the pdf is non-zero (yellow), and the region

of the plane where X + Y 1 (green).

We integrate overx first, theny:

1

5.0

0

)1(

5.0

0

1

5.0

0

1

21

2

2

21

e

dyeee

dydxee

dxdyeeYXP

yyy

y

y

xy

y

y

yx

Example 5.17

X

Y

1

1

0.5


35/39


Example

SupposeXand Yhave the joint density shown.

Find the marginal densities.

Solution:

Integrating out the undesired variables, we obtain

otherwise0

15.0,15.02

5.00,101

, yx

yx

yxfXY

otherwise0

15.05.15.005.0

xx

xfX

otherwise0

101 yyfY


36/39


Example

SupposeXand Yhave the joint density

shown.

Find P[XY].

Solution:

Integrate the pdf in the overlap of the regions

where the pdf is non-zero andXY(red).Since the pdf is constant in each region, the

integral reduces to the area of the region

times the constant

otherwise0

15.0,15.02

5.00,101

, yx

yx

yxfXY

2 21 10.5 1 0.5 22 2P X Y

3

Thus,8

P X Y


37/39


Random variables thatdiffer in type

In problems where it is necessary to deal with random variables ofdifferenttypes (e.g. X is discrete and Y is continuous), it is usually preferable to workwith probabilities that have the form of a mixture of a pmf and a cdf.

rather than the joint CDF.

Events of this form are easier because the discreteness ofXis explicit.

However, we can always compute the joint CDF from events of this form ifnecessary.

21

or

yYykXP

yYkXP


38/39


LetXbe the input to a communication channel and let Ybe the output. Theinput is either 1 volt or +1 volt with equal probability. The output is the input

plus a noise voltageNthat is uniformly distributed in the interval from 2 volts

to +2 volts. Find P[X= +1, Y< 0].

Solution:

]1[]1|0[]0,1[ XPXYPYXPWhen the inputX= +1, the output Yis uniformly distributed in the interval

[-1,3]. Thus,

Alternatively, the noiseNmust be less than or equal to -1.

Therefore,

4

1]1|0[ XYP

According to the definition of the conditional probability:

Example 4.14

8

1

2

1

4

1]0,1[ YXP


39/39

39

Major Points from this Lecture:

Joint CDF of pairs of random variables

Joint PDF of pairs of random variables

Marginal densities (and how to compute them)

Product form events

lecture 14 spring13

Documents