probability theory 2010 conditional distributions conditional probability: conditional probability...
Post on 21-Dec-2015
222 views
TRANSCRIPT
Probability theory 2010
Conditional distributions
Conditional probability:
Conditional probability mass function: Discrete case
Conditional probability mass function: Continuous case
)(
),(
)(
),(|)|( ),(
| xf
yxf
xXP
xXyYPxXyYPxyf
X
YXXY
)(
),()|( ),(
| xf
yxfxyf
X
YXXY
)(
)()|(
BP
BAPBAP
Probability theory 2010
Conditional probability mass functions- examples
Throwing two dice Let Z1 = the number on the first die
Let Z2 = the number on the second die
Set Y = Z1 and X = Z1+Z2
Radioactive decay Let X = the number of atoms decaying within 1 unit of time Let Y = the time of the first decay
?)5|(| yf XY
?)1|(| yf XY
Probability theory 2010
Using conditional probability mass functions to compute joint and marginal densities
Discrete case
Continuous case
)|()(),( |),( xyfxfyxf XYXYX
x
XYXY xyfxfyf )|()()( |
)|()(),( |),( xyfxfyxf XYXYX
dxxyfxfyf XYXY )|()()( |
Probability theory 2010
Using conditional probability mass functions to compute marginal densities - Gibb’s sampler
Suppose that for two random variables X and Y we know
Then
Moreover, the solution to this fixed-point equation can be obtained by successively sampling
)|( )|( || yxfxyf YXXY and
dttftxhdttfdytyfyxf
dydttftyfyxf
dyyfyxfdyyxfxf
XXXYYX
XXYYX
YYXYXX
)(),()()|()|(
)()|()|(
)()|(),()(
||
||
|),(
)|(
)|(
'''1
'''
'1´
'´
jjYj
jjXj
xXyfY
yYxfX
j
j
Probability theory 2010
Conditional expectation
Discrete case
Continuous case
Notation
y
XYy
xyfyxXyYPyxXYE )|(|)|( |
dyxyfyxXYE XY )|()|( |
)()|( xhxXYE )()|( XhXYE
Probability theory 2010
Conditional expectation - rules
...)|( 21 xXYYE
cxXcE )|(
...)|( xXcYE
...)|)(( xXYXgE
...if)()|( YExXYE
Probability theory 2010
Calculation of expected valuesthrough conditioning
Discrete case
Continuous case
General formula
x
Xx
xXYExfxXYExXPYE )|()()|()()(
dxxXYExfYE X )|()()(
)())|(( YEXYEE
Probability theory 2010
Calculation of expected values through conditioning- example
Primary and secondary events
Let N denote the number of primary events Let X1, X2, … denote the number of secondary events for each primary
event Set Y = X1 + X2 + … + XN
Assume that X1, X2, … are i.i.d. and independent of N
?)( YE
Probability theory 2010
Calculation of variances through conditioning
))|(())|(()( XYEVarXYVarEYVar
Variation in theexpected value of Y
induced byvariation in X
Average remainingvariation in Y
after X has been fixed
Probability theory 2010
Variance decomposition in linear regression
0
1
2
3
4
5
6
7
8
0 1 2 3 4 5
x
y
y fitted y-value
j
jj
jjj
j yyyyyy 222 )ˆ()ˆ()(
Probability theory 2010
Proof of the variance decomposition
We shall prove that
It can easily be seen that
))]|(([)())]|([)|(())|(( 2222 XYEEYEXYEXYEEXYVarE
))|(())|(()( XYEVarXYVarEYVar
2222 )]([))]|(([)]|(([))]|(([))|(( YEXYEEXYEEXYEEXYEVar
Probability theory 2010
Regression and prediction
Regression function:
Theorem: The regression function is the best predictor of Y based on X
Proof:
)|()...,,|()...,,( 111 xX YExXxXYExxh nnn
22
22
))()|(())}()|())(|({(2))|((
))()|()|(())((
XdXYEEXdXYEXYEYEXYEYE
XdXYEXYEYEXdYE
Probability theory 2010
Best linear predictor
Theorem: The best linear predictor of Y based on X is
Proof: Differentiate with respect to the parameters of the linear predictor.
)()( xx
yy XXL
Ordinary linear regression
Probability theory 2010
Expected quadratic prediction errorof the best linear predictor
Theorem:
Proof: …….
)1())(( 222 yXLYE
Ordinary linear regression
Probability theory 2010
Martingales
The sequence X1, X2,… is called a martingale if
Example 1: Partial sums of independent variables with mean zero
Example 2: Gambler’s fortune if he doubles the stake as long as he loses and leaves as soon as he wins
1 allfor )...,,|( 11 nXXXXE nnn
Probability theory 2010
Exercises: Chapter II
2.8, 2.11, 2.23, 2.35, 2.37
Use conditional distributions/probabilities to explain why the envelop-rejection method works