moments shiu-sheng chen -...
TRANSCRIPT
![Page 1: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/1.jpg)
StatisticsMoments
Shiu-Sheng Chen
Department of EconomicsNational Taiwan University
Fall 2019
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 1 / 57
![Page 2: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/2.jpg)
Moments
Section 1
Moments
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 2 / 57
![Page 3: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/3.jpg)
Moments
Moments
Moments can help us to summarize the distribution of a randomvariable.
Analogy to: the height, weight, hair color...etc. of a personEssential but Concise
Two important moments:ExpectationVariance
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 3 / 57
![Page 4: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/4.jpg)
Moments
Expectation
Definition (Expectation)Let S = supp(X). The expectation of a random variable X is definedas
E(X) =⎧⎪⎪⎪⎨⎪⎪⎪⎩
∑x∈S x f (x) discrete
∫x∈S x f (x)dx continuous
Expected value; Mean (value)A probability-weighted sum of the possible values.
Expectation is a constant.Conventional notation: E(X) = µ
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 4 / 57
![Page 5: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/5.jpg)
Moments
Example: Fair Price for a Stock
An investor is considering whether or not to invest in a stock forone year.Let Y represent the amount by which the price changes over theyear with the following distribution
y −2 0 1 4
f (y) 0.1 0.4 0.3 0.2
Then the expected earning is
E(Y) = 0.9
That is, “on average, the investor expects to earn 0.9.” ⇐ Whatdoes this mean?
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 5 / 57
![Page 6: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/6.jpg)
Moments
Simulation Results and Interpretation
If you invest in this stock N years. Let Yi denote the price changefor year i, and the average earning is thus ∑
Ni=1 YiN
We can see that ∑Ni=1 YiN is very close to E(Y) = 0.9 when N is
large.
N
Ave
rage
Ear
ning
0 200 400 600 800 1000
−0.
50.
00.
51.
01.
5
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 6 / 57
![Page 7: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/7.jpg)
Moments
Expectation
Theorem (The Rule of Lazy Statistician)Let X be a random variable, and let g(⋅) be a real-value function.Then
E(g(X)) =⎧⎪⎪⎪⎨⎪⎪⎪⎩
∑x g(x) f (x)
∫x g(x) f (x)dx
Example:
X =
⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩
2 with P(X = 2) = 1/31 with P(X = 1) = 1/3−1 with P(X = −1) = 1/3
Consider g(X) = X2, find E(g(X)) = ?Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 7 / 57
![Page 8: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/8.jpg)
Moments
Variance and Standard Deviation
Definition (Variance/SD)The variance of a discrete random variable X is defined by
Var(X) = E [(X − E(X))2] =⎧⎪⎪⎪⎨⎪⎪⎪⎩
∑x(x − E(X))2 f (x)
∫x(x − E(X))2 f (x)dx
It describes how far values lie from the mean.Conventional notation: Var(X) = σ 2
The standard deviation is SD(X) =√Var(X), and denoted by
SD(X) = σ .
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 8 / 57
![Page 9: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/9.jpg)
Moments
Constant as a Random Variable
Given a constant c, then
E(c) = c,
andVar(c) = 0.
Therefore,E(E(X)) = E(X)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 9 / 57
![Page 10: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/10.jpg)
Moments
Some Important Properties
Given constants a and b:
E(aX + b) = aE(X) + b
Var(aX + b) = a2Var(X)
It can shown thatE(X − E(X)) = 0
Var(X) = E(X2) − [E(X)]2
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 10 / 57
![Page 11: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/11.jpg)
Moments
Example: Fair Price for a Stock
DefinitionThe fair price of a stock is defined by a price such that the expectedreturn equals the risk free rate.
Suppose that the stock price is p.The return is
(p + Y) − pp
=Yp
As an alternative, the investor can put the money in the bankwith a 5% interest rate (risk-free).Recall that E(Y) = 0.9. Hence, E (Yp ) = 0.05 shows that p = 18 isthe fair price of the stock.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 11 / 57
![Page 12: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/12.jpg)
Moments
Expectation as the Best Constant Predictor
Consider a constant predictor of X, say c.Mean Square Prediction Error
MSPE = E [(X − c)2]
It can be shown that
E(X) = argminc
E [(X − c)2]
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 12 / 57
![Page 13: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/13.jpg)
Moments
More on Expectation
In general, unless g(⋅) is linear,
E(g(X)) ≠ g(E(X))
For instance, in the previous example, g(X) = X2,
E(X2) = 2 ≠ 49= [E(X)]2
One more example,E ( 1
X) ≠
1E(X)
Proof: by Jensen’s Inequality
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 13 / 57
![Page 14: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/14.jpg)
Moments
Jensen’s Inequality
TheoremIf X is a random variable and g(⋅) is a convex function, then
E(g(X)) ≥ g(E(X))
Proof.Since g(⋅) is a convex function, there exist some constants a and bsuch that g(X) ≥ aX + b, and g(E(X)) = aE(X) + b.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 14 / 57
![Page 15: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/15.jpg)
Moments
Standardized Random Variables
As expectation and variance are the two most importantmoments, sometimes we will denote the random variable as
X ∼ (E(X),Var(X)) or X ∼ (µ, σ 2)
Definition (Standardized Random Variables)Given X ∼ (µ, σ 2), and let
Z = X − µσ
Then Z ∼ (0, 1) is called a standardized random variable.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 15 / 57
![Page 16: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/16.jpg)
Moments
Moments
k-th MomentsE(Xk)
k-th Central Moments
E[(X − E(X))k]
k-th Standardized Moments
γk = E⎛⎜⎝
⎡⎢⎢⎢⎢⎣
X − E(X)√Var(X)
⎤⎥⎥⎥⎥⎦
k⎞⎟⎠
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 16 / 57
![Page 17: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/17.jpg)
Moments
Expectation
E(X) = 0 vs. E(X) = 5 (Var(X) = 1)
−10 −5 0 5 10
0.0
0.1
0.2
0.3
0.4
x
f(x)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 17 / 57
![Page 18: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/18.jpg)
Moments
Variance
Var(X) = 1 vs. Var(X) = 9 (E(X) = 0)
−10 −5 0 5 10
0.0
0.1
0.2
0.3
0.4
x
f(x)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 18 / 57
![Page 19: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/19.jpg)
Moments
Skewness
3rd standardized moment (Skewness): γ3 = E [( X−E(X)√Var(X))
3]
γ3 > 0
0 5 10 15 20
0.00
0.05
0.10
0.15
0.20
x
f(x)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 19 / 57
![Page 20: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/20.jpg)
Moments
Kurtosis
4th standardized moment (Kurtosis): γ4 = E [( X−E(X)√Var(X))
4]
Excess Kurtosis = γ4 − 3Fat tail: Excess Kurtosis > 0
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
x
f(x)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 20 / 57
![Page 21: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/21.jpg)
Moment Generating Functions
Section 2
Moment Generating Functions
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 21 / 57
![Page 22: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/22.jpg)
Moment Generating Functions
Moment Generating Functions
Definition (MGF)Let X be a discrete (continuous) random variable, and the pmf (pdf)is f (x). Given h > 0 and for all −h < t < h, if the following function
MX(t) = E(e tX)
exists and is finite, it is called the moment generating function(MGF) of the random variable X.
One use the MGFs is that, in fact, it can generate moments of arandom variable.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 22 / 57
![Page 23: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/23.jpg)
Moment Generating Functions
Properties
Theorem (Moment Generating)
E(Xk) = M(k)X (0) = M(k)X (t)∣t=0,
where M(k)X (t) denotes the k-th derivative of MX(t).
Proof: expand e tX as
e tX = 1 + tX + (tX)2
2!+(tX)33!+(tX)44!+⋯
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 23 / 57
![Page 24: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/24.jpg)
Moment Generating Functions
Properties
Theorem (Uniqueness)For all t ∈ (−h, h), if MX(t) = MY(t), then X and Y has exactly thesame distribution, FX(c) = FY(c) for all c ∈ R.
Proof: Beyond the scope of this course (via so-called the inverseFourier transform).
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 24 / 57
![Page 25: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/25.jpg)
Moment Generating Functions
Properties
Theorem (MGF of Linear Transformations)Given the MGF of X is MX(t). Let Y = aX + b, then
MY(t) = ebtMX(at).
Proof: By definition.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 25 / 57
![Page 26: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/26.jpg)
Moment Generating Functions
Examples
Find the MGFs of the following random variables:X ∼ Bernoulli(p)X ∼ Binomial(n, p)X ∼ Uniform[l , h]
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 26 / 57
![Page 27: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/27.jpg)
Covariance and Correlation
Section 3
Covariance and Correlation
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 27 / 57
![Page 28: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/28.jpg)
Covariance and Correlation
Expected Values of Functions of Bivariate Random Variables
DefinitionLet X and Y be discrete (continuous) random variables with jointpmf (pdf) fXY(x , y). Let g(X ,Y) be a function of these two randomvariables, then:
E[g(X ,Y)] =∑x∑yg(x , y) fXY(x , y)
E[g(X ,Y)] = ∫x∫
yg(x , y) fXY(x , y)dydx
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 28 / 57
![Page 29: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/29.jpg)
Covariance and Correlation
Covariance
Definition (Covariance)
Cov(X ,Y) = E([X − E(X)][Y − E(Y)])
It is typically denoted by σXY .A measurement of comovement among two random variables.
x − E(X) y − E(Y) Cov(X ,Y)+ + +− − ++ − −− + −
It can be shown that
Cov(X ,Y) = E(XY) − E(X)E(Y)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 29 / 57
![Page 30: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/30.jpg)
Covariance and Correlation
Properties
TheoremGiven constants a, b, c, and d
E(aX + bY) = aE(X) + bE(Y)
Cov(X , X) = Var(X)
Cov(X , c) = 0
Var(aX + bY) = a2Var(X) + b2Var(Y) + 2abCov(X ,Y)
Cov(X ,Y) = E(XY) − E(X)E(Y)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 30 / 57
![Page 31: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/31.jpg)
Covariance and Correlation
Correlation Coefficient
Definition (Correlation Coefficient)The correlation coefficient is defined by
ρXY = Corr(X ,Y) =Cov(X ,Y)
√Var(X)
√Var(Y)
A unit-free measure of comovement.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 31 / 57
![Page 32: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/32.jpg)
Covariance and Correlation
Correlation Coefficient
TheoremThe correlation coefficient lies between 1 and -1:
−1 ≤ ρXY ≤ 1
Proof: by Cauchy-Schwarz inequality,
[E(UV)]2 ≤ E(U2)E(V 2)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 32 / 57
![Page 33: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/33.jpg)
Covariance and Correlation
Correlation Coefficient
Note:ρXY = 1 (perfect correlation)ρXY = −1 (perfect negative correlation)ρXY = 0 (zero correlation, no correlation, uncorrelated)
However, no correlation does not mean that there is norelationship between X and YIt just suggests that there is no linear relationship between X andY
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 33 / 57
![Page 34: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/34.jpg)
Independent Bivariate Random Variables
Section 4
Independent Bivariate Random Variables
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 34 / 57
![Page 35: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/35.jpg)
Independent Bivariate Random Variables
Expectation of Functions of Independent Bivariate Random Variables
TheoremLet X and Y are independent variables. Then
E[g(X)h(Y)] = E[g(X)]E[h(Y)]
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 35 / 57
![Page 36: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/36.jpg)
Independent Bivariate Random Variables
Independent Bivariate Random Variables and MGF
TheoremX and Y are independent random variables. Their MGFs are MX(t)and MY(t), respectively. Let Z = X + Y , then
MZ(t) = MX(t)MY(t)
Proof. By definition and the previous theorem.Example: Revisit the MGF of a Binomial(n,p) random variable
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 36 / 57
![Page 37: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/37.jpg)
Independent Bivariate Random Variables
Independent Bivariate Random Variables
TheoremGiven that X and Y are independent:
E(XY) = E(X)E(Y)
Cov(X ,Y) = 0
Var(X + Y) = Var(X) + Var(Y)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 37 / 57
![Page 38: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/38.jpg)
Independent Bivariate Random Variables
Independent vs. Uncorrelated
X, Y independent implies X, Y uncorrelated, however, the reverseis not true.
Independence require all possible realizations x and y to satisfy
P(X = x ,Y = y) = P(X = x)P(Y = y).
To check X, Y uncorrelated, only one equation needs to hold:
∑x∑y(x − E(X))(y − E(Y))P(X = x ,Y = y) = 0.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 38 / 57
![Page 39: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/39.jpg)
Independent Bivariate Random Variables
Example
Consider random variable X has the following distribution:
x P(X = x)
-1 1/30 1/31 1/3
Now let Y = X2
It can be shown that Cov(X ,Y) = 0 but clearly they are notindependent.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 39 / 57
![Page 40: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/40.jpg)
Conditional Expectation and Variance
Section 5
Conditional Expectation and Variance
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 40 / 57
![Page 41: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/41.jpg)
Conditional Expectation and Variance
Conditional Expectation
DefinitionThe conditional expectation of Y given X = x is
E(Y ∣X = x) =∑yy fY ∣X=x(y)
E(Y ∣X = x) = ∫yy fY ∣X=x(y)dy
Hence, E(Y ∣X = x) = g(x)Since E(Y ∣X = x) is a function of x, it follows that
E(Y ∣X) = g(X)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 41 / 57
![Page 42: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/42.jpg)
Conditional Expectation and Variance
Conditional Variance
DefinitionThe conditional variance of Y given X = x is
Var(Y ∣X = x) = E([Y − E(Y ∣X = x)]2∣X = x)
Hence, Var(Y ∣X = x) = h(x)Since Var(Y ∣X = x) is also a function of x, it follows that
Var(Y ∣X) = h(X)
It can be shown that (will be shown later)
Var(Y ∣X = x) = E(Y 2∣X = x) − [E(Y ∣X = x)]2
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 42 / 57
![Page 43: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/43.jpg)
Conditional Expectation and Variance
Example
Given two continuous random variables X and Y with joint pdf
fXY(x , y) =32,
supp(Y) = {y∣x2 < y < 1}, supp(X) = {x∣0 < x < 1}
Find fY ∣X=x(y), E(Y ∣X = x), and E(Y ∣X = 1/2).
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 43 / 57
![Page 44: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/44.jpg)
Conditional Expectation and Variance
Important Theorems
TheoremUseful Rule
E[h(X)Y ∣X] = h(X)E[Y ∣X]
Simple Law of Iterated Expectation
E(E[Y ∣X]) = E(Y)
E(E[XY ∣X]) = E(XY)
Application:
Var(Y ∣X) = E(Y 2∣X) − [E(Y ∣X)]2
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 44 / 57
![Page 45: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/45.jpg)
Conditional Expectation and Variance
Example
Given two continuous random variables X and Y with joint pdf
fXY(x , y) =32,
supp(Y) = {y∣x2 < y < 1}, supp(X) = {x∣0 < x < 1}
Find E(Y 2∣X = x), Var(Y ∣X = x), and Var(Y ∣X = 1/2).
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 45 / 57
![Page 46: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/46.jpg)
Conditional Expectation and Variance
Important Theorems
Theorem (Variance Decomposition)
Var(Y) = Var(E(Y ∣X)) + E(Var(Y ∣X))
Example:X ∼ Bernoulli(P)
whereP ∼ Uniform[0,1]
Find E(X) and Var(X).
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 46 / 57
![Page 47: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/47.jpg)
Conditional Expectation and Variance
Important Theorems
Theorem (Best Conditional Predictor)Conditional expectation E(Y ∣X) is the best conditional predictor of Yin the sense of minimizing the conditional mean squared error:
E(Y ∣X) = argming(X)
E[(Y − g(X))2]
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 47 / 57
![Page 48: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/48.jpg)
Conditional Expectation and Variance
Example: GPA vs. Study Hours
Let Y = GPA, X = Study HoursWe would like to know E(Y ∣X) (to forecast Y)We further assume that E(Y ∣X) is a linear function:
E(Y ∣X) = α + βX
It can be shown that
β = E(XY) − E(X)E(Y)E(X2) − E(X)2
=Cov(X ,Y)Var(X)
α = E(Y) − βE(X)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 48 / 57
![Page 49: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/49.jpg)
Conditional Expectation and Variance
Example: GPA vs. Study Hours
We can define the forecast error as
є ≡ Y − E(Y ∣X) = Y − (α + βX)
Hence,Y = α + βX + є
Interpretation: your GPA is determined by(a) Systematic Part: α + βX, which can be explained by study hours(b) Irregular Part: є, which captures other factors other than study
hours. For instance, good/bad luck, mood, illness, etc.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 49 / 57
![Page 50: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/50.jpg)
Multivariate Random Variables
Section 6
Multivariate Random Variables
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 50 / 57
![Page 51: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/51.jpg)
Multivariate Random Variables
Expected Values of Functions of Random Variables
For discrete random variables, the expected values ofg(X1, X2, . . . , Xn) is given by
E[g(X1, X2, . . . , Xn)]
=∑x1⋯∑
xng(x1, x2, . . . , xn) fX(x1, x2, . . . , xn)
For continuous random variables,
E[g(X1, X2, . . . , Xn)]
= ∫x1⋯∫
xng(x1, . . . , xn) fX(x1, . . . , xn)dxn⋯dx1
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 51 / 57
![Page 52: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/52.jpg)
Multivariate Random Variables
Properties
E (n∑i=1
Xi) =n∑i=1
E(Xi)
Var (n∑i=1
Xi) =n∑i=1
Var(Xi) + 2n∑i=1
i−1∑j=1
Cov(Xi , X j)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 52 / 57
![Page 53: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/53.jpg)
Multivariate Random Variables
Expectation of Functions of Independent Random Variables
TheoremLet X1, X2, . . . , Xn are independent variables. Then
E[h(X1)h(X2)⋯h(Xn)] = E[h(X1)]E[h(X2)]⋯E[h(Xn)]
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 53 / 57
![Page 54: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/54.jpg)
Multivariate Random Variables
Independent Random Variables and MGF
TheoremX1, X2,. . . , Xn are independent with MGF: MX1(t), MX2(t),. . .,MXn(t). Let Y = ∑n
i=1 Xi, then
MY(t) = MX1(t)MX2(t)⋯MXn(t) =n∏i=1
MX i(t)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 54 / 57
![Page 55: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/55.jpg)
Multivariate Random Variables
IID Random Variables
Given that {Xi}ni=1 are i.i.d. random variables.
Clearly,E(X1) = E(X2) = ⋯ = E(Xn)
Var(X1) = Var(X2) = ⋯ = Var(Xn)
Cov(Xi , X j) = 0 for any i ≠ j
I.I.D. random variables with mean µ and variance σ 2 are denotedby
{Xi}ni=1 ∼
i .i .d . (µ, σ 2)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 55 / 57
![Page 56: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/56.jpg)
Multivariate Random Variables
Properties of i.i.d. Random Variables
TheoremLet {Xi}
ni=1 are i.i.d. random variables with E(Xi) = µ and
Var(Xi) = σ 2, and let
Y =n∑i=1
Xi .
ThenE(Y) = nµ,
Var(Y) = nσ 2.
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 56 / 57
![Page 57: Moments Shiu-Sheng Chen - 國立臺灣大學homepage.ntu.edu.tw/~sschen/Teaching/Lecture4_Moments_2019.pdf · Moments Shiu-Sheng Chen Department of Economics National Taiwan University](https://reader033.vdocuments.net/reader033/viewer/2022052720/5f0959307e708231d4266816/html5/thumbnails/57.jpg)
Multivariate Random Variables
Example
Let{Xi}
ni=1 ∼
i .i .d . Bernoulli(p)
That is,E(Xi) = p and Var(Xi) = p(1 − p)
LetY =
n∑i=1
Xi
What is the distribution of Y?Find E(Y) and Var(Y)
Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 57 / 57