1 lecture 15: statistics and their distributions, central limit theorem devore, ch. 5.3-5.5
TRANSCRIPT
1
Lecture 15: Statistics and Their Distributions, Central Limit Theorem
Devore, Ch. 5.3-5.5
Topics
I. The Concept of a “Statistic”
II. Independent, Identically Distributed (iid) Samples
III. Deriving Sampling Distribution of Statistic– By Probability Rules– By Simulation
• Application – Tolerances
IV. Distribution of the Sample Mean / Total
V. Central Limit Theorem
VI. Distribution of a Linear Combination
I. Concepts of a “Statistic”
• Consider taking two samples of size n from the same population distribution.– A: 30.7, 29.4, 31.1 Mean 30.4– B: 28.8, 30.0, 31.1 Mean 29.97
• Which group has the larger mean?
• Propositions
– The uncertainty of individual values xi when sampling from a population distribution implies a r.v.
– This uncertainty further implies that any statistic calculated from the population distribution also varies from sample to sample.
Example: Minitab
• Suppose X ~ Weibull (shape= 2, scale = 5)– E(X) = 4.4311; V(X) = 5.365
• Using Minitab generate samples of 10 and observe differences in mean and variance.
Sample 1 2 3 4 5 6Mean 4.401 5.928 4.229 4.132 3.620 5.761Median 4.360 6.144 4.608 3.857 3.221 6.342Std Dev 2.642 2.062 1.611 2.124 1.678 2.496
Results shown are from Devore, p. 226-227
Point Estimates / Sampling Distributions
• Point Estimate – value for a sample statistic from a particular sample.
• Statistic – rv whose value may be calculated from a sample of data -- lowercase letter indicates the calculated or observed value of the statistic.
S s
• Probability Distribution of a Statistic is known as its Sampling Distribution.
xX
valuecomputed theis where)( xxXP
II. iid Random Samples
• Sampling Distribution depends on several items:– Population Distribution (parameters)– Sample size, n– Method of Sampling (with or without replacement)
• rv’s X1, X2, .. Xn form a random sample of size n if:1. Xi’s are independent rv’s (independent)2. Every Xi has the same probability distribution (identically distributed)
• If satisfy above two conditions we say Xi’s are iid
– sampling with replacement or from infinite population iid– sampling w/o replacement requires sample sizes n much smaller than
population N to assume iid (rule: n/N <= 0.05).
III. Deriving Sampling Distribution of a Statistic
• By Probability Rules– used for simple cases with a few Xi’s– cases where derivation is already done.
• By Simulation (more common!)– typically used when derivation via probability
rules is complicated, or if:– Underlying distribution of interest in unknown
(assumed).
Deriving Via Simple Probability
• Example: Suppose you sell two brands of DVD players for A: $150, and B: $200.
• Sales records indicate the following:– A – 60% of Sales; B: 40% of Sales
• Let X1 – revenue from selling A; X2 revenue from B
• Suppose you take samples of size n=2.– List the possible outcome, p(x1,x2), sample mean and variance.
DVD Example: Sampling Distribution
• What is the relationship between the expected value of X-bar and variance of X-bar and the original statistics?
22)(][
)(][
xx
x
xpxXV
xpxXE
Compute:
E(X) 170 V(X) 600
n=2X1 X2 p(x1,x2) x-bar s2
150 150 0.36 150 0150 200 0.24 175 1250200 150 0.24 175 1250200 200 0.16 200 0
x-bar 150 175 200px-bar(x-bar) 0.36 0.48 0.16 1
s2 0 1250
ps2(s2) 0.52 0.48 1
DVD Example n=3
n=3
X1 X2 X3 p(x1,x2) x-bar s2
150 150 150 0.22 150.00 0.00150 150 200 0.14 166.67 833.33150 200 150 0.14 166.67 833.33150 200 200 0.10 183.33 833.33200 150 150 0.14 166.67 833.33200 150 200 0.10 183.33 833.33200 200 150 0.10 183.33 833.33200 200 200 0.06 200.00 0.00
x-bar 150 166.67 183.33 200px-bar(x-bar) 0.22 0.43 0.29 0.06 1E[X-bar] 170
s2 0 833.33ps2(s2) 0.28 0.72 1
V[X-bar] 200
Now, whatis the relationshipbetween meanand variance of originaldistribution X VersusX-bar?
Deriving Sampling Distributions for Continuous
Variables• Similar to discrete distributions, we can
also derive the sampling distributions of continuous variables.
22)()()(
)()(
XEdxxfXXV
dxxfXXE
x
x
Example: Two Exponential
• Exponential Distribution– f(x) = e- x
– E(X) = 1/ V(X) = 1/
• Suppose you have two independent rv’s, each following an exponential distribution and you are interested in the sum of the two rv’s (n=2).
• It can be shown that:
)2/(1)( ;/1)( 2 XVXE
Practical Applications
• For many well-known distributions, the sampling distributions of their primary statistics (mean, variance) have already been determined.
• In those cases where the sampling distribution is unknown or complicated, a very useful alternative is simulation.
Simulation Experiment
• To perform a simulation, you need:
– statistic of interest (e.g., X-bar, S, median, ..)
– population distribution (e.g., normal, uniform, ..)
– sample size n (e.g., n=10, n=100)
– number of k replications (e.g., k=500)
Simulation #1: Range Vs. S
• Conduct an experiment to determine the relationship between Range and S for n=2, n=5, and n=100.– Assume X ~ N(0,12)
Simulation #2: Jointly Distributed Variables with
Tolerance Stack-Up• Develop tolerances for the mean +/- 4 for
the volume of an engine cylinder whose:– bore ~ N(81 mm, 0.252 mm) and– stroke ~ N(83.5 mm, 0.202 mm)
• What is the volume equation?
IV. Distribution of Sample Mean/ Total
• Proposition -– Let X1, X2, .. Xn random sample from a distribution with mean
value and std deviation of , then:
– Let Total, To = X1 + X2 + .. Xn , then:
– Note difference between average and summing rv’s.
nXV
XE
X
X
/)(
)(
22
2)( )( nTVnTE oo
Sample Problem: Using the Avg or Sum of rv’s
• Let Y = # Parking Tickets issued on any given weekday.– Suppose Y has Poisson distribution with = 50.– Assuming you may approximate with normal,
a) What are the mean and variance of the avg # tickets per 5-day week?
b) What are the mean and variance of the sum of tickets per 5-day week?
c) What is the probability that the average # tickets per 5-day week is less than 48?
d) What is the probability that the total # tickets per 5-day week is between 225 and 275?
V. Central Limit Theorem (CLT)
• Theorem
– Let X1, X2, .. Xn be a random sample from a distribution with mean value and variance , and if n is sufficiently large, then
n
X
X
X
/
:with ondistributi normal aely approximat has
22
Rule of Thumb: n > 30.
But can be much less!
Understanding the CLT
• Using Minitab, let us generate 100 groups of service times (4 samples per group) from an exponential distribution with mean = 20 min.– Describe what is happening to the distribution?
150100500
100
50
0
exp-all
Fre
qu
ency
Histogram - 400 times
80706050403020100
20
10
0
Avg1-4
Fre
qu
ency
Histogram - 100 Group Avgs
Increasing sample size
• What is happening to the distribution of the sample averages? (Note: underlying distribution - exponential)
3532292623201714118
20
10
0
avg-e20
Fre
qu
ency
Average Multiple Distributions
• Suppose you have samples from 3 different distributions (e.g., exp, weibull, and uniform).– Minitab results from exponential ( = 20), weibull
(shape = 2, scale = 12) and uniform (20, 80).
10896847260483624120
100
50
0
stack4-6
Fre
qu
ency
55504540353025201510
25
20
15
10
5
0
Avg4-6
Fre
qu
ency
ALL 300 Observations Sample Averages
Summarizing the CLT• Regardless of the underlying distribution, averaging produces a
distribution which is more bell-shaped than before.
• Usefulness of CLT– If n becomes sufficiently large and we wish to compute a
probability of the sample mean, we may approximate with a normal.
– CLT provides analytical robustness!
• Issue of how robust depends on n and the underlying distribution -- the closer the underlying distribution resembles a normal (bell-shape) the smaller the n that is needed.
Other Applications
• Bernoulli Trials (Binomial Distribution)– Let a sample n consist of Xi Bernoulli trials (where each trial
equals 0 for failure, 1 for success).– As n (# of trials) becomes large and both:
• np > 10 and nq > 10 then the distribution of the sample mean (np) will become normally distributed.
– Consider the following example:• 10K bernoulli trials, if you group them in samples of size 100,
what will be the distribution of the groups?
Bernoulli Trial Example
• What does this experiment show about the importance of sample size, particularly for binary attributes?
0.330.310.290.270.250.230.210.190.170.15
20
10
0
bern-avg
Fre
qu
ency
Rules of Thumb with CLT
• How large a sample size do you need to invoke the CLT?
– Uniform n >= 4– Symmetric Triangular n >= 3– Normal n >= 1– Unimodal with extreme n >= 30
• (e.g., exponential)
– Discrete - apply normal approx rules• Binomial ~ np >= 10 for p < 0.5 (Or, np >= and nq >= 10)• Poisson ~ >= 15
VI. Distribution of Linear Combination (Independent
Xi’s)• Let X1, X2, .. Xn be a collection of random
variables with constraints a1, a2, .. an then,
• Linear Combination Y =
• If X1, X2, .. Xn are independent:
XnaXaXaY n...2211
)()...()()( 2211 XnEaXEaXEaYE n
)()...()()( 22
221
21 XnVaXVaXVaYV n
Differences Between Variables
• If Y = X1 - X2,
– E(X1 - X2) = a1E(X1) - a2E(X2)
– V(X1 - X2) = a12V(X1) + a2
2V(X2)
• Regardless of whether Xi are added or subtracted, the variances are additive!
Linear Combination: Tolerance
• Suppose you need to slide tube A into tube B.• What is the linear combination of assembly
clearance if tube A is N(24.8, 0.052) and tube B N(25, 0.052)?
• Assume the tube measurements are independent.
Tube A Tube BTube A Tube B
Weighted Linear Combination: Tolerance
• Suppose you are welding two pieces of metal together: a thick piece and a thin piece.– Let Xthin be the position of the thin piece.– Let Xthick be the position of the thick piece.
• From experience, you find the final position is based on the following:– Yassembly = 0.2 Xthin + 0.8 Xthick
– What is the expected variance of the assembly if the standard deviation thin piece is 0.4 mm, and the standard deviation of the thick piece is 0.15 mm? (assume the measurements of each piece is independent)