lecture 13 - university of california, san...

Lecture 13

CentralLimitTheorem

Lecture 13ECE 278 Mathematics for MS Comp Exam

ECE 278 Math for MS Exam- Winter 2019 Lecture 13 1

Lecture 13

CentralLimitTheorem

The Central Limit Theorem

A random variable that describes an event may arise as the sum of manyindependent random variables xi for repeated instances of someunderlying constituent event.If the variances of the random variables xi are finite and equal, then theprobability density function px(x) for the normalized sum

x =1√N

N∑i

(xi − 〈x〉)

as N goes to infinity will usually tend towards a gaussian probabilitydensity function

with mean 〈x〉

Statement is true irrespective of the functional form of the probabilitydensity functions px

i(xi) of the individual constituent events.

When the density functions are all the same, the formal statement iscalled the central limit theorem.The central limit theorem explains why the gaussian probability densityfunction and its variants are ubiquitous in statistical analysis.


Lecture 13

CentralLimitTheorem

The Central Limit Theorem-2

As an example, consider the probability density function of thenormalized sum of N independent and identically distributed (IID)complex random variables Aie

iφi added to a constant Aeiθ,

Random variables Ai and φi are zero-mean, independent, andidentically-distributed random variablesThe probability density function of φ

iis uniform over [0, 2π).

The resulting normalized sum is a complex random variable written as

S =1√N

N∑i=1

(Aie

iφi −Aeiθ)


Lecture 13

CentralLimitTheorem


This can be written asS = x+ iy

where

x =1√N

N∑i=1

(Ai cosφi−A cos θ)

and

y =1√N

N∑k=i

(Ai sinφi−A sin θ)

In the limit as N goes to infinity, asserting the central limit theoremyields a joint probability density function fx,y(x, y) for S that is acircularly-symmetric gaussian probability density function centered on theconstant Aeiθ.


Lecture 13

CentralLimitTheorem


.

Many independentcontributions

Sum of independentrandom vectors

Joint gaussian

pdf

Real Axis (in-phase)Real Axis (in-phase)

Imag

inar

y A

xis

(qua

drat

ure)

Imag

inar

y A

xis

(qua

drat

ure)

φi

Ai

A

θ

A

θ

Figure: The limit of the sum of many independent random vectorssuperimposed on a constant signal is a circularly-symmetric gaussian probabilitydensity function centered on the constant Aeiθ.


Lecture 13

CentralLimitTheorem


Although the central limit theorem is quite powerful, the convergence toa gaussian distribution is not complete for any finite number of summandrandom variables.This means that calculations of small probabilities of events by usingthe central limit theorem to validate the use of a gaussian distributionmay not be valid.


Lecture 13

CentralLimitTheorem

The Central Limit Theorem- suggestive proof (not rigourous)

Start with the expression

x =1√N

N∑i=1

(xi − 〈x〉)

and find the characteristic function

Cx(ω) =⟨eiωx⟩ = ⟨exp

[iωN−1/2

N∑i=1

(xi − 〈x〉

]⟩Now sum inside exponential can be converged into a product outside sothat

Cx(ω) =⟨eiωx⟩ = N∏

i=1

⟨exp

iω N−1/2(xi − 〈x〉︸︷︷︸new random variable X

⟩

= CX (N−1/2ω)N

where CX (ω) is the characteristic function of the random variable x− 〈x〉


Lecture 13

CentralLimitTheorem


Now expand the characteristic function CX (ω) in a power series to give

CX (ω) ≈ 1− 12σ

2ω2 + θ(ω)|ω|3 + . . .

where the remainder term is bounded so that θ(ω) < M for some MThe reason there is no linear term in the expansion is that term in theexpansion would be the mean, but we are expanding with the mean sothe highest order term is the quadratic termExpansion may or may not be possible, but can be checked


Lecture 13

CentralLimitTheorem


Now find CX (N−1/2ω)

CX (N−1/2ω) ≈ 1− σ2ω2

2N + θ(N−1/2ω)N−3/2|ω|3 + . . .

and raise to the Nth power(CX (N−1/2ω)

)N≈(

1− σ2ω2

2N + θ(N−1/2ω)N−3/2|ω|3 + . . .

)NAs N goes to infinity, there is only one term that remains(

CX (N−1/2ω))N≈(

1− σ2ω2

2N

)N


Lecture 13

CentralLimitTheorem


Now uselimN→∞

(1 +

x

N

)N= ex

on the first term so that(CX (N−1/2ω)

)N= e−σ

2ω2/2

The characteristic equation is approximated by a gaussianTherefore the pdf is a zero-mean gaussian with variance σ2


Lecture 13

CentralLimitTheorem

The CLT in Practice

Approximate the probability using a gaussian so that

Pr(y1 < y < y2) ≈ erfc(y1σy

)− erfc

(y2σy

)where the random variable y has zero mean and variance σ2

y

The random variable y is the sum of iid random variables

y =

N∑i=1

xi

so that

Pr(s1 <

N∑i=1

xi < s2) ≈ erfc(s1 −N〈x〉√

Nσx

)− erfc

(s2 −N〈x〉√

Nσx

)(1)

where σ2x is the variance of the underlying distribution


Lecture 13

CentralLimitTheorem

Example

A batch of resistor is characterized by a uniform probability distributionbetween 950 Ω and 1050 Ω with a mean of 1000 Ω

What is the probability that 100 of these resistors in series has a totalresistance that is within 0.5% of 100k Ω?SolutionTo apply the CLT, we need the variance of the uniform distribution whichis given by (see first lecture for probability)

σ2x

.= 〈(x− 〈x〉)2〉 = 1

b− a

∫ b

a

(x− 〈x〉)2 =112 (b− a)2 =

1002

12

=1

b− a

∫ b

a

(x− 1

2 (b− a))2

=112 (b− a)2 =

1002

12where a =950 and b = 1050 so that b− a = 100.


Lecture 13

CentralLimitTheorem

Example-2

The lower limit for (1) for 0.5% is

s1 = 0.995N〈x〉

where N = 100 and 〈x〉 = 1000Similarly,

s2 = 1.005N〈x〉

so that (1) in Ω is

erfc(s1 −N〈x〉√

Nσx

)− erfc

(s2 −N〈x〉√

Nσx

)

= erfc(− 0.005(100)

10(0.1/√

12)

)− erfc

(0.005(100)

10(0.1/√

12)

)= 1− 2erfc

(0.005(100)

√12)

= 1− erfc (1.732) ≈ 0.971


lecture 13 - university of california, san...

Documents