5. point and interval estimation - estad.uc3m.es5. point and interval estimation introduction...

5. Point and interval estimation

Introduction

Properties of estimatorsFinite sample sizeAsymptotic properties

Construction methodsMethod of momentsMaximum likelihood estimation

Sampling in normal populations

1

Interval estimationAsymptotic intervalsIntervals for normal populations

2

5. Point and interval estimation

INFERENCIA ESTADÍSTICA

Introduction

3

sample thefrom

about n informatioobtain To :Problem

sample ddistribute

y identicall andt independen ,...,,

parameter unknown ;population ;

21

θ

θθ

nXXX

FX

INFERENCIA ESTADÍSTICA

Point estimation

4

n

n

nn

m

S

X

XXX

median Sample

varianceSample

mean Sample

:Examples

ofestimator ),...,,(ˆ

12

21

−

= θθθ

STATISTICAL INFERENCE

Properties of estimators

5

Unbiased estimator

is an unbiased estimator of if

(bias of )

The bias of an unbiased estimator is zero:

nθ̂ θ θθ =nE ˆ

nθ̂θθθ −= )ˆ()ˆ( nn Eb

0)ˆ(ˆ =⇔ nn bunbiased θθ

6

Efficiency

122

2

1 ˆˆˆˆ

ˆθθθ

θθ

θθVVbecausepreferwe

E

E<

=

=

2

1

ˆ

ˆ

θθ

θ



7

Mean squared error

2θ̂E θ

22 )ˆ(ˆ)ˆ( nnn bVEMSE θθθθ +=−=STATISTICAL INFERENCE


8

Mean squared error

If the estimator is unbiased, then and the best one is chosen in terms

of variance.

The global criterion to select between twoestimators is:

is preferred to if )()( nn SMSETMSE ≤

nVMSE θ̂=

nTnS



Standard error

9

nn Vse θθ ˆ)ˆ( =


10

Properties of estimators when ∞→n

Consistency

is a consistent estimator for parameter ifnθ̂ θθθ →P

n̂


Asymptotic behavior

(weak consistency)

is strongly consistent for ifnθ̂ θθθ →as

n̂

11

Asymptotically normal

is an asymptotically normal estimator with

parameters if

nθ̂

),( nn ba

)1,0(ˆ

Nb

a d

n

nn →−θ


Asymptotic properties

Construction of estimators:method of moments

12STATISTICAL INFERENCE

X with or and we have a sample

The kth moment is

Method of moments:

(i) Equal population moments to sample moments.

(ii) Solve for the parameters.

θp

.,...,1 iidXX n

.kk EX=α

θf

13

Properties:

(i) Consistency

Let be a method of moments estimator of Then

nθ~ .θ

θθ →Pn

~



14

(ii) Asymptotic normality



).( and ),..., ,)',...,,(

,')'(

where

),,0()~

(

11

2 θαθ

θθ

θ

−

∂∂===

=Σ

Σ→−

jjkk

dn

gg(ggXXXY

gYYgE

Nn

Construction of estimators:maximum likelihood

15STATISTICAL INFERENCE

X; i.i.d. sample

The likelihood function is the probabilitydensity function or the probability mass function ofthe sample:

nXX ,...,1

)()...(),...,;(

)()...(),...,;(

11

11

nn

nn

xfxfxxL

xpxpxxL

θθ

θθ

θ

θ

=

=

16

is the maximum likelihood estimator of ifθnθ̂



),...,;(max),...,;ˆ( 11 nnn xxLxxL θθ θ=

The maximum likelihood estimator of is the valueof making the observed sample most likely.

θθ

17

Properties

(i) Consistency

Let be a maximum likelihood estimator of .

Then

(ii) Invariance

If is a maximum likelihood estimator of , then is a maximum likelihood estimatorof

θnθ̂

nθ̂ θ

.ˆ θθ →Pn

)ˆ( ng θ).(θg



18

Properties

(iii) Asymptotic normality

(iv) Asymptotic efficiency

The variance of is minimum.nθ̂

nn VeswithN

esθθθ ˆˆ)1,0(

ˆ

ˆ=→−




19INFERENCIA ESTADÍSTICA

)ˆ(

1ˆ

)(

1

n)informatio

(Fisher ));(());(()(

function) (score );( log

);(

11

nnn

n

ii

n

iin

Ies

Ise

XsVXsVI

XfXs

θθ

θθθ

θθθ θ

≈≈

==

∂∂=

∑∑==

Sampling in normal populations:Fisher’’’’s lemma

20

Let

Given the i. i. d. sample let

Then:(i)

(ii)

(iii) are independent.

).,(~ 2σµNX

,,...,1 nXX

),(2

nn NX σµ≡

21

)(2

2

−− ≡∑

n

XX ni χσ

12, −nSX


.1

)(a

12

12

−−

== ∑∑ −n

XXSndX

nX ni

i

nin

21

distribution

Let independent.

Then

We define

and it verifies

2χ

niNZ i ,...,1)1,0( =≡

.21

2 χ≡iZ

22n

n

iiZY χ≡=∑

.2nVY

nEY

==



22

If the population is normal, the distribution of the estimators is exactly known for any sample size.



Confidence intervals

23

Let , and the sample

Construct an interval with

such that

θFX ≡ . ,...,1 iidXX n

),...,(),...,(

1

1

n

n

XXbbXXaa

==

.1)( αθ −≥<< baP

is the confidence coefficient.α−1


24

Exact interval:

Asymptotic interval:

αθ −=<< 1)( baP

αθ − →<< ∞→ 1)(n

baP


Confidence intervals

Confidence intervals:asymptotic intervals

25

an asymptotically normal estimator of

Then

nθ̂ θ

)1,0(ˆ

ˆ e., i. ),1,0(

ˆ

ˆN

esN

esnn ≈−→− θθθθ

)ˆ

ˆ(1 b

esaP n <−<≈− θθα


26

Define such that

Then

where



2αz

.2

)(2

αα => zZP

),ˆ

ˆ()

ˆ

ˆ(1

22αα

θθθθα zes

zPbes

aP nn <−<−=<−<≈−

).ˆˆˆˆ(122

eszeszP nn αα θθθα +<<−≈−

27

Then, the confidence interval for is

eszI n ˆˆ2

1 αθα ±=−

θ



Statistics

28

29

Remark:

For large samples, we can obtain asymptotic confidence intervals.

For small samples, we can obtain exact confidence intervals if the population is normal.

Interval estimation:Asymptotic intervals


30

i. i. d. sample

(i) Confidence interval for µ with known σ02.

Then

),( 2σµNX ≡ .,...,1 nXX

)1,0(),(/0

20 NNX

n

Xn ≡⇒≡ −

σµσµ


Intervals for normal populations

nzXI 0

21

σα α±=−

31

(ii) Confidence intervals for µ with unknown σ2.

σ2 is unknown: we estimate it.

)1,0(),(/

2

NNXn

Xn ≡⇒≡ −

σµσµ



32

Student t distribution

Let

be independent. Then

2

)1,0(

nYNZχ≡

≡



)1,0(Nt

n

Y

Znn →≡ ∞→

33

Let

Then

.)1()( 2

12

12

2

2

−− ≡−=

−∑n

ni SnXXχ

σσ



11

)1(

)1(

/

12

21

2 −−

−−

−

≡−⇒≡

−−n

nS

n

n

Sn

n

X

tX

tnn

µ

σ

σµ

34

The confidence interval is

thus



)(121

22 ;1;1 αα

µα −− <−<−=−−

n

nS

nt

XtP

n

nS

nntXI 1

2

2;11−

−− ±= αα

35INFERENCIA ESTADÍSTICA

We change from an expression with σ2 and N(0,1)to another expression with S2

n-1 and tn-1

nS

nntXI 1

2

2;11−

−− ±= αα


nzXI 0

21

σα α±=−

Statistics

36

37

(iii) Confidence interval for σ2 with known µ0.

Each satisfies:

and for the whole sample:

iX

21

2

)1,0(

χσ

µσ

µ

≡

−

≡−

oi

oi

X

NX

22

2)(n

oiXχ

σµ

≡−∑



38

and then



))(

(1 22/;2

22

2/1; αα χσ

µχα n

oi

n

XP <

−<=− ∑

−

))()(

(12

2/1;

22

22/;

2

αα χµ

σχ

µα

−

∑∑ −<<

−=−

n

oi

n

oi XXP

39

(iv) Confidence interval for σ2 with unknown µ.

If , then applying Fisher’s Lemma:

The confidence interval is:

),( 2σµNX ≡



212

2)(−≡

−∑n

i XXχ

σ

))()(

(12

2/1;1

22

22/;1

2

αα χσ

χα

−−−

∑∑ −<<

−=−

n

i

n

i XXXXP

5. point and interval estimation - estad.uc3m.es5. point and interval estimation introduction...

Documents