1 the asymptotic properties of estimators are their properties as the number of observations in a...

1

The asymptotic properties of estimators are their properties as the number of observations in a sample becomes very large and tends to infinity. We shall be concerned with the concepts of probability limits and consistency, and the central limit theorem.

ASYMPTOTIC PROPERTIES OF ESTIMATORS: PLIMS AND CONSISTENCY

Asymptotic properties of estimators:

nProperties as

• probability limits• consistency• central limit theorem

2

These topics are usually given little attention in standard statistics texts, generally without an explanation of why they are relevant and useful. However, asymptotic properties lie at the heart of much econometric analysis. For students of econometrics they are important.



nProperties as


3

This sequence is addressed to probability limits and consistency. A subsequent one will treat the central limit theorem.



nProperties as


4

We will start with an abstract definition of a probability limit and then illustrate it with a simple example.


Probability limits

0lim

nn

ZP

5

A sequence of random variables Zn is said to converge in probability to a constant a if, given any positive e, however small, the probability of Zn deviating from a by an amount greater than e tends to zero as n tends to infinity.


Probability limits

0lim

nn

ZP

6

The constant a is described as the probability limit of the sequence, usually abbreviated as plim.


Probability limits

0lim

nn

ZP

nZ plim


0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

7

We will take as our example the mean of a sample of observations, X, generated from a random variable X with population mean mX and variance s2

X. We will investigate how X behaves as the sample size n becomes large.

X

n 1 50

X

8


0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

For convenience we shall assume that X has a normal distribution, but this does not affect the analysis. If X has a normal distribution with mean mX and variance s2

X, X will have a normal distribution with mean mX and variance s2

X / n.

X

n 1 50

X

9


0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

For the purposes of this example, we will suppose that X has population mean 100 and standard deviation 50, as in the diagram.

X

n 1 50

X

10


0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

The sample mean will have the same population mean as X, but its standard deviation will be 50/ , where n is the number of observations in the sample.n

X

n 1 50

X

11


0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

The larger is the sample, the smaller will be the standard deviation of the sample mean.

X

n 1 50

X

12

If n is equal to 1, the sample consists of a single observation. X is the same as X and its standard deviation is 50.


0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

X

n 1 50

X

13

We will see how the shape of the distribution changes as the sample size is increased. We have added the distribution of X when n = 4.


n = 4n = 1

0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n 1 504 25

X

14

We add the distribution for n = 25. The distribution becomes more concentrated about the population mean.


n = 25

0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 4n = 1

X

n 1 504 25

25 10

X

15

We add the distribution for n = 100. To see what happens for n greater than 100, we will have to change the vertical scale.


n = 100

n = 25

n = 4

0.00

0.02

0.04

0.06

0.08

0.10

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 1

X

n 1 504 25

25 10100 5

X

16

We have increased the vertical scale.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 100

n 1 504 25

25 10100 5

X

17

We increase the sample size to 400. The distribution continues to contract about the population mean.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 100

n = 400

n 1 504 25

25 10100 5400 2.5

X

18

We increase the sample size again. In the limit, the variance of the distribution tends to zero. The distribution collapses to a spike at the true value. The plim of the sample mean is therefore the population mean.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 100

n = 400

n = 1600

n 1 504 25

25 10100 5400 2.5

1600 1.3

X

Formally, the probability of X differing from mX by any finite amount, however small, tends to zero as n becomes large.

19


Probability limits

0lim

nn

ZP

nZ plim

Sample mean as estimator of population mean

0lim

Xn

XP

20


Probability limits

0lim

nn

ZP

nZ plim

Sample mean as estimator of population mean

0lim

Xn

XP

XX plim

Hence we can say plim X = mX.

An estimator of a population characteristic is said to be consistent if it satisfies two conditions. The first is that the estimator possesses a probability limit, and so its distribution collapses to a spike as the sample size becomes large.

21


Two conditions:(1) The estimator possesses a probability limit.

Consistency

22


The second is that the spike is located at the true value of the population characteristic.

Two conditions:(1) The estimator possesses a probability limit.(2) The limit is the true value of the population characteristic.

Consistency

23

The sample mean in our example satisfies both conditions and so it is a consistent estimator of mX.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 1600

24

Most standard estimators in simple applications satisfy the first condition because their variances tend to zero as the sample size becomes large. The only issue then is whether the distribution collapses to a spike at the true value of the population characteristic.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 1600

25

A sufficient condition for consistency is that the estimator should be unbiased and that its variance should tend to zero as n becomes large.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 1600

26

It is easy to see why this is a sufficient condition. If the estimator is unbiased for a finite sample, it must stay unbiased as the sample size becomes large.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 1600

27

Meanwhile, if the variance of its distribution is decreasing, its distribution must collapse to a spike. Since the estimator remains unbiased, this spike must be located at the true value. The sample mean is an example of an estimator that satisfies this sufficient condition.


0.0

0.1

0.2

0.3

0 50 100 150

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

X

n = 1600

28

However the condition is only sufficient, not necessary. It is possible for a biased estimator to be consistent, if the bias vanishes as the sample size becomes large. In this example, the true value is 100, and the estimator is biased for sample size 25.


0.0

0.1

0.2

0.3

80 100 120 140

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 25

29

Here the sample size is greatr and the bias is smaller.


0.0

0.1

0.2

0.3

80 100 120 140

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 100

n = 25

30

A further reduction in the bias with a further increase in the sample size.


0.0

0.1

0.2

0.3

80 100 120 140

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

n = 100

n = 400

n = 25

31

This is an example where the bias disappears altogether as the sample size tends to infinity. Such an estimator is biased for finite samples but nevertheless consistent because its distribution collapses to a spike at the true value.


n = 100

n = 400

n = 1600

n = 25

0.0

0.1

0.2

0.3

80 100 120 140

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

32

We will encounter estimators of this type when we come to Model B, and they will be important to us.


n = 100

n = 400

n = 1600

n = 25

0.0

0.1

0.2

0.3

80 100 120 140

pro

bab

ilit

y d

ensi

ty f

un

ctio

n

The foregoing example was just a general graphical illustration of what might happen as the sample size increases. Here is a simple mathematical example.

33



Consistency

n

iiXn

Z11

1

Example

We are supposing that X is a random variable with unknown population mean mX and that we wish to estimate mX.

34



Consistency

n

iiXn

Z11

1

Example

35


As defined, the estimator Z is biased for finite samples because its expected value is nmX/(n + 1). But as n tends to infinity, n /(n + 1) tends to 1 and the bias disappears.


Consistency

n

iiXn

Z11

1

nnn

ZE XX as 1

Example

36


n

iiXn

Z11

1

nnn

ZE XX as 1

nn

nZ X as 0

1)var( 2

2

The variance of the estimator is given by the expression shown. This tends to zero as n tends to infinity. Thus Z is consistent because its distribution collapses to a spike at the true value.

Example


Consistency

37


In practice we deal with finite samples, not infinite ones. So why should we be interested in whether an estimator is consistent?


Consistency

Why should we be interested in consistency?

38


One reason is that often, in practice, it is impossible to find an estimator that is unbiased for small samples. If you can find one that is at least consistent, that may be better than having no estimate at all.


Consistency

• If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one.


39


A second reason is that often we are unable to say anything at all about the expectation of an estimator. The expected value rules are weak analytical instruments that can be applied in relatively simple contexts.


• It is often not possible to determine the expectation of an estimator, but still possible to evaluate probability limits.

Reason: plim rules are stronger than the rules for handling expectations.



Consistency

40


In particular, the multiplicative rule E{g(X)h(Y)} = E{g(X)} E{h(Y)} applies only when X and Y are independent, and in most situations of interest this will not be the case. By contrast, we have a much more powerful set of rules for plims.


Consistency


• It is often not possible to determine the expectation of an estimator, but still possible to evaluate probability limits.

Reason: plim rules are stronger than the rules for handling expectations.


41


plim rules

Here are six rules for decomposing plims. Note that, in each case, the validity of the rule depends on plims existing for each of the components on the right side of the equation.

1. ZYXZYX plim plim plim plim

2. XbbX plim plim

3. bb plim

4. YXXY plim plim plim

5.

6. XfXf plim plim

0 plim provided plim plim

plim YYX

YX

42


The first three rules are straightforward counterparts for corresponding rules for decomposing expectations. The first rule, the additive rule, depends on X, Y, and Z each having their individual plims.

plim rules


2. XbbX plim plim

3. bb plim

43


The second rule, a multiplicative rule, b being a constant, depends on X having a plim. The third rule states the obvious fact that a constant is its own limit.

plim rules


2. XbbX plim plim

3. bb plim

44


The fourth rule, a multiplicative rule for two (or more) variables, depends on each variable having a plim. It does not require X and Y to be independent.

plim rules


2. XbbX plim plim

3. bb plim

45


By contrast, the corresponding rule for expectations E{g(X)h(Y)} = E{g(X)} E{h(Y)} applies only when X and Y are independent. This is often not the case.

plim rules


2. XbbX plim plim

3. bb plim


46


The quotient rule for plims, which we shall use very frequently in Model B, requires that X and Y both have plims and that plim Y is not zero.

plim rules


2. XbbX plim plim

3. bb plim


47


There is no counterpart for expectations, even if X and Y are independent. If X and Y are independent, E(X/Y) = E(X) E(1/Y), provided that both expectations exist, and that is as far as one can go.

plim rules


2. XbbX plim plim

3. bb plim


5. 0 plim provided plim plim

plim YYX

YX

48


The plim of a function of a variable is equal to the function of the plim of the variable, provided that the variable possesses a plim and provided that the function is continuous at that point.

plim rules


2. XbbX plim plim

3. bb plim


5.

6. XfXf plim plim

0 plim provided plim plim

plim YYX

YX

49

To illustrate how the plim rules can lead us to conclusions when the expected value rules do not, consider this example. Suppose that you know that a variable Y is a constant multiple of another variable Z


Example use of asymptotic analysis

ZY Model:

50

Z is generated randomly from a fixed distribution with population mean mZ and variance s2Z.

a is unknown and we wish to estimate it. We have a sample of n observations.



ZY Model:

51

Y is measured accurately but Z is measured with random error w with population mean zero and constant variance s2

w. Thus in the sample we have observations on X, where X = Z + w, rather than Z.



ZY wZX

Model:

52

One estimator of a (not necessarily the best) is Y/X.


ZY wZX


Model:Estimator of a : X

Y

53

We can decompose the estimator into the true value and an error term, as shown.


ZY wZX



Y

wZw

wZZ

XY

54

We can decompose the estimator into the true value and an error term, as shown.


ZY wZX



Y

wZw

wZZ

XY

55

To investigate whether the estimator is biased or unbiased, we need to take the expectation of the error term. But we cannot do this.


ZY wZX



Y

wZw

wZZ

XY


56

The random variable w appears in both the numerator and the denominator and the expected value rules are too weak to allow us to determine the expectation of a ratio when both the numerator and the denominator are functions of the same random variable.

ZY wZX



Y

wZw

wZZ

XY

ZZ ZZ plim plim ZwZw plim0 plim

wZw

wZZ

XY

57

However, we can show that the error term tends to zero as the sample becomes large. We know that a sample mean tends to a population mean as the sample size tends to infinity, and so plim w = 0 and plim Z = mZ.


ZY wZX



Y

58

Hence plims exist for both the numerator and denominator of the estimator.


ZY wZX



Y


wZw

wZZ

XY

Z

Z

XY

plim


wZw

wZZ

XY

59


Since the plims of the numerator and the denominator of the error term both exist, we are able to take the plim of the estimator.

ZY wZX



Y

60


Z

Z

XY

plim


Thus, provided that mZ ≠ 0, we are able to show that the estimator is consistent, despite the fact that we cannot say anything analytically about its finite sample properties.

0:assumption Z

wZw

wZZ

XY

ZY wZX



Y

Copyright Christopher Dougherty 2012.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be

used as a resource for teaching an econometrics course. There is no need to

refer to the author.

The content of this slideshow comes from Section R.14 of C. Dougherty,

Introduction to Econometrics, fourth edition 2011, Oxford University Press.

Additional (free) resources for both students and instructors may be

downloaded from the OUP Online Resource Centre

http://www.oup.com/uk/orc/bin/9780199567089/.

Individuals studying econometrics on their own who feel that they might benefit

from participation in a formal course should consider the London School of

Economics summer school course

EC212 Introduction to Econometrics

http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx

or the University of London International Programmes distance learning course

EC2020 Elements of Econometrics

www.londoninternational.ac.uk/lse.

2012.11.02

http://www.oup.com/uk/orc/bin/9780199567089/

http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx

1 the asymptotic properties of estimators are their properties as the number of observations in a...

Documents

consistency n

mean x

sample size n

sample mean

probability of z n deviating

concepts of probability

random variable x

sample of observations