loss functions for detecting outliers in panel data charles d. coleman thomas bryan jason e. devine...

Loss Functions for Detecting Outliers in Panel Data

Charles D. ColemanThomas Bryan

Jason E. DevineU.S. Census Bureau

Prepared for the Spring 2000 meetings of the Federal-State Cooperative Program for Population Estimates, Los Angeles, CA,

March, 2000

Panel Data

A.k.a. “longitudinal data.”

xit:

– i indexes cross-sectional units: retain identities over time. Exx: Geographic areas, persons, households, companies, autos.

– t indexes time.– Chronological or nominal.– Chronological time measures time elapsed between two dates.– Nominal time indexes different sets of estimates, can also

index true values.

Notation

• Bi is base value for unit i.

• Fi is “future” value for unit i.

• Fit is future value for unit i at time t.

• Bi, Fi, Fit > 0.

i=|Fi-Bi| is absolute difference for unit i.

• Subscripts will be dropped when not needed.

What is an Outlier?

“[An outlier is] an observation which deviates so much from other observations as to arouse suspicions that it was generated by a different mechanism.”

D.M. Hawkins, Identification of Outliers, 1980, p. 1.

Meaning of an Outlier

• Either– Indication of a problem with the data

generation process.

• Or– A true, but unusual, statement about reality.

Loss Functions• Motivations: The i come from unknown

distributions. Want to compare multiple size classes on same basis.

• L(Fi;Bi)(i,Bi) is loss function for observation i.

• Loss functions measure “badness.”

• Loss functions produce rankings of observations to be examined.

• Loss functions are empirically based, except for one special case in nominal time.

Assumption 1

Loss is symmetric in error:

L(B+; B) = L(B–; B)

Assumption 2

Loss increases in difference:

/ > 0

Assumption 3

Loss decreases in base value:

/B < 0

Property 1

Loss associated with given absolute percentage difference (| / B|) increases in B.

Simplest Loss Function

L(F;B) = |F – B|Bq (1a)

or

(,B) = Bq (1b)

with

0 > q > –1.

~( ; )L F B F B

F B

Br

s

Loss as Weighted Combination of Absolute Difference and

Absolute Percentage Difference

• This generates loss function with q = –s/(r + s).• Infinite number of pairs (r, s) correspond to any given q.

Outlier Criterion

• Outlier declared wheneverL(F;B)(,B) > C

• C is “critical value.”

• C can be determined in advance, or as function of data (e.g., quantile or multiple of scale measure).

Loss Function Variants

• Time-Invariant Loss Function

• Signed Loss Function

• Nominal Time

Time-Invariant Loss Function

• Idea: Compare multiple dates of data on same basis.

• Time need not be round number.

• L(Fit;Bi,t) = |Fit – Bi|Btq

• Property 1 satisfied as long as t < –1/q.

• Thus, useful horizon is limited.

Signed Loss Function• Idea: Account for direction and magnitude of loss.

S(F;B) = (F – B) Bq

• Can use asymmetric critical values and “q”s:– Declare outliers whenever

S+(F;B) = (F – B) Bq+ > C+

or

S–(F;B) = (F – B) Bq– < C–

with C+ –C–, q+ q–.

Nominal Time

• Compare 2 sets of estimates, one set can be actual values, Ai.

• Assumptions:– Unbiased: EBi = EFi = Ai.

– Proportionate variance: Var(Bi) = Var(Fi) = 2Ai.

• q = –1/2.

• Either set of estimates can be used for Bi, Fi.

– Exception: Ai can only be substituted for Bi.

How to Use: No Preexisting Outlier Criteria

• Start with q = – 0.5.– Adjust by increments of 0.1 to get “good”

distribution of outliers.

• Alternative: Start with

q = log(range)/25 – 1, where range is range of data. (Bryan, 1999)– Can adjust.

How to Use: Preexisting Discrete Outlier Criteria

• Start with schedule of critical pairs (j, Bj).

– These pairs (approximately) satisfy equation Bq = C for some q and C. They are the cutoffs between outliers and nonoutliers.

• Run regressionlog j = –q log Bj + K

• Then, C = eK.

Loss Functions and GIS

• Loss functions can be used with GIS to focus analyst’s attention on problem areas.

• Maps compare tax method county population estimates to unconstrained housing unit method estimates.

• q = –0.5 in loss function map.

Persons

0 - 50005000 - 2500025000 - 50000Over 50000No Data

Note: The tax method estimates are the base

Map 1Absolute Differences between the Two Sets of Population EstimatesAbsolute Differences between the Population Estimates

Percent

0 - 55 - 1010 - 20Above 20No Data

Note: The tax method estimates are the base

Map 2Absolute Percent Differences between the Two Sets of Population EstimatesPercent Absolute Differences between the Population Estimates

0 - 10001000 - 20002000 - 4000Above 4000No Data

Loss

Map 3Loss Function Values

Note: The tax method estimates are the base

Loss Function Values

Outliers Classified by Another Variable

• Di is function of 2 successive observations.

• Ri is “reference” variable, used to classify outliers.

• Start with schedule of critical pairs (Dj, Rj).

• Run regressionlog Dj = a + log Rj

• Then, L(D, R) = DRb and C = ea.

What to Do with Negative Data

• From Coleman and Bryan (2000):

L(F,B) = |F–B|(|F|+|B|)q, B 0 or F 0,

0 , B = F = 0.

S(F,B) = (F–B)(|F|+|B|)q, B 0 or F 0,

0 , B = F = 0.

• 0 > q > –1. Suggest q –0.5.

Summary

• Defined panel data.

• Defined outliers.

• Created several types of loss functions to detect outliers in panel data.

• Loss functions are empirical (except for nominal time.)

• Showed several applications, including GIS.

URL for Presentation

http://chuckcoleman.home.dhs.org/fscpela.ppt