1

Long-Term, Heterogeneous Treatment Long-Term, Heterogeneous Treatment

Effects from Non-Pecuniary Environmental Effects from Non-Pecuniary Environmental

Programs: A Large-Scale Field ExperimentPrograms: A Large-Scale Field Experiment

Paul J. FerraroPaul J. Ferraro

Department of EconomicsDepartment of Economics

Georgia State UniversityGeorgia State University

Juan Jose MirandaJuan Jose Miranda

Department of EconomicsDepartment of Economics

Georgia State UniversityGeorgia State University

2

3

Information CampaignsInformation Campaigns

(1) Info on behavioral and technological

modifications

(2) Request users to voluntarily change

their behavior for the public good

(3) Provide social comparisons to induce

individuals to conform to a social

norm.

4

Field ExperimentField Experiment

• Water system in Metro Atlanta: with

Ferraro, implemented targeted,

residential information campaign as

randomized experiment.

• Three treatments sent via first class

mail in May 2007.

• Ferraro and Price focus on short-term,

mean treatment effects in summer 2007.

5

Ferraro and MirandaFerraro and Miranda

Longer-term impacts (2008, 2009).

Heterogeneous Treatment Effects

• Relevant for policy and practice

• Relevant for understanding

mechanisms

6

Treatment 1: Tip SheetTreatment 1: Tip Sheet

Information on behavioral

changes and technologies that

can reduce water consumption

7

Treatment 2: Weak Social Norms Treatment 2: Weak Social Norms (standard message)(standard message)

Tip sheet+ Civic Duty Language

8

Treatment 3: Strong Social NormsTreatment 3: Strong Social Norms

Tip sheet+ Letter with “Civic

Duty” Language + Social

Comparison

9

Social ComparisonSocial Comparison

Your own total consumption June to October 2006: 52,000 gallons Your neighbors’ average (median) consumption June to October 2006:35,000 gallons You consumed more water than 73% of your Cobb County neighbors.

10

Reduction in Summer ’07 Consumption

{95% CI}

Number of Homes

Control -- 71,643

Treat 1 (Tip) - 0.7%{-1.7%, 0.4%}

11,675

Treat 2 (Weak Norms)

- 2.7%{-3.7%, -1.8%}

11,675

Treat 3 (Strong Norms)

- 4.8%{-5.7%, -3.9%}

11,676

11

Summer 2008

Summer 2009

Treat 1 (Tip) ~0% ~0%

Treat 2 (Weak Norms)

~0% ~0%

Treat 3 (Strong Norms)

-2.5%(p<0.01)

-1.3%(p<0.05)

Persistence of Treatment EffectsPersistence of Treatment Effects

12

Heterogeneous Treatment EffectHeterogeneous Treatment Effect

Nonparametric Tests (Crump et al.

2008)

Test Null of Zero Conditional Average

Treatment EffectsCannot reject that Treatment 1 effect is

zero for all subgroups.

13

Heterogeneous Treatment EffectHeterogeneous Treatment Effect

Nonparametric Tests (Crump et al. 2008)

Test Null of Constant Conditional Average

Treatment Effects

Reject null that Treat 2 (p<0.10) and Treat 3 (p<0.01) have constant treatment effects

14

Treatment 1

(tips)

Treatment 2

(weak norm)

Treatment 3 (strong norm)

-4.0

0-3

.00

-2.0

0-1

.00

0.0

01.0

02.0

0T

reatm

ent 1

0 .2 .4 .6 .8 1Quantile

-4.0

0-3

.00

-2.0

0-1

.00

0.0

01.0

02.0

0T

reatm

ent 2

0 .2 .4 .6 .8 1Quantile

-4.0

0-3

.00

-2.0

0-1

.00

0.0

01.0

02.0

0T

reatm

ent 3

0 .2 .4 .6 .8 1Quantile

15

Subgroup AnalysisSubgroup Analysis

Household

Wealth (market value of house),

age of home, ownership

Neighborhood

characteristics Education,

race, ownership

Treatment 3 (2007) by SubgroupsTreatment 3 (2007) by Subgroups

More responsive: wealthy More responsive: wealthy

households, owners and households households, owners and households

in neighborhoods with more white and in neighborhoods with more white and

educated householders and fewer educated householders and fewer

renters.renters.

No difference conditional on year in No difference conditional on year in

which home was built.which home was built.16

Nonpecuniary vs PecuniaryNonpecuniary vs Pecuniary

Strong social norm message:Strong social norm message: 2007 effect equivalent to average 2007 effect equivalent to average

price increase of ~12-15%.price increase of ~12-15%. Immediately detectable one month Immediately detectable one month

later.later. High-income households High-income households

Most responsive to normsMost responsive to norms Least responsive to price (Mansur & Least responsive to price (Mansur &

Olmstead 2007)Olmstead 2007) Effect declines over time.Effect declines over time.

17

MechanismsMechanisms Evidence consistent with behavioral Evidence consistent with behavioral

changes with recurring costs rather than changes with recurring costs rather than one-time fixed-cost investments. Mostly one-time fixed-cost investments. Mostly from outdoor use.from outdoor use.

Evidence consistent with interpretation Evidence consistent with interpretation that social comparison operates through that social comparison operates through social norms rather than private social norms rather than private efficiency signals to a boundedly rational efficiency signals to a boundedly rational agent.agent.

18

19

Choosing Among Non-Choosing Among Non-experimental Estimatorsexperimental Estimators

“Design-replication study”

Form a non-experimental comparison group from households in neighboring counties. Then use “best-practice” econometric methods (regression, matching, panel data, regression discontinuity) to estimate treatment effect. Contrast non-experimental estimates to experimental estimates.