Survey Experiments on the Internet

Lynn VavreckUCLA

Study Director, Polimetrix Cooperative Congressional Election Study

The 2006 Cooperative Congressional Election Study

Steve Ansolabehere, MITPrincipal Investigator

Lynn Vavreck, UCLAStudy Director

Doug Rivers,CEO & President Polimetrix, Inc.

Steering CommitteeDon Kinder, University of MichiganBob Erikson, Columbia UniversityLiz Gerber, University of MichiganWendy Rahn, University of

MinnesotaJohn Sides, George Washington

Univ.Jeremy Pope, Brigham Young Univ.

Arizona State UniversityDartmouth CollegeGeorge Washington UniversityUniversity of MarylandMITStanford UniversityUC San DiegoUCLA1Vanderbilt UniversityUniversity of AkronUniversity of ChicagoColumbia UniversityUniversity of Michigan1University of Michigan2University of MinnesotaUC DavisUC RiversideOhio State UniversityHarvard UniversityReed CollegeUC MercedUniversity of PittsburghUniversity of Texas, AustinBrigham Young UniversityCalifornia Institute of TechnologyFlorida State UniversityUC BerkeleyUNCDuke UniversityYale UniversityUniversity of IllinoisMichigan State UniversityUniversity of WisconsinUniversity of Washington2UCLA 2Dartmouth College 2Harvard University 2Trinity CollegeNotre Dame UniversityUniversity of PennsylvaniaTemple University

37 Universities

37 Teams

Over 100 Political Scientists

39,000 Respondents

1600 Unique Survey Questions

Disclosures:

Not Enron.

I have been paid as a consultantto do work for Polimetrix.

I own part of the company.

Privileged.

Lynn Vavreck UCLA Political Science

CCES had Two Parts

Common Content – First 10 minutes of every survey. Content designed by Steering Committee. Roughly 38,000 completes.

Team Content – Second 10 minutes of every survey. Each team was assigned 1,000 of the 38,000 respondents.

Lynn Vavreck UCLA Political Science

CCES had Two Parts

Common Content

MIT

UCLA OSU UNC StanfordDuke

UM

First 10 Minutes

Second 10 Minutes

Lynn Vavreck UCLA Political Science

TopicsRepresentationReligionCampaign EffectsNeighborhood ContextRaceMedia EffectsForeign PolicyDiplomacyInformation/SophisticationIdentity Politics (Gay Rights

Initiatives)Health Care PolicyObesityAbortionElection Day Experience

Lots of experimentsPeople leveraged the Internet’s unique audio and visual capabilities

Assess advertising recall

Calibrate levels of social norms

Isolate effectiveness of different direct mail pieces

Test for priming effects in advertising

Differentiate Attack from Promotional Ad Effectiveness

Determine whether male or female voices affect listeners in ads

Explore whether a Democrat could have gotten tough on North Korea

Determine whether Americans exaggerate their belief in God

Lynn Vavreck UCLA Political Science

Preview Results from 2 Projects The effects of

advertising content on candidate assessments

Do characteristics of the voice-over matter as you vary the tone of advertising?

John Geer & Lynn Vavreck

Why do so many Americans believe in God?

A more precise measure of the level of belief in God in the US

Simon Jackman

Lynn Vavreck UCLA Political Science

Candidate & Challenger Information Matters Voters form perceptions based on

information/behavior of candidates• But also on information from challengers

When candidates attack one another, voters may change placements of candidates

• People thought McGovern was “too” liberal because Nixon portrayed him this way

• We observe this placement of McGovern and conclude voters cannot correctly conceive of politics using ideology

Lynn Vavreck UCLA Political Science

Ideological Placements of Candidates Radio advertising experiment

Produced 6 ads• Non-ideological Promotional Ad (male or

female voice)• “Too Conservative” Attack (male or female

voice)• “Too Liberal” Attack (male or female voice)

Lynn Vavreck UCLA Political Science

Ideology Placements of “Rob Smith”

self

20 30 40 50 60 70 80

PositiveMale

PositiveFemale

McCainH. Clinton

Democrats

Republicans

self

20 30 40 50 60 70 80

Republicani

Democrati

FEMALE VOICE ONLYMALE VOICE ONLY

With an ATTACK for being TOO CONSERVATIVEWith an ATTACK for being TOO LIBERAL

Lynn Vavreck UCLA Political Science

Religion in America How many of the

following statements apply to you?

• I have had dreams in which I see myself dying

• I believe in life after death

• I believe miracles sometimes happen

Treatments:• I do not believe in

God• I believe in God

Lynn Vavreck UCLA Political Science

Gary Jacobson: Belief in George Bush’s Divine Mission

Percent Approve Iraq Part ofin of Support Iraq had War on

Category Bush Iraq War WMD Terrorism

Republicans (N=219)Bush was chosen by God 29 97 89 87 99Don’t know 28 82 72 61 83Bush was not chosen by God 43 67 63 49 64

Independents (N=221)Bush was chosen by God 10 89 85 68 87 Don’t know 16 55 57 56 56Bush was not chosen by God 74 18 20 19 19

Democrats (N=216)Bush was chosen by God 3 60 20 20 60 Don’ know 1 0 50 50 50Bush was not chosen by God 96 1 3 8 9

All Respondents (N=656)Bush was chosen by God 14 92 81 76 93Don’t know 15 72 69 61 74 Bush was not chosen by God 71 20 21 22 23

Lynn Vavreck UCLA Political Science

Cooperative Polimetrix Election Study, 2007-2008 (CPES) Coverage in every state

• Primary Elections• Electoral College

Flexibility in terms of where and when teams can field their modules

Pre-wave with same Common/Team design• Brief post-election wave to gather turnout and

vote choice information