“who really thinks global warming is happening?”€¦ · “who really thinks global warming is...

13
“Who Really Thinks Global Warming is Happening?” An Experiment on Response Order Effects in Attitude towards Global Warming Sujata Pal, Abt SRBI Dr. Mindy Rhindress, Abt SRBI Gemma Natori, Abt SRBI Geoffrey Feinberg ,YPCCC Annual AAPOR Conference Hollywood, FL, 12-17 May, 2015

Upload: others

Post on 24-May-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

“Who Really Thinks Global Warming is Happening?”

An Experiment on Response Order Effects in Attitude towards Global Warming

Sujata Pal, Abt SRBI

Dr. Mindy Rhindress, Abt SRBI

Gemma Natori, Abt SRBI

Geoffrey Feinberg ,YPCCC

Annual AAPOR Conference

Hollywood, FL, 12-17 May, 2015

Page 2: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 2

Background: Public Opinion Research on Global

Warming

Public opinion polls are now routinely conducted on climate change and global warming;

Major polls show that most Americans believe global climate change is real;

– 69% agree that world’s temperature may have been going up slowly1

– 63% say global warming is happening2

– 61% say there is solid evidence that the earth is getting warmer3

There is also research into methodological and measurement issues in public opinion research on climate change;

1NY Times/Stanford/SSRS/Resources for the Future, 20152Yale Project on Climate Change/George Mason University Center for Climate Change Communication, 20153Political Polarization and Typology Survey, PEW 2014

Page 3: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 3

Goal 1: “Response Order” and Global Warming

Beliefs – Does it Matter?

Definition of Response Order Effect:

– “changes in answers to closed-ended survey questions produced by varying the order in which response options are presented”;

1

There is evidence of two kinds of effects; both will increase the likelihood that an item will be chosen;

– Primacy Effect

– Recency effect

Note: Primacy effect is more typical in self-administered; recency effect is more common in interviewer assisted;

We expect recency effect in our study since it uses telephone survey;

1Krosnick Jon A. and Duanne F. Alwin. 1987. “An Evaluation of a Cognitive Theory of Response-Order Effects in Survey Measurement.” Public Opinion Quarterly. 51-201-219

Page 4: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 4

Response Order Experiment

We use data from a national, dual frame telephone survey of

adult US population conducted by the Yale Program on

Climate Change Communication (YPCCC);

Respondents were asked, “Which of the following statements

comes closest to your view?”

Split-ballot experiment in which respondents were randomly

read 5 response choices in two orders -1-5 OR 5-1;

Page 5: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 5

Response Order Experiment: Items

Standard Order

(1-5)

Reverse Order

(5-1)Statements

n=378 n=380

1 5 Global warming isn't happening

2 4Humans can't reduce global warming, even if it

is happening

3 3

Humans could reduce global warming, but

people aren't willing to change their behavior,

so we're not going to

4 2

Humans could reduce global warming, but it's

unclear at this point whether we will do what's

needed

5 1Humans can reduce global warming, and we are

going to do so successfully

Table 1: Standard (1-5) and Reverse Order (5-1)

Page 6: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 6

Result 1: More People Said “Global Warming was

not Happening” when Offered Last (Reverse

Order)

10.1%

15.3%

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

Standard Order (1-5) Reverse Order (5-1)

Figure 1. Percent Choosing “Global Warming is not Happening” by Response Order

50% more people said they believed global warming was not happening when this item was placed last;

Difference=5.2%*significant

“Which of the following statements comes closest to your view? N=758

Page 7: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 7

Result 1: No Significant Effect was Noted for the

Other Items

Table 2. Percent Distributions for Each Item for the Standard Order and Reverse Order and Effect Size

Standard Order

(1-5)

Reverse Order

(5-1)

Difference

(Reverse Order -

Standard Order)

Significant

n=378 n=380

1. Global warming isn't happening 10.1% 15.3% 5.2% *

2. Humans can't reduce global warming, even if it is

happening13.0% 10.5% -2.5%

3. Humans could reduce global warming, but people

aren't willing to change their behavior, so we're not

going to

29.4% 28.4% -1.0%

4. Humans could reduce global warming, but it's unclear

at this point whether we will do what's needed38.1% 40.0% 1.9%

5. Humans can reduce global warming,

and we are going to do so successfully9.5% 5.8% -3.7%

Page 8: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 8

Goal 2: Does the Effect Size Vary by Education

and Topic Interest?

Does order effect vary by education? Previous research1

has

shown, those with low educational attainment are more

susceptible to response order effect;

– We compare those with at least a college degree vs. those

without;

Does order effect vary by topic interest? We expect that

respondents with low topic interest will be more susceptible to

a change in response order;

– “How much had you thought about global warming before today

– not at all. a little, some or a lot?”

1Krosnick Jon A. and Duanne F. Alwin. 1987. “An Evaluation of a Cognitive Theory of Response-Order Effects in Survey Measurement.” Public Opinion Quarterly. 51-201-219

Page 9: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 9

Result 2: The Difference was about Three Times

Larger for those without a College Degree

8.9% 11.9%15.9%

14.5%

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

Respondents without a College Degree Respondents with a College Degree or Higher

Standard Order (1-5)

Reverse Order (5-1)

Q. What is the last grade of school you completed? N=758

Difference=7%*Significant

Difference =2.6%

Figure 2. Percent Choosing “Global Warming is not Happening” by Response Order and Education

Page 10: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 10

Result 2: The Difference was more than Twice

for those with Low Topic Interest

12.8%

8.7%

22.2%

12.5%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

Not at All/A Little Some/A Lot

Standard Order (1-5)

Reverse Order (5-1)

Q. How much had you thought about global warming before today? N=758

Difference= 3.8%

Difference= 9.4%

Figure 3: Percent Choosing “Global Warming is not Happening” by Response Order and Topic Interest

Page 11: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 11

Conclusion

Experiment demonstrated support for the hypothesized recencyeffect in questions about global warming;

– More pronounced among those with lower levels of education and lower levels of interest in topic;

Recommended solution: Randomizing the response options in order to avoid systematic reporting bias;

Underscores need for further research into effects of education and topic awareness on public opinion research in global warming/climate change issues;

Page 12: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Abt SRBI | pg 12

Acknowledgements

We acknowledge the research team at YPCCC, mainly Geoffrey Feinberg and Abt

SRBI survey methodologists Dr. Courtney Kennedy and Marci Schalk for their

methodological support.

Page 13: “Who Really Thinks Global Warming is Happening?”€¦ · “Who Really Thinks Global Warming is ... 2Yale Project on Climate Change/George Mason University Center for Climate

Contact Information:Sujata Pal

Abt SRBI

tel. 646-486-8451

[email protected]