mankarious evon 20130208

Evon MankariousEmily Kothe

Measurement Reactivity

A Meta-Analysis of the Effects of Presenting Theory of Planned Behaviour Constructs on

Follow-Up Behaviour

Outline of presentation

2

• Introduction-What is measurement reactivity and why does it occur?

• Methods

-Inclusion and Exclusion criteria

-Database search and search terms

-Data-extraction and meta-analytical process

• Results

-Literature search

-Number of studies included

-Number of studies measuring particular behaviours

-Moderator analyses and sub-group analyses

• Discussion

-What do the results suggest?

-Strengths and limitations

-Future direction

What is measurement reactivity and why does it occur?

3

• Phenomenon occurs when the mere presentation of questions about intention at baseline changes follow-up behaviour

• Likely to occur in any theory where intention is a central construct- Theory of planned behaviour was used here.

Theory of planned behaviour

4

Measurement reactivity changes in behaviour at follow-up

• Measurement reactivity has been reported to change follow-up behaviour in many health behaviours:

- Cervical cancer screening (Sandberg and Conner, 2009)

- Blood donation (Godin et al., 2008)

- Physical activity (Godin et al., 2011)

- Illicit drug use (Williams et al., 2006)

5

Limitations of measurement reactivity literature

6

Measurement reactivity

Participation effects

Behavioural change

Problems with isolating measurement reactivity from participation effects

• Kypri et al. (2011) – What participants think about the nature of the study may affect subsequent behaviour and potentially bias study findings

• doesn’t address the issue of measurement reactivity as measurement of intention remains common across all groups.

• Solomon four-group designs can experimentally manipulate baseline assessment but do not isolate measurement reactivity from participation effects.

7

Significance of the current analysis and aims

• Best way- investigate changes in non-intervention studies which measure intention at baseline and behaviour at both baseline and follow-up through meta-analytical techniques.

• Using studies not designed to investigate measurement reactivity reduces risk of potential publication bias in measurement reactivity literature.

• No research has attempted to systematically investigate the existence of measurement reactivity within non-intervention studies.

8

Aims

to determine the extent to which behaviour changed in non-intervention studies which applied the theory of planned behaviour questionnaire to measure intention at baseline.

9

Investigation of Moderator Variables

• Type of behaviour (Socially desirable vs. undesirable)

- inconsistencies in measurement reactivity studies measuring socially undesirable behaviour.

- Included to determine if classification of behaviour had a moderating effect

• Length of follow-up

- Based on previous literature, was hypothesised that increasing length of follow-up time would decrease magnitude of measurement reactivity.

10

Inclusion/Exclusion criteria

11

Inclusion Exclusion

Only studies looking at health behaviours

Cross-sectional, qualitative, and intervention studies

measured all theory of planned behaviour constructs at baseline

Articles published in languages other than English

measured behaviour at baseline and follow-up

Database search and search terms

• Literature search and data extraction phases were performed in June 2012.

• PsychINFO (OVID), MEDILINE (OVID) and Web of Science (ISI web of knowledge) databases

• Search strategy used here was modelled on a recent meta-analysis on the theory of planned behaviour in prospective studies conducted by McEachan and colleagues (2011).

12

Data extraction and met-analysis procedure

• Mean, standard deviation, and sample size at both baseline and follow-up were extracted to allow for the calculation of effect sizes

• Cohen’s d was calculated to determine the standardised mean difference for behaviour from baseline to the first follow-up

• Meta-analysis conducted using the Metafor package for R

• Follow-up length of time and behaviour type examined in a mixed-effects model.

• Sub-group analyses were also conducted to determine the mean effect size for behaviour investigated in more than one study

13

Results: Literature search

14

Literature Search n

Database search (after duplicates removed) 4034

Title 1630

Abstract 2221

Full-text 183

Total 23

Number of studies measuring particular behaviours and Type of behaviour

15

Length of follow-up

Length of follow-up n

1 week 5

2 weeks 3

4 weeks 1

5 weeks 3

6 weeks 1

8 weeks 2

3 months 2

6 months 6

16

Moderator analysis, changes in behaviour across all studies and sub-group analyses

• Non-significant effects were found for behaviour type (p = .35) and for follow-up length (p = .83)

• Average change in behaviour across all studies was small (d = -.08, 95% CI [-0.17, 0.01])

• Physical Activity

- Average change from baseline to follow-up was small (k = 13, d < .001, 95% CI [-0.07, 0.07]

• Binge Drinking behaviour:

- Average change from baseline to follow-up was small and decreased (k = 2, d = -0.14, 95% CI [-0.41, 0.13]

17

Discussion: Does measurement reactivity occur?

• Results did not support a measurement reactivity account of behavioural change

• Changes in behaviour in intervention studies may actually be the result of other factors, for example participation effects and not measurement reactivity

18

Is length of follow-up time a moderator?

• No- non-significant effects were found

• Hypothesis that increasing follow-up length of time would reduce magnitude of measurement reactivity was not supported.

• Researchers using this theory should use the theory to accurately predict behaviour and it does not appear that they should be concerned about measurement reactivity.

19

• No

• Changes in behaviour in studies investigating socially desirable behaviours did not differ from studies investigating socially undesirable behaviours.

• Inconsistencies observed between studies which have measured socially undesirable behaviours may be the result of:

- Question framing

- socially undesirable behaviours may also affected by social norms

20

Does behaviour type matter?

Sub-group analyses

• Binge Drinking:

- Small decrease in behaviour

- Caution should be used as only 2 studies were found

- Previous studies have shown that presenting questions about intention at baseline can decrease binge drinking behaviour

- As only 2 studies were included, it is possible that changes in binge drinking behaviour may be the result of measurement reactivity.

• Physical Activity:

- Results suggest that changes in behaviour are unlikely to be the result of measurement reactivity and may reflect participation effects.

- Not consistent with previous research

21

Strengths and Limitations

22

Strengths Limitations

• No previous meta-analysis looking at measurement reactivity

• Use of large body of literature reduced publication bias

• Sub-group analyses could not be conducted on many behaviours

• Studies may have measured behaviour twice but reported it once

Future direction

• To increase number of studies included in future meta-analyses, other theories that have intention as a core construct could be included

• To our knowledge, no research exists which allows for the separation of behavioural change believed to be the result of measurement reactivity vs. behavioural changes as a result of participation effects.

• New design would be an 8-arm 2x2x2 fully crossed factorial design.

23

Example of new study design

24

Thank-you

25

References

26

1. Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211.

2. Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, N.J.: Prentice-Hall.

3. Chandon, P., Morwitz, V. G., & Reinartz, W. J. (2004). The short- and long-term effects of measuring intent to repurchase. Journal of Consumer Research, 31(3), 566-572.

4. Chapman, K. J. (2001). Measuring intent: There's nothing “mere” about mere measurement effects. Psychology and Marketing, 18(8), 811-841.

5. Chatzisarantis, N. L. D., Frederick, C., Biddle, S. J. H., Hagger, M. S., & Smith, B. (2007). Influences of volitional and forced intentions on physical activity and effort within the theory of planned behaviour. Journal of Sports Sciences, 25(6), 699-709.

6. Chatzisarantis, N. L. D., Hagger, M. S., & Smith, B. (2007). Influences of perceived autonomy support on physical activity within the theory of planned behavior. Special Issue: Familiarity Impacts Person Perception, 37(5), 934-954.

7. Cooke, R., Sniehotta, F., & Schüz, B. (2007). Predicting binge-drinking behaviour using an extended TPB: Examining the impact of anticipated regret and descriptive norms. Alcohol and Alcoholism, 42(2), 84-91.

8. Elliott, M. A., Armitage, C. J., & Baughan, C. J. (2003). Drivers' compliance with speed limits: An application of the theory of planned behavior. Journal of Applied Psychology, 88(5), 964-972.

9. Elliott, M. A., & Thomson, J. A. (2010). The social cognitive determinants of offending drivers' speeding behaviour. Accident Analysis and Prevention, 42, 1595-1605.

10. French, D. P., & Sutton, S. (2010). Reactivity of measurement in health psychology: How much of a problem is it? What can be done about it? British Journal of Health Psychology, 15(3), 453-468.

11. Fulham, E., & Mullan, B. (2011). Hygienic food handling behaviors: attempting to bridge the intention-behavior gap using aspects from temporal self-regulation theory. Journal of Food Protection, 74(6), 925-932.

12. Godin, G., Bélanger-Gravel, A., Amireault, S., Vohl, M. C., & Pérusse, L. (2011). The effect of mere-measurement of cognitions on physical activity behavior: A randomized controlled trial among overweight and obese individuals. International Journal of Behavioral Nutrition and Physical Activity, 8(1), 2.

13. Godin, G., Sheeran, P., Conner, M., & Germain, M. (2008). Asking questions changes behavior: Mere measurement effects on frequency of blood donation. Health Psychology, 27(2), 179-184.

14. Hagger, M. S., Chatzisarantis, N., Biddle, S. J. H., & Orbell, S. (2001). Antecedents of children's physical activity intentions and behaviour: Predictive validity and longitudinal effects. Psychology & Health, 16(4), 391-407.

15. Jackson, C., Smith, R. A., & Conner, M. (2003). Applying an extended version of the theory of planned behaviour to physical activity. Journal of Sports Sciences, 21(2), 119-133.

16. Kwan, M. Y. W., Bray, S. R., & Ginis, K. A. M. (2009). Predicting Physical Activity of First-Year University Students: An Application of the Theory of Planned Behavior. Journal of American College Health, 58(1), 45-52.

17. Lowe, R., Eves, F., & Carroll, D. (2002). The influence of affective and instrumental beliefs on exercise intentions and behavior: A longitudinal analysis. Journal of Applied Social Psychology, 32, 1241-1252.

18. Martin, J. J., Oliver, K., & McCaughtry, N. (2007). The theory of planned behavior: Predicting physical activity in Mexican American children. Journal of Sport & Exercise Psychology, 29, 225-238.

19. McEachan, R. R. C., Conner, M., Taylor, N. J., & Lawton, R. J. (2011). Prospective prediction of health-related behaviours with the Theory of Planned Behaviour: A meta-analysis. Health Psychology Review, 5(2), 97-144.

20.Nejad, L. M., Wertheim, E. H., & Greenwood, K. M. (2004). Predicting dieting behavior by using, modifying, and extending the theory of planned behavior. Journal of Applied Social Psychology, 34(10), 2099-2131.

21.Norman, P., Armitage, C. J., & Quigley, C. (2007). The theory of planned behavior and binge drinking: Assessing the impact of binge drinker prototypes. Addictive Behaviors, 32(9), 1753-1768.

22.Norman, P., & Conner, M. (2005). The theory of planned behavior and exercise: Evidence for the mediating and moderating roles of planning on intention-behavior relationships. Journal of Sport & Exercise Psychology, 27(4), 488-504.

23.Norman, P., Conner, M., & Bell, R. (1999). The theory of planned behavior and smoking cessation. Health Psychology, 18(1), 89-94.

24.Norman, P., Conner, M., & Bell, R. (2000). The theory of planned behaviour and exercise: Evidence for the moderating role of past behaviour. British Journal of Health Psychology, 5(3), 249-261.

25.Norman, P., & Cooper, Y. (2011). The theory of planned behaviour and breast self-examination: Assessing the impact of past behaviour, context stability and habit strength. Psychology & Health, 26(9), 1156-1172.

26.Norman, P., & Smith, L. (1995). The theory of planned behaviour and exercise: An investigation into the role of prior behaviour, behavioural intentions and attitude variability. European Journal of Social Psychology, 12(4), 403-415.