The Key Challenge in Behavioural ResearchUnderstand the Issue and Uncover the Solution

1

©2015 QUALTRICS LLC.

Welcome

Housekeeping• The recording and slides for today’s presentation will be

made available to you through email after the webinar commences

• Use the chat bar to ask questions

• Join the conversation on Twitter by tweeting @Qualtrics

Agenda• Introduction

• Common Method Bias - Explained

• Common Method Bias - Sources

• Remedies to Solve Common Method Bias

Dr. Esther TippmannLecturer at the Department of Management

University College Dublin

Elena SugrueAcademic Account Executive

Qualtrics

We often use self-report surveys

Respondent reads the question and selects a response without researcher interference

Involves asking about respondent’s behaviours, feelings, attitudes, beliefs, etc.

Self-report surveys often have data quality issues

Common method bias as one data quality issue

Bias attributable to measurement method: Respondents have a tendency to

answer questions in a consistent manner

Can cause systematic measurement error

Concern most severe if:Data on both the outcome variable (dependent variable such as performance indicators) and explanatory variable (such as activities, behaviours, beliefs, attitudes) at the same time from the same respondent

(Source: Podsakoff, MacKenzie, Lee & Podsakoff (2003) Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5): 879-903)

Common method bias may inflate covariation among variables

Relationship in data collected from a single respondent

Concern of common method bias

Relationship in data collected from two respondents

(One respondent reports outcome variable, another explanatory variable)

(For example: Podsakoff, MacKenzie & Podsakoff (2012) Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63(8): 539-569)

Research design remedies to solve common method bias

• Collect information for key constructs at different points in time

• Questionnaire design

• Using multiple respondents (sources) to collect information for key constructs

1st respondent reports on outcome

variable

2nd respondent reports on

explanatory variables

Multiple respondents and pairing surveys

Multiple respondents and pairing surveys

Reducing Bias in Survey Responses:The Role of Randomisation

Problem:• Selecting the first reasonable option is one

of the strategies employed by respondents that are not engaging in the optimal process of responding to a question

Solution: 1. Answer choice randomisation (fixed

display and subsets)

2. Block randomisation

3. Question randomisation

Label AnchorsDifferent question types

Different ways question types

Different anchor points

Order of labels

Q&A

13

©2015 QUALTRICS LLC.

Thank You!

14

©2015 QUALTRICS LLC.