Postgraduate Course

British Academy of Management

Linking Research Questions to Appropriate Designs

London, Dec 13, 2012

Postgraduate Course

Randomized controlled study? Grounded theory approach? Cohort / panel study? Qualitative field research? Longitudinal study? Post-test only study? Survey? Action research? Case study?

What is the best design?

Postgraduate Course

What is the BEST car?

Postgraduate Course

(more / less appropriate)

Postgraduate Course

What is the best design?

quants vs quallies, positivists vs post structuralist, etc

Postgraduate Course

on the research question

Postgraduate Course

Types of questions

Does it work?

Does it work better than ....?

Does it have an effect on ....?

What is the success factor for ....?

What is required to make it work ...?

Will it do more good than harm?

Effect

Postgraduate Course

Types of questions: non-effect

Needs: What do people want or need?

Attitude: What do people think or feel?

Experience: What are peoples’ experiences?

Prevalence: How many / often do people / organizations ...?

Procedure: How can we implement ...?

Process: How does it work?

Explanation: Why does it work?

Economics: How much does it cost?

Postgraduate Course

Effect vs Non-effect

Postgraduate Course

Internal validity

Postgraduate Course

internal validity = indicates to what extent the results of the research may be biased and is thus a comment on the degree to which alternative explanations for the outcome found are possible.

Internal validity

Postgraduate Course

Postgraduate Course

We are pattern seeking primates:

we are predisposed to see order and causal relations in the world

Causal relations

Postgraduate Course

Three criteria:

1. the "cause" and the "effect" are related

2. the "cause" precedes the "effect" in time

3. there are no plausible alternative explanations for the

observed effect

So, when do we know there is causal relation?

Causality

Postgraduate Course

1. Are the "cause" and the "effect” related?

2. Does the "cause" precede the "effect" in time?

3. Are there no plausible alternative explanations for the observed effect?

Considerations for research:

‘Effect’ questions

effect size

before and after measurement

randomization, blinding, control group

Postgraduate Course

Bias & Confounding

Research shows:

Shoe size > quality of handwriting

Smoking youngsters > better lung function

Postgraduate Course

Successful companies

Charismatic leaders

Research shows:

Postgraduate Course

Levels of internal validity

Postgraduate Course

Explanation

Which design for which question?

Postgraduate Course

Different types of research questions require different types of research designs,

but ...

Postgraduate Course

Best research design?

But 1: feasibility

Postgraduate Course

But 1: feasibility

Postgraduate Course

All research designs are flawed – though each is flawed differently:

Controlled studies: high internal validity, but sometimes less suited to generalization

Surveys and field research: low internal validity, but sometimes more useful for management practice.

But 2: it’s a trade off

McGrath, J. (1981). Dilemmatics, The Study of Research Choices and Dilemmas. American Behavioral Scientist, 25(2).

Postgraduate Course

SO:Evidence accrual requires convergence of findings derived from multiple studies that research the same constructs and variables with different research designs and in different contexts.

Postgraduate Course

The appropriateness of a research design is also determined by the body of evidence’s stage of development.

mature: extensively studied > controlled studies to identify independent, dependent and moderating variables

nascent: open-ended the research questions > exploratory qualitative studies to shape the understanding of the topic .

But 3: stage of development

Edmondson, A. C., & McManus, S. E. (2007). Methodological fit in management field research. Academy of Management Review, 32(4), 1155-1179.

Postgraduate Course

Figure 5: Controlled vs uncontrolled studies

Postgraduate Course

Different types of research questions require different types of research designs, but ...

feasibility

past research

stage of development

also have to be taken into account!

In summary