MIXED METHODS RESEARCH

Anneke Fitzgerald

Objectives

To identify the advantages and disadvantages of employing multiple research methods;

To consider those factors that influence the choice of research method and sequence of the multiple methods;

To understand triangulation and its variations; To examine the management and analysis of

data in the context of multi method research; To consider the need for clear planning and

managing a mixed methods approach

Methods, Data Analyses, Data Displays & Stories

Text of the Story

Narrative & discourse analysis

Systemic networks, category systems, & tree structures

Multi-way Qualitative Matrix Displays (with or without pattern coding)

Diagrams, maps, models and other pictorial summaries

Statistical summaries & tables

Graphs & figures

Hybrid devices, including dynamic systems modeling & Monte Carlo simulations

Interviews Observations Rich pictures, mind maps, cause

maps, concept maps, cognitive maps Surveys/questionnaires/scales Stories & metaphors Drawings, pictures, photographs, videos,

films Documents and other traces Physical measures & traces Experiments/quasi-experiments Focus groups Diaries, journals, and other self-monitoring

techniques Structured analytical tools & templates Think aloud protocols & other process

tracing methods Simulation models & ‘flight’ simulators

(varying in degree of structure, depth, & intrusiveness)}

...

.. .

.

Text of the Story

Narrative & discourse analysis

Systemic networks, category systems, & tree structures

Multi-way Qualitative Matrix Displays (with or without pattern coding)

Diagrams, maps, models and other pictorial summaries

Statistical summaries & tables

Graphs & figures

Hybrid devices, including dynamic systems modeling & Monte Carlo simulations

Interviews Observations Rich pictures, mind maps, cause

maps, concept maps, cognitive maps Surveys/questionnaires/scales Stories & metaphors Drawings, pictures, photographs, videos,

films Documents and other traces Physical measures & traces Experiments/quasi-experiments Focus groups Diaries, journals, and other self-monitoring

techniques Structured analytical tools & templates Think aloud protocols & other process

tracing methods Simulation models & ‘flight’ simulators

(varying in degree of structure, depth, & intrusiveness)}

...

.. .

.

......

.. .

..

. ..

GOAL: Achieve Desired Balance to tell Different Types of Stories• Theme/category-based• Time-based• Event-based• Perception-based• Mental model-based• Role-based• Cognition-based• Cohort-based• Group-based• Intervention-based• Hypothesis-based• Pattern-based• Impact-based• Change-based• Relationship-based• Cross-case• Generalising• Data integration• Hybrids of above• Emergent stories . . .

•Interaction-based strategiesStructured, Semi-structured, Unstructured interviews; Focus groups

•Participant-centred strategiesMapping & drawing; Storytelling; Self-recording

•Observation-based strategiesParticipant observation; Systematic observation; Unobtrusive observation

•Document-based strategiesTextual evidence gathering; Multimedia evidence gathering; Meta-analysis; Secondary database methods

•Experience-structuring strategiesExperiments/Quasi-experiments; Organising frameworks; Projective techniques; Process-tracing methods; Simulations & games

•Measurement strategiesObjective tests & assessments; Self-report instruments; Physical measurements

STRATEGICCHOICES

In a group of 3-4 discuss the advantages and disadvantages of using multi-methods

Advantages Each method can enhance the unique qualities of the other and

result in vast opportunities for reciprocal advantages Multi-method research overcomes the weaknesses of individual

approaches, whereby one method’s strength may compensate for the other method’s weakness

It appreciates the complexity of a research problem Multi-method research reduces bias It might increase response rate It elicits more detailed information It enhances theory building It enhances hypothesis testing It enhances generalisations It enhances validity It enhances accuracy of perceptions It might bridge macro and micro levels of social reality Multi-method research exposes the researcher to different ways

of thinking (and behaving)

Disadvantages

Multi-method research requires more time in planning and analysing the data

It requires more effort in planning and analysing the data

It requires more sensitivity because researchers have to synthesise the techniques to achieve synergy among them

Merging approaches

Concurrent – convergent parallel design – triangulation models A two-phase design (qual and quant) A dominant / less-dominant design (embedded)

Sequential – exploratory design Emphasis on reflective learning and contextualised

understanding Emphasis on early learning providing feedforward

Sequential – transformative Quant – qual – action for change

Sequential – multiphase –discrete yet linked studies Study 1 informs Study 2 informs Study 3 (mixed methods)

Mixed Methods Approaches - Prototypes

[Adapted from: Creswell, J.W. & Plano Clark, V.L, (2011). Designing and conducting mixed methods research (2nd Ed.). Thousand Oaks, CA: Sage Publications, pp. 69-70.]

Quantitative Data

Collection and Analysis

Qualitative Data

Collection and Analysis

InterpretationCompare or relate

(a) Concurrent - Convergent parallel design ~ triangulation models

Quantitative Data

Collection and Analysis

Qualitative Data

Collection and Analysis

InterpretationFollow up with

(c) Sequential – Explanatory design – emphasis on reflective learning &contextualised understanding (d) Sequential – Exploratory design ~ emphasis on early learning providing feedfoward

Qualitative Data

Collection and Analysis

Quantitative Data

Collection and Analysis

InterpretationBuilds to

Quantitative (or Qualitative Design

Quantitative (or Qualitative)Data Collection and Analysis

Qualitative (or Quantitative) Data Collection and Analysis

(before, during, or after)

Interpretation

(b) Concurrent - Embedded design w/ differential emphasis on data types

(f) Sequential - Multiphase design ~ discrete, self-contained yet linked studies

Study 1:Qualitative

InformsOverallProgramObjective

Study 2:Quantitative

Study 3:Mixed Methods

Informs

(e) Sequential - Transformative design ~ guided by an over-arching framework for achieving change/improvement; may have implied or explicit feedback loops or learning cycles, as in action research

Quantitative Data Collection and

Analysis

Qualitative Data Collection and

AnalysisInterpretationFollow up with

Transformative framework

Actions for change

Thinking of your own study, which merging approach are you taking and why?

Sequence selection? The research topic

What does your discipline suggest about the order of data collection methods?

The research question Is there a validated research tool that will answer your question?

Timing Is there a holiday period that will impact your ability to recruit

participants? Should you interview or utilise your survey first? You should probably

administer the survey first and follow up with interviews to avoid biasing the responses of research participants to the survey.

The context within which the research will take place Where will you be conducting the research?

What else will be going on in your life at that time? The resources available Do you need money to conduct the research?

Do you need access to particular computer software packages?

Triangulation

Triangulation is the use of several different research techniques in the same study to confirm and verify data gathered in different ways.

Triangulation adds rigour, richness and depth to the design and to the data.

Types of triangulation

Source triangulation Investigator triangulation Theory triangulation Methodological triangulation Recruitment triangulation Data collection and analysis triangulation Research team triangulation Interdisciplinary triangulation

Developing a Mixed Methods Mindmap

[Adapted from: Cooksey, R.W. & McDonald, G. (2011). Surviving and thriving in postgraduate research. Prahran, Vic: Tilde University Press, p. 346.].

Research quality

Is the research story, as a whole, convincing with respect to the arguments being made? Context

Juxtapositioning with other research Contextual knowledge Researcher positioning

Realization Internal coherence Analytical integrity Extensional reasoning – implications to other contexts

Explication Handling of unexpected outcomes Value for learning Fertilization of new ideas Acknowledgement of limitations Presentation to audience

What have other researchers done and where does this

research fit in?

How well does the researcher use knowledge about context to add richness

and/or qualification to their research process and the resulting findings?

Where & how does everyone or everything else fit within the

research context?

Where & how does the researcher fit within the

research context?

Does the research, as a whole, hang together as a coherent process to

permit the conclusions the researcher seeks or claims?

Do the analyses lead to or support appropriate, defensible & clear conclusions

given the quality of data to hand?

Do the research findings have meanings or implications for

other contexts?

How well has the researcher dealt with surprises and unanticipated findings?

What can consumers/ readers/users take away from

the research as important messages or lessons?

Can others run with or build upon what has been shown by or learned from the research?

What constrains the learning value, implications and

applicability of the research?

Is the presentation of the research suitable for the

intended audience(s) (i.e., is the researcher hitting the mark)?

Is the research story, as an integrated whole,

convincing with respect to the arguments being made?

[adapted from Cooksey, R.W. (2008). Paradigm-independent meta-criteria for social & behavioural research. Proceedings of the 2nd Annual Postgraduate Research Conference, University of New England, Armidale, NSW, pp. 4-17.]

MANAGING MIXED METHODS RESEARCH

Some Useful Mixed Methods References

Bergmann, M.M. (ed.) (2008). Advances in mixed methods research: Theories and applications. London: Sage Publications.

Brocklesby, J. (1997). Becoming multimethodology literate: An assessment of the cognitive difficulties of working across paradigm. In J. Mingers & A. Gill (Eds.), Multimethodology: The theory and practice of combining management science methodologies. Chichester, UK: John Wiley & Sons, 189-216.

Cooksey, RW & McDonald, G. (2011). Surviving and thriving in postgraduate research. Prahran, VIC: Tilde University Press.

Creswell, J.W. (2003). Research design: Qualitative, quantitative and mixed methods approaches,(2nd Ed.). Thousand Oaks, CA: Sage Publications.

Creswell, J.W. & Plano Clark, V.L. (2011) Designing and conducting mixed methods research (2nd Ed.). Thousand Oaks, CA: Sage Publications.

International Journal of Mixed Methods in Applied Business & Policy Research, online journal, Sage International Journal of Multiple Research Approaches, e-Content Management. Journal of Mixed Methods Research, Sage. Morse, J.M. & Niehaus, L. (2009). Mixed method design: Principles and procedures. Walnut Creek, CA: Left

Coast Press. Plano Clark, V.L. & Creswell, J.W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage Publications. Tashakkori, A. & Teddlie, C. (Eds). (2003). Handbook of mixed methods in social & behavioral research.

Thousand Oaks, CA: Sage Publications. Teddlie, C. & Tashakkori, A. (2009). Foundations of mixed methods research: integrating quantitative and

qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage Publications.

MIXED METHODS Special Interest group at ANZAM

Launch of the Mixed Methods SIG is on

Wednesday 7 December1400, Amora 4

(with thanks to Ray Cooksey)