qualitative and mixed methods annual meeting handouts/handou… · qualitative and mixed methods...
TRANSCRIPT
Qualitative and Mixed Methods
For Emergency Medical Services Research
January 8, 2013
Anne Lang Dunlop, MD, MPH
Emory University
Learning Objectives
After this presentation, learners will have knowledge of:
• Qualitative research methods and how they contribute to disaster preparedness and response research;
• Approaches for conducting rigorous mixed methods research;
• Strategies for combining methods to increase research depth and rigor.
Mixed Methods Research
What is Qualitative Research?
• Qualitative research is an approach which seeks to understand human experiences, perceptions, motivations, intentions and behavior.
• By employing interactive, inductive, flexible, and reflexive methods of data collection and analysis.
Qualitative Research: Goal
• To understand those being studied from their perspective;
• To develop concepts and explore themes in order to understand phenomena in natural settings in an observational manner, emphasizing the experiences and views of participants.
Qualitative Research: Goal
• To understand those being studied from their perspective;
• To develop concepts and explore themes in order to understand phenomena in natural settings in an observational manner, emphasizing the experiences and views of participants.
In contrast to study in controlled settingsin an experimental manner
Qualitative Research: Research Questions
Qualitative studies are concerned with research questions such as:
“What is X and how does X vary in different circumstances, and why?”
In contrast with research questions such as:
“How many X’s are there?”
“Not everything that can be counted counts and not everything that counts can be counted.”
- Albert Einstein
1) Exploration is the essential feature of qualitative approach in order to understand the perceptions and actions of participants.
Qualitative Research: Main features
2) Inductiveness: Focused on developing concepts and generating hypotheses. The researcher is open to ideas that emerge from listening and observing people.
The inductive approach is particularly useful when little is known about the topics one wants to study.
Qualitative Research: Main features
3) Interactive & Reflexive: Researchers use interaction between the research team and participants to get closer to the topic under study.
The researcher becomes an instrument of data collection. The researchers probe, facilitate, and note tone, hesitations and repetition in participants responses.
Qualitative Research: Main features
4) Holistic: Researchers are allowed to put their responses in context.
Qualitative Research: Main features
5) Flexible methods: Qualitative research relies on personal and individual world of participants. So flexible, imaginative, creative and varied strategies are used to facilitate this process.
Qualitative Research: Main features
Four Fundamental Tasks of Research
1. Selecting subjects to study
2. Interacting with subjects to gather data
3. Avoiding arbitrary findings
4. Convincing others of what you found
Quantitative & qualitative research
approach these tasks differently:
Quantitative Four R’s
Qualitative Four P’s
Research Task4 R’s
(Quantitative)
4 P’s
(Qualitative)
How do I select research subjects?
Representativeness:Random samples of pre-
determined groups
Purposefulness:Sites/subjects sampled
according to needs
How do I work with subjects to get data?
(non-)Reactivity: Fixed data collection instrument
Participation: Flexible data collection strategies
How do I avoid arbitrary findings?
Reliability: Hypothesis testing via statistical
inference
Process: Iterative coding, synthesizing
How do I convince others of my findings?
Replicability: Tables, figures of specific
categories
Particularity: Narrative reports of findings in context
Qualitative Methods: Sample Selection
Qualitative inquiry focuses on purposefully selecting samples (purposive sampling):
Pre‐specified sampling logic to select information‐rich cases for study in depth
Not intended to achieve statistically representative sample!
Specific sampling strategy dependent upon purpose of the evaluation: See Table.
Type Purpose
Extreme or deviant case sampling Learn from highly unusual manifestations of the phenomenon (outstanding successes, notable failures, crises).
Intensity sampling Learn from information-rich cases that manifest the phenomenon intensely but not extremely.
Maximum variation sampling Documents diverse variations that have emerged in adapting to different conditions.
Homogeneous sampling Reduce variation in different possible subtypes to explore in depth for one subtype.
Typical case sampling Illustrates or highlights what is typical.
Stratified purposeful sampling Illustrates characteristics of particular subgroups of interest; facilitates comparisons.
Snowball sampling Identifies cases of interest from people who know people who know what cases are information-rich.
Criterion sampling Picking all cass that meet some criterion.
Theory-based sampling Finding manifestations of a theoretical construct so as to examine the construct.
Opportunistic sampling Following new leads during fieldwork to take advantage of the unexpected.
Table. Types of Purposeful Sampling for Qualitative Research
There are no hard “rules” for sample size in qualitative inquiry:
– The validity, meaningfulness, and insights generated have more to do with the information‐richness of selected cases and the analytic capabilities of the researcher than the size of the sample.
• Sample needs to be large enough to achieve saturation (the point at which no new ideas are being generated).
Qualitative Methods: Sample Size
Strengths & Weaknesses of Data Collection MethodsMethod of Data
Collection
Strengths Weaknesses
Participant observation Allows observer to “see” what is
happening; Occurs in natural environment
Presence may bias behavior/practices
May present ethical challenges
Key informant interview In‐depth information from one person
Individuals may be more likely to share
sensitive info
Can probe emergent areas/themes
Single person’s perspective
Results highly dependent upon
interviewer
Focus group interview Multiple perspectives on focused topic
Dialogue among participants exposes
themes
Limited information from each person
Dissent/sharing in group may be
limited
Document review Provide insights into perspectives Can be difficult to analyze, interpret
Open‐ended survey Anonymity may make it easier to share
sensitive info
Limited training needed to distribute,
administer
Literacy may limit responses
Requires training to develop valid
survey
Thematic Analysis
Coding of content into relevant themes and patterns AFTER reviewing responses
– Key concept is that themes, and patterns of themes, emerge out of the data (rather than being imposed on them prior to data collection and analysis)
Can be done manually (color coding schemes, index cards) or via computer software (MAXqda, Nvivo,
Atlas‐ti, HyperResearch).
Thematic Analytic – the Process1. Transcribe interview data into text files
2. Actively read and review the textual data:
a) Initial reading – to allow for inductive, emergent themes
b) Re‐read – annotating thoughts in margin to generate open codes
c) Sort items into proto‐themes and themes
3. Examine proto‐theme defined themes Code Book
4. Re‐examine text applying coding schema (multiple coders)
5. Analysis: Reducing and synthesizing data into patterns
Reporting Results of Analysis
Narrative – summarizes key response themes:
– Make mention of tally or hierarchy of themes;
– Include particularly vivid examples verbatim to provide richness and depth.
Matrix – tabular representation of the key response themes by specific variables (e.g. gender or education):
– Neatly “packaged” for review.
Reporting Criteria
Consolidated criteria for reporting qualitative research
(COREQ):
– 32‐item checklist
– 3 Domains addressed by checklist items:
1. Research team and reflexivity
2. Study design
3. Analysis and findings
Validity in Qualitative Research
Addressed via:
1. Triangulation
2. Instrument validity
3. Documentation of processes
Validity: Triangulation
1. Using a rich variety of data– Multiple sources of information
– Multiple data collection techniques
– Multiple analysis techniques
2. Any stated conclusion must rely on multiple types of evidence
3. Contradictory evidence– emerging theory must be modified in the presence of evidence that contradicts the current version of the theory
– contradictory evidence must be sought out
Validity: Instrument Validity
Eliminating bias by:
– Ensuring interview guides and survey questions convey the intended meaning
– Ensuring observers understand and interpret what they see correctly
Terminology is HUGE
Pre test and Pilot test!
Validity: Documentation
Auditing – tracing all processes from data sources to collection methods to analysis (coding, code book development) to conclusions
The chain of evidence and logic that leads to any conclusion must be clear to any reasonable observer
Qualitative Methods: Emergency Svcs Research
Two special features of public health emergency research distinguish it from other types of research:
1. The rapidity and evolution of the occurrence of the event and the response to it;
2. The great importance of the special circumstances or context of the event.
These same two features make qualitative methods particularly appropriate.
Well‐conducted qualitative methods in public health emergency research contribute to a “cross‐hazards” repository of information…
….by providing a rich contextual foundation for a given disaster event AND the associated behaviors of individuals/organizations affected by or participating in the response.
Qualitative Methods: Emergency Svcs Research
Defining Mixed Methods
“The collection, analysis, and integration of
quantitative and qualitative data into a single or
multiphase study. Its central premise is that the
use of quantitative and qualitative approaches
in combination provides a better understanding
of research problems than either approach
alone.” – Hanson, Creswell, Plano Clark, Petska 2005
Rationale for Mixed Methods
Quantitative research can reveal generalizable information fora large group of people….BUT these data often fail to providespecific answers, reasons, explanations or examples
Qualitative research provides meaning and context regardingthe people and environments of study…BUT the findings areoften not generalizable because of the small numbers &narrow range of participants
Mixed Methods: When used together,
these methods can be complimentary
Purpose of Combining Methods
1. Develop and enhance the validity of scales, questionnaire and tools
2. Develop, implement and evaluate interventions
3. Confirm or cross‐validate data
4. Study different aspects of the same topics
5. Explore complex phenomena from different perspectives
6. Further explore or test findings of one method
Mixed Methods – To Explain
Quantitative: Develop
concepts & ideas
Qualitative: Explain
concepts & ideas
Primary focus is to explain a phenomenon
Dig deeper into results to contextualize
Mixed Methods ‐ Strategy A
QUANTITATIVE
• Closed‐ended survey to assess knowledge, attitudes, behavior
QUALITATIVE
• Focus group interviews to add depth to survey findings
Quant/Qual
• Site visit to enrich and confirm findings (participant observation, checklists)
Mixed Methods – To Explore
Qualitative: Develop
concepts & ideas
Quantitative: Explore
concepts & ideas
Primary focus is to explore a phenomenon
Generalize qualitative findings to projectable samples
Mixed Methods ‐ Strategy B
QUALITATIVE
• Key informant interviews to inform survey language and content
QUANTITATIVE
• Closed‐ended survey to rank factors
QUALITATIVE
• Focus group interviews to add depth and context to interpret survey findings
Key Take‐Home Messages
We need a range of methods if we are to understand the complexities of disaster preparedness and response
Qualitative methods are complementary to quantitative methods:
– As an essential prerequisite to quantitative work
– To validate quantitative work
– To explore complex and sensitive issues not approachable by quantitative methods.
“Best Practices” for Mixed Methods Research
From NIH Office of Behavioral & Social Sciences Research
Available at: http://obssr.od.hih.gov/mixed_methods_research
Purpose: To develop practices that ‐
1. Assist investigators using mixed methods as they develop competitive applications for support from NIH;
2. Assist reviewers and staff for review panels at NIH who evaluate applications that include mixed methods research;
3. Provide the NIH Institutes and Centers with "best practices" to use as they consider potential contributions of mixed methods research and set priorities.
References• Creswell JW. Qualitative Inquiry and Research Design. 2nd
Edition. SAGE Publications, Thousand Oaks, CA: 2006.
• Creswell JW, et al, for the Office of Behavioral and Social Sciences Research. Best practices for mixed methods research in the health sciences. August 2011. National Institutes of Health. http://obssr.od.nih.gov/mixed_methods_research
• Hesse‐Biber S. Qualitative Approaches to Mixed Methods Practice. Qualitative Inquiry 2010; 16(6): 455‐468.
• Johnson RB. Mixed Methods Research: A Research Paradigm Whose Time Has Come. Educational Researcher 2009; 33(7): 14‐26.
• Tong A. Consolidated criteria for reporting qualitative research (COREQ). International J Qual Health Care 2007; 19(6): 349‐57.