critical appraisal and interpretation of qualitative evidence · research? what are their...
TRANSCRIPT
School of Health Sciences and Social Work
Critical Appraisal and Interpretation of Qualitative Evidence
School of Health Sciences and Social Work
Focus of this presentation
• Critically appraising a qualitative paper using the Critical Appraisal Skills Programme (CASP) worksheet for qualitative research
School of Health Sciences and Social Work
Screening Questions
What is qualitative
research?
Why would we use
qualitative
research?
What types of questions
does qualitative
research ask?
How does qualitative
research differ from
quantitative? What can qualitative
research contribute to
healthcare? Critical Appraisal Skills Programme (CASP)
School of Health Sciences and Social Work
What is qualitative research?
So, what is qualitative research?
Wh
at
is q
ualita
tive r
esearch
?
“Not everything that can be counted counts and not everything that counts can be counted”
PEOP
LE
Experiences
Meaning
Feelings
Words
Naturalistic
Multiple realities
OBJECTIVE
EASY SMALL SAMPLE
X Y
TOUCHY-FEELY
School of Health Sciences and Social Work
Life History Study
Naturalistic enquiry
Interpretative
Grounded theory
Different types of qualitative research
Different types of qualitative
research
Discourse Analysis
Ethnomethodology
Action research
Interactionism
Write and Draw
Rapid Appraisal
Content Analysis
Phenomenology
Conversational
Analysis
Participative
research
Cognitive
Anthropology
Feminist Research
Ethnoscience
Narrative
Humanistic
Research
Qualitative
Evaluation
Case Study Heuristic Research
Symbolic
Interactionism
Hermeneutics
Evaluative Research
Ethnography
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
What is the research
question and will a
qualitative approach
answer it?
What does the research
promise to offer
healthcare?
Is the research
relevant?
School of Health Sciences and Social Work
Sampling
Who do we collect data from in
qualitative research?
How many people are enough?
How do we find people?
Critical Appraisal Skills Programme (CASP)
School of Health Sciences and Social Work
Sampling: How do we find people to take part in qualitative research?
Convenience Sampling
Purposive Sampling
Snowball Sampling
Theoretical Sampling
School of Health Sciences and Social Work
Sampling: How do we know we have enough people?
Have we stopped finding any novel data? (Reached data saturation)
Are there enough people to cover the range of different experiences? E.g. People of different ages, with different disease severities (provide a concrete example)
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
Have they used
an appropriate
sampling
strategy?
Have they described the
sample well enough for you to
decide whether this applies to
your patient(s)?
How did they decide
how many people to
sample? Was this
appropriate?
Do we know
who collected
the data?
Do we know who didn’t
take part? And if not,
why not?
School of Health Sciences and Social Work
Data collection
What data collection tools are used in qualitative
research?
What are their advantages/disadvantages?
How do we decide which tool to use?
School of Health Sciences and Social Work
Data collection
FOCUS GROUPS
Affordable and
useful to explore
group interaction,
and discuss/
debate range of
experiences. Can
be difficult to
arrange.
DIARIES
Affordable, and good
at recording how
people feel at the time.
Difficult to ensure that
the data is
appropriate.
DOCUMENTS
Affordable, but
data not
recorded with
research in
mind.
INTERVIEWS
Useful to
explore in-depth
experiences,
but expensive to
conduct
OBSERVATION
Good to record actual
rather than reported
behaviour. Misses
different perspectives.
Potential ethical
issues.
How is data collected
in qualitative
research?
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
Was the data collected in a
way that addressed the
research issue?
Do we have enough
detail of how and
where the data was
collected?
Are the limitations of
the chosen method
presented?
Was the method
of data
collection
justified?
School of Health Sciences and Social Work
Reflexivity
How might the researcher affect
qualitative research?
School of Health Sciences and Social Work
The importance of reflexivity
In qualitative the researcher has an influence on the data in a number of ways:
• Influence on the participant • Body language • Type of questioning • Interview schedule • Manner with participant • Existing relationship with the participant
• During data analysis: • Pre-existing ideas about what they think they will/want to
find
It is important that the researcher considers their influence on the participant and the analysis, and that they present an account of their position so readers are able to judge this.
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
Did the researcher’s own
experience inform the
research question?
Has the researcher
talked about who they
are and the influence
they may have on the
data?
Did the researcher
do anything to limit
their impact on the
data?
School of Health Sciences and Social Work
Ethical issues
What ethical issues might arise
in qualitative research?
School of Health Sciences and Social Work
What are the ethical issues involved in qualitative research?
• As with all research:
– If conducted within the NHS must have LREC approval.
– All participants should give informed consent
• Data protection and confidentiality issues (presentation of anonymous quotes)
• Dealing with sensitive issues:
– Effect on the participant and the researcher
• Potential to reveal issues such as abuse or neglect
• Safety of the researcher whilst collecting data outside of the institution
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
How was informed consent,
confidentiality and data
protection managed?
Had the researcher
considered the ethical
implications of the
research?
Has approval been
sought from an
ethics committee?
School of Health Sciences and Social Work
Data analysis
How is data analysed in qualitative research?
How is rigour ensured in qualitative research?
School of Health Sciences and Social Work
What kind of data is produced by qualitative
methods?
AUDIO
Recordings of interviews
or focus groups
These will usually be
transcribed before
analysis
VISUAL
Videos of observations
(perhaps including audio)
Drawings
Photographs
WRITTEN
Documents
Diaries
Notes from
observations
School of Health Sciences and Social Work
Analysing qualitative data
The aim of data analysis is to go from a
large amount of data that is in no
particular order to a nice ordered
summary of what the data is telling us.
The most important thing is to make
sure that the summary really does
reflect what participants have said –
and that is the really tricky bit!
School of Health Sciences and Social Work
So how is data analysed?
DISPLAY THE DATA
If necessary, transcribe the recordings from interviews or focus groups
The data is read and any field notes added
Read the whole transcription and add analytical notes
Interviewer: So, you were given a prescription by your doctor for antibiotics, can you tell me what happened next? Mary: Well, I left the doctors and I meant to go straight to the chemist to put in my prescription. But I really was feeling very ill and it is a long way to the chemists. I only live about 2 minutes walk from the doctors so I went home instead. After I got home I started to feel even worse and I didn’t have anybody to ask to go and pick up my prescription for me. By the time I felt well enough to get to the chemist’s I thought it was probably not worth it anymore.
She looked
like she felt a
bit guilty
about this.
Followed this
up later on.
Intention?
Does she
mean this or
does she
think she has
to say it?
School of Health Sciences and Social Work
So how is data analysed?
DATA REDUCTION
Identify significant statements/units of meaning
Interviewer: So, you were given a prescription by your doctor for antibiotics, can you tell me what happened next? Mary: Well, I left the doctors and //I meant to go straight to the chemist to put in my prescription.// But I really was feeling very ill and it is a long way to the chemists. I only live about 2 minutes walk from the doctors so I went home instead. //After I got home I started to feel even worse// and I didn’t have anybody to ask to go and pick up my prescription for me. //By the time I felt well enough to get to the chemist’s I thought it was probably not worth it anymore.//
She looked
like she felt a
bit guilty
about this.
Followed this
up later on.
Intention?
Does she mean
this or does
she think she
has to say it?
School of Health Sciences and Social Work
So how is data analysed?
DATA INTERPRETATION
Group into categories
Develop into themes
Check representativeness of these from your transcripts
Find examples of these
Display
Qualitative research and analysis represents an inductive approach to research where we start with real-life data and work towards abstract ideas or general principles.
School of Health Sciences and Social Work
What do qualitative findings look like?
Qualitative data analysis often produces themes:
–Themes are used to describe the data in a concise and easy to understand manner
–Themes should be used to express the experiences of all participants
School of Health Sciences and Social Work
SYMPTOM EXPERIENCE
FEAR
BLAME AND STIGMA
Patients had not
realised that their
symptoms were
indicative of
anything serious
Patients delayed
going to the doctor
because they were
scared of finding out
that they had cancer
or were scared of
the treatments.
KNOWLEDGE
Patients did not
have much
knowledge about
lung cancer and its
symptoms
Patients were worried
that they would be
blamed or
stigmatised if they
were diagnosed with
lung cancer.
CULTURE
Some patients,
especially males,
felt the need to be
stoic about their
illness
Tod, Craven & Allmark (2008), looked at the reasons for diagnostic delay in patients with lung cancer and found the following themes.
School of Health Sciences and Social Work
Dependability and confirmability
How do we check the dependability and confirmability of the reported
findings?
• Can we follow the decision trail?
• Is there an audit trail?
• Have I been given enough information about the methods used?
• Is there enough detailed description for us to interpret the meaning and context of what is being researched?
Can we trust the analysis?
How careful was the
researcher to prevent bias
affecting their analysis?
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
Is there an in-
depth description
of the analysis
process?
Is it clear how the
themes were derived
from the data?
Do you know how the
presented quotes
were chosen?
Is there enough data to
support the findings?
Does the researcher
consider their influence
on the analysis and
presentation of
findings?
School of Health Sciences and Social Work
Credibility
Are the findings based on the data? Or on what
the researcher expected to
see?
• Did the researcher have prolonged engagement
with the participants or phenomenon?
• Did the researcher attempt to triangulate their
findings?
• Did the researcher discuss their data and
interpretations with a peer(s)?
• Did the researcher check for negative cases?
• Did the researchers send the findings to
participants to ask their opinion?
• Can you see any actual data (quotes)?
• Do you know why they reported those particular
quotes?
• Do the author’s arguments make sense?
How do we check the credibility of the
reported findings?
Do we believe the findings?
School of Health Sciences and Social Work
Reading qualitative research
Reading qualitative research: Important questions
Does the researcher
present data that
contradicts their
arguments?
Are the
findings
explicit?
Has the researcher
discussed
credibility?
Are the findings
related back to the
original question?
School of Health Sciences and Social Work
Value of the research
What makes for valuable qualitative research?
School of Health Sciences and Social Work
Can you tell if the findings are
transferable to your practice? • Is this relevant to our practice?
• Is there a description of the sample so you can decide it is relevant?
• Are there explanations as to what is going on in a particular social location/operation of a set of social processes?
• Does it provide comparisons that are relevant in “supporting your argument”