![Page 1: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/1.jpg)
QUALITATIVE EVALUATION OVERVIEW
Kim KoesterDirector of Qualitative ResearchAIDS Policy Research CenterCenter for AIDS Prevention StudiesUCSF
![Page 2: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/2.jpg)
Objectives
To teach you to understand the application of and methods associated with qualitative evaluation
To provide an overview a peek “under the hood” of the qualitative analytic process.
![Page 3: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/3.jpg)
Outline of talk
1. Explain qualitative research process1. Methods2. Procedures 3. Sampling4. Analysis
2. Demystify the analytic activities 1. Briefly describe different ways to approach qualitative
data analysis.3. Dig in to practical issues
1. Talk about resources required2. Touch on issues that must be addressed in writing up
data for publication
![Page 4: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/4.jpg)
Questions for you
What comes to mind when you think of qualitative methods?
Who plans to incorporate qualitative methods into your evaluation plan for this class?
Has anyone participated in a qualitative research project, as either an informant or as a member of a research team?
![Page 5: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/5.jpg)
What is Qualitative Research?
Qualitative research is an approach to scientific inquiry that relies on more
naturalistic, humanistic and interactive processes. The methods are primarily
language based, with data in the form of words rather than numbers.
Joan S. Ash and Kenneth P. Guappone 2007
![Page 6: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/6.jpg)
Qualitative Inquiry Areas of Emphasis Emphasizes a holistic perspective
Strive to understand a program and situation as a whole
Emphasizes the importance of understanding the meanings of human behavior and the social-cultural context of social interaction.
Emphasizes the emic perspective
![Page 7: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/7.jpg)
Qualitative researchers empathize and identify with
the people they study in order to understand how those
people see things. Taylor and Bogdan
![Page 8: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/8.jpg)
Qualitative Methods
Geared towards exploration, discovery, inductive logic
Defined in comparison to deductive research that begins with an hypothesis.
![Page 9: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/9.jpg)
Comparing Qualitative & Quantitative Methods
Inductive Small n Purposeful sample Open-ended
questions, geared towards discovery
Seeking holistic understanding
Not generalizable Investigates how or
why
Deductive Large n Random sample Closed-ended
questions, predetermined measures
Individual variables Generalizable Investigates how
many when, where
![Page 10: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/10.jpg)
Qualitative Techniques
In-depth interviewing (IDI) Key informant interviews Semi-structured interviews Narrative interviews
Life history interviews Sexual history interviews Critical incident interviews
Focus groups or Discussion groups Ethnography
Participant observation – shadowing/tag alongs
![Page 11: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/11.jpg)
Selecting the method
The method you select depends entirely on the research question.
Give me an example and we’ll talk through the best method.
What methods used in the articles you read for today?
![Page 12: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/12.jpg)
Sampling
Sample size in qualitative research vary from project to project; “there are no rules for sample size”
Purposive or purposeful sampling is the most common strategy in applied research/evaluation.
Purposeful sampling relies on the selection of information-rich cases; information rich cases allow learning a great deal about the issues that are of central importance to the evaluation
Patton delineated 16 different types of sampling frames
![Page 13: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/13.jpg)
Designing Interview Guides
The purpose of the interview guide is to list the questions or areas of inquiry to be approached in the interview setting. Semi-structured interviews In-depth interviews
![Page 14: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/14.jpg)
Example Interview Guide
DOMAIN: PERSONAL HISTORY/LIFE CIRCUMSTANCES INTENT: The intent of this section is to learn about
what has happened since the person was released and about how they are doing, in general.
QUESTIONS:Tell me how you have been meeting your day-to-day survival
needs, for example, housing, money, food, etc.?Tell me about your experiences re-connecting with friends and
family since you got out of prison.
![Page 15: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/15.jpg)
![Page 16: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/16.jpg)
Qualitative Analysis
Qualitative analysis is the process by which you turn your data into findings.
What are you doing when you are analyzing data? Reducing the sheer volume of raw data Separating out trivia from significance Identifying significant patterns
![Page 17: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/17.jpg)
Qualitative Analysis
Driven by goals and aims of the evaluation Describe program Elucidate program dynamics Identify patterns Confirm or disconfirm quantitative findings
![Page 18: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/18.jpg)
Key analytic activities
Thinking Reading Noticing Reacting Learning Producing insights Reading Sharing & discussing
•Refining insights•Reading •Labeling •Producing insights •Labeling•Reading•Parsing•Assembling•Understanding•Interpreting •Reading •And so on
![Page 19: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/19.jpg)
Phenomenology
What is the meaning, structure, and essence of the lived experience of this phenomenon for this person or group of people?
Goal: to elucidate meaning of lived experience
![Page 20: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/20.jpg)
Narrative Analysis
What does this narrative or story reveal about the person and world from which it came? How can this narrative be interpreted so that it provides an understanding of and illuminates the life and culture that created it? Comparing stories and cases to create meaning
Goal: to interpret stories to reveal cultural patterns
![Page 21: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/21.jpg)
Grounded Theory
What theory emerges from a systematic comparative analysis and is grounded in fieldwork so as to explain what has been and is observed? Theoretical sampling Theoretical saturation Open and axial coding
Goal: to generate explanatory theory from social processes
Best uses: unstructured in-depth interviews, structured data sets with rich, lengthy narratives
![Page 22: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/22.jpg)
Framework Analysis
Commonly used in applied policy research, this approach involves a set of distinct, yet highly interconnected analytic stages.
Familiarization w/data Identify thematic framework Indexing or coding Charting Mapping and Interpretation
Goal: to follow a transparent analytic process Best uses: interdisciplinary, team-based
research, in depth interviews and focus group data
![Page 23: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/23.jpg)
![Page 24: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/24.jpg)
Content Analysis
• Involves tagging a set of texts or other artifacts with codes that are derived from theory or from prior knowledge and then analyzing the distribution of the codes, usually statistically
• Once codes are developed, they become the unit of analysis rather than the utterances to which codes are assigned
• Goal: begin with deductive application of a priori codes, but remain open to inductive revelations.
• Best uses: Structured in-depth interviews, focus group
![Page 25: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/25.jpg)
Software
Qualitative software packages exist to facilitate the organization of qualitative data.
Dedoose single user $(15 per month)
Atlas.ti single user $1800 Multi-user (5) - $6700
Nvivo single user - $1300 Multi-user (5) - $5200
![Page 26: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/26.jpg)
Simply observing and interviewing do not ensure that the research is qualitative; the qualitative researcher must also interpret the beliefs and behaviors of participants.
Valerie Janesick
![Page 27: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/27.jpg)
Disposition Ideas
Characteristics of analysts: Significant amount of patience and
tolerance for reading and re-reading data Able to sit with ambiguity Non-linear orientation Creative
![Page 28: QUALITATIVE EVALUATION OVERVIEW Kim Koester Director of Qualitative Research AIDS Policy Research Center Center for AIDS Prevention Studies UCSF](https://reader035.vdocuments.net/reader035/viewer/2022062409/56649cf45503460f949c22e3/html5/thumbnails/28.jpg)