jemimah qualitative data collection
TRANSCRIPT
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
Qualitative Methods for Monitoring, Evaluation and Impact Assessment
Jemimah Njuki
INTERNATIONAL LIVESTOCK RESEARCH INSTITUTE
Open-ended questions to identify types of indicators or impacts
• May feed into survey questions
Provides interpretive power
• What as well as why
Exploring topics less amenable to survey questions
Express local voice and perceptions
Uses of qualitative approaches
Advantages of qualitative methods
Qualitative methods are effective at capturing these issues because
• Elaboration, and sometimes several tries and lengthy follow-up and probing, are needed to convey the questions adequately and understand answers
• People do not always tell the truth the first time: Getting candid information often requires time, trust, rapport, triangulation, observation
Discover unanticipated issues
Solicit local solutions to problems
Qualitative as complementary to quantitative tools
Identifies issues and questions for surveys and hypotheses for testing
Identifies response options for survey questions
Clarifies terms/language for use in surveys
Confirms validity of constructs and proxies
Triangulation, confirmation, contradiction
Explanation/interpretation of survey findings
• Depth, texture, context
Quantitative as a complementary to qualitative
Identifies stratification strategy
Provides community and household characteristics for sampling
Identifies issues for investigation
Triangulation, confirmation, contradiction
Determines prevalence of qualitative findings in wider population
Reveals representation of qualitative sample
Some misconceptions about qualitative approaches
It is easy to do and cheap, and anyone can do it.
“If you can talk to farmers, you can use qualitative approaches and tools”
With qualitative tools, all you need is to document the results. No analysis is needed—
”just write up the report, its basic stuff”
“You don‟t need a research design! Just talk to a few farmers and do a few focus group discussions”
A lack of understanding of the tools and approaches
“We are using a PRA tool”
Different types of tools
Mapping tools
PRA tools Interviews Ethnographic tools
Participatory impact diagrams
Diffusion Maps
Before and After resource /asset maps
Social network analysis
Transect Walks
Trend lines
Venn diagrams
Seasonal calendars
Focus Group Discussions
Community meetings
Ranking /Rating/Scoring
Community surveys
Semi-structured interviews
Unstructured d interviews
Key informant interviews
Organizational assessment
Case studies
Innovation Histories
Life Histories
Personal diaries
Useful for identifying important traits / criteria for organising issues, items by preference
Used mainly for technology evaluation, ranking of priority options e.g priority household income options, asset preferences
Can be used to compare preferences across groups e.g men and women,
Scoring /Ranking /Rating methods
Ranking Priority Assets for Men and Women
Scoring /ranking /Rating methods
Advantages
Can be used with symbols and counters especially with groups with low literacy levels
Can be done individually or in groups
Allows for group contribution of list to be ranked /rated and the criteria to use
Can be easily quantified
Disadvantages
Takes time especially when community groups identify their own lists and criteria for evaluation
Mapping Tools-Participatory Impact Diagrams
Mapping Tools: Before and After Maps
Before After
friend
DENGORE
Sister’s
husband
Mhaber
Brother
WOLENCHETI
TOWN
MAP LEGEND
Methods of Seed Dissemination
Gift (3kg or less)
Gift (3kg or more)
Exchange
Sale
Neighboring
Village
brother
mother brother
friend
friend
Wife’s father
DONI
10 km
6 km
MERKO
4 km
Mekanajo
Neighbor-
struggling widow
MOME
Mekanajo/
Mhaber
7 km
Mekanajo
75 km
Iddir
neighbor
Appendix 3b. Technology Diffusion / Social Network Mapping: Simplified Sample from Worka Village
Iddir
Iddir
Mekanajo
/Mhaber
Mapping Tools: Diffusion
Mapping tools
Advantages
Can be used with groups that have low literacy levels
Very engaging
Easy visual presentation
Can provide massive amounts of information that combine qualitative as well as simple numbers
Can be done on paper or on the ground
Disadvantages
Requires close facilitation
Time consuming
Can be dominated by those that can write (who holds the pen /chalk, stick?)
Focus Group Discussions: Description
A group discussion of approximately 6 - 12 persons guided by a facilitator, during which group members talk freely and spontaneously about a certain topic.
Its purpose is to obtain in-depth information on concepts, perceptions and ideas of a group
Can be useful to:
• Focus research and develop relevant research hypotheses by exploring in greater depth the problem to be investigated and its possible causes
• Formulate appropriate questions for more structured, larger scale surveys
• Help understand and solve unexpected problems in interventions
• Explore controversial topics
FGDs; Advantages and Disadvantages
Advantages
Cost- and time-efficient—cover more people
Can be a safe space for discussion of sensitive issues
Early identification of important issues
Discussions trigger ideas, recollections, opinions
Disadvantages
Less time to explore and probe
Inability to triangulate data on individuals/households
Louder and quieter voices, peer pressure
No very appropriate for sensitive topics
Page
16
FGDs Analysis: Analysis
Text Analysis
• QDA Miner
• Relational themes
• Identifying Key themes in FGDs.docx
Conducted with a fairly open framework which allow for focused, conversational, two-way communication
Starts with more general questions or topics. Followed by more specific probing questions
Not all questions are designed and phrased ahead of time
Uses an interview guide rather than a set of questions
Used to:
• Obtain specific quantitative and qualitative information from a sample of the population
• Obtain general information relevant to specific issues, (ie: to probe for what is not known)
• Gain a range of insights on specific issues
Semi-structured interviews; Description
Semi-structured interviews: Advantages and disadvantages
Advantages
Depth and detail
Rapport
Analytic power: Ability to relate data to other data at individual, household, and community levels
No peer pressure
Disadvantages
More time consuming and costly, so smaller sample sizes
No trigger and interaction effects of group
Pag
e
19
Most Significant Change
It is a story-based, qualitative and participatory approach to monitoring and evaluation
Involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these..
Can be used for different domains of change and provides case studies
„Looking back over the last 1 year, what do you think was the most
significant change in [particular domain of change]?‟
Can used for:
• program evaluation
• organizational review and evaluation.
• building community ownership through participatory evaluation
Most Significant Change
Advantages
Participatory, involves multiple stakeholders
Does not use pre-set indicators and therefore can capture unexpected /unanticipated changes
Disadvantages
Time consuming especially in selecting most significant change stories
Network Analysis
Set of integrated techniques to depict relations among actors and to analyze the social structures that emerge from the recurrence of these relations
Conducted by collecting relational data organized in matrix form.
Actors are depicted as nodes, and their relations as lines among pairs of nodes
Network Analysis: Advantages and Disadvantages
Advantages
Can be done individually or in groups
Data /information can be quantified (UCINET)
Can be done through visual tools (Venn diagrams) or short questionnaire survey
Disadvantages
Limited to evaluation of interactions
Requires multiple software (UCINET for analysis, Netdraw for network maps)
Analysis
Three stages of analysis
• During and post interview analysis (“follow your nose!”
• Field analysis
• Formal analysis
Communication
• Language
• Technical expertise
Field implementation issues
Sample selection
• For groups, sample stratification based on key variables such as gender, socio-economic status
Representation
• For semi-structured and case study interviews, representation of key interest groups
Triangulation
• Most qualitative tools not enough for evaluation by themselves, use of other tools for triangulation
Examples of use of Qualitative tools in evaluation Physical set up
• For group discussions, sitting arrangements, venues can influence discussion outcomes
Discussion guides /interview guides
• Semi-structured or unstructured guides with key issues /questions
Skills
• Good facilitation skills required for qualitative data collection
• Analytic skills
Recording /documentation
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
Thank You
INTERNATIONAL LIVESTOCK RESEARCH INSTITUTE