ipdet module 10: data analysis and interpretation
TRANSCRIPT
![Page 1: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/1.jpg)
IPDET Module 10: Data Analysis and
Interpretation
SubevaluationsQualitative vs.
QuantitativeQualitativeQuantitative
Intervention or
Policy
![Page 2: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/2.jpg)
IPDET 22
Introduction
• Data Analysis Strategy
• Analyzing Qualitative Data
• Analyzing Quantitative Data
• Linking Quantitative Data and Qualitative Data
![Page 3: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/3.jpg)
IPDET 33
Data Collection and Analysis
Time
Hou
rs S
pent
Data Collection
Data Analysis
Pilo
t
![Page 4: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/4.jpg)
IPDET 44
Qualitative Analysis
• Best used when for in-depth understanding of the intervention
• Answers questions like:– Is the intervention being implemented according to
plan?– What are some of the difficulties faced by staff?– Why did some participants drop out early?– What is the experience like for participants?– Are there any unexpected impacts on families and
communities?
![Page 5: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/5.jpg)
IPDET 55
Quantitative Analysis
• Can be used to answer questions like?– What is the percent distribution?
– How do participants rate the usefulness and relevance of the intervention?
– How much variability is there in the data?
– What is the relationship between a program and the outcome measures?
– Are the results statistically significant?
![Page 6: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/6.jpg)
IPDET 66
Qualitative Data
• Description of program, process, and experiences
• To understand context of the situation
• To understand perceptions
• Research evolves as questions emerge
• Flexible design
![Page 7: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/7.jpg)
IPDET 77
Qualitative Data Analysis
• Used for any non-numerical data collected as part of the evaluation– unstructured observations– open-ended interviews– analysis of written documents– focus groups transcripts– diaries, observations
• Analysis challenging• Take care for accuracy (validity concern)
![Page 8: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/8.jpg)
IPDET 88
Making Good Notes
• Capture as much information as possible
• Pay close attention to language
• Write down observations
• Capture your immediate thoughts
• Leave time to write up notes immediately
![Page 9: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/9.jpg)
IPDET 99
Triangulation
• Can use three or more sources of information to verify and substantiate your data
• Examples:– interviews, focus groups, questionnaires
– questionnaires, available data, expert panels
– observations, program records, interviews
– interviews, diaries, available data
![Page 10: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/10.jpg)
IPDET 1010
Early Steps in Qualitative Analysis (1 of 3)
• While collecting data:– keep good records
– write up interview, impressions, notes from focus groups
– make constant comparisons as you progress
– meet with team regularly to compare notes and make adjustments
![Page 11: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/11.jpg)
IPDET 1111
Early Steps in Qualitative Analysis (2 of 3)
• Write contact summary report– one page summary after each major
interview or focus group
– main issues
– major information obtained
– what was the most interesting, illuminating, or important?
– what new questions need to be explored?
![Page 12: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/12.jpg)
IPDET 1212
Early Steps in Qualitative Analysis (3 of 3)
• Use tools to help you– create a subjectivity file with your own
reactions during the study, including your feelings, hunches, and reactions
– file your ideas that emerge as you proceed
– keep a file of quotations from the data collection
![Page 13: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/13.jpg)
IPDET 1313
Maintain an Iterative Dialogue
• Share information early and often with key informants
• Have others review early drafts with the intention of eliciting information, questions, other ways of interpreting data
![Page 14: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/14.jpg)
IPDET 1414
Drawing-out Themes and Patterns
• As you review, begin to make notes• Goal is to summarize what you have seen or
heard:– common words– phrases– themes– patterns
• Also identify where they are so you can find them again if you need to verify
• May want to use a spreadsheet
![Page 15: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/15.jpg)
IPDET 1515
Computer Help for Qualitative Data Analysis
• Software packages to help you organize data
• Search, organize, categorize, and annotate textual and visual data
• Help you visualize the relationships among data
![Page 16: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/16.jpg)
IPDET 1616
Examples of QDA
• N6 from QSR (previously called NUD*IST)
• Ethnograph
• Qualpro
• Hyperqual
• Anthropax
• Atlas-ti
![Page 17: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/17.jpg)
IPDET 1717
Controlling for Bias
• We tend to see what we want to see and may miss things that do not conform to our expectations
• Use well trained recorders
• Evaluators review documents and code them in themes
![Page 18: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/18.jpg)
IPDET 1818
Concluding Thoughts on Qualitative Data
• Qualitative data collection is not the easy option– labor intensive and time consuming
– reliability among coders, using a coding scheme is essential
• Can reveal valuable information
![Page 19: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/19.jpg)
IPDET 1919
Quantitative Data: Statistics
• Quantitative data are analyzed with statistics– descriptive statistics: used with census or
non-random sample data
– inferential statistics: used with random sample data
![Page 20: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/20.jpg)
IPDET 2020
Descriptive Statistics
• Describes the frequency and/or percentage distribution of a single variable
• Tells how many and what percent
• Example:– 33% of the respondents are male and 67%
are female (table on next slide)
![Page 21: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/21.jpg)
IPDET 2121
Example of Descriptive Statistics in a Table
Male Female Total
Number Percent Number Percent Number
100 33% 200 67% 300
Write up: Of the 300 people in this program, 67% are women and 33% are men.
How many men and women are in the program?
Table 11.5: Distribution of Respondents by Gender
Source: Fabricated Data
![Page 22: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/22.jpg)
IPDET 2222
Distributions
• Measures of central tendency– how similar are the data?
– example: How similar are the ages of this group of people?
• Measures of dispersion– how dissimilar are the data?
– example: How much variation in the ages?
![Page 23: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/23.jpg)
IPDET 2323
Measures of Central Tendency
• The 3-M’s– mode: most frequent response
– median: mid-point of the distribution
– mean: arithmetic average
• Which to use depends on the type of data you have– nominal, ordinal, interval/ratio
![Page 24: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/24.jpg)
IPDET 2424
Nominal Data
• Data of names or categories
• Examples: – gender (male, female)
– religion (Buddhist, Christian, Jewish, Muslim)
– country of origin (Burma, China, Ethiopia, Peru)
• Use mode as a measure of central tendency
![Page 25: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/25.jpg)
IPDET 2525
Ordinal Data
• Data that has an order to it but the “distance”between consecutive responses is not necessarily the same
• Lacks a zero point
• Examples: – opinion scales that go from “most important” to “least
important” or “strongly agree” to “strongly disagree”
• Use mode or median as a measure of central tendency
![Page 26: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/26.jpg)
IPDET 2626
Interval/Ratio Data• Data of real numbers, numbers with a zero
point and can be divided and compared into other ratio numbers
• Examples:– age, income, weight, height
• Use mode, median, or mean as a measure of central tendency — the choice depends on the distribution– for normal data, mean is best – for data with few high – or - few low scores, median
is best
![Page 27: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/27.jpg)
IPDET 2727
Calculating
• Mode: the one with the most
• Median: place in order then count down to half way
• Mean: (most people think of it as the average)
![Page 28: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/28.jpg)
IPDET 2828
Example Data
Country % Urban
Bolivia 65
Algeria 60
Central Africa Republic 41
Georgia 61
Panama 58
Turkey 75Source: Fabricated Data
Table 11.7: Sample Data
![Page 29: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/29.jpg)
IPDET 2929
Example Calculations for % Urban Data
• Mode: no mode, all have only one data point
• Median: total entries is 6, with data in order two middle scores are (61 and 60) ÷ 2 = 60.5
• Mean:(65+60+41+61+58+75) ÷6 = 60
![Page 30: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/30.jpg)
IPDET 3030
Measures of Dispersion
• Range– difference between the highest and lowest
value– simple to calculate, but not very valuable
• Standard deviation– measure of the spread of the scores
around the mean– superior measure, it allows every case to
have an impact on its value
![Page 31: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/31.jpg)
IPDET 3131
Example Calculation for Range
• Range: high score – low score = rangerange = 75 – 41
range = 34
![Page 32: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/32.jpg)
IPDET 3232
Normal Curve (Bell)
y
x0
Fre
quen
cy
Value
![Page 33: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/33.jpg)
IPDET 3333
Standard Deviation
y
x0
Mean One standard deviation from the mean
68%
95%
Two standard deviations from the mean
98%
Three standard deviations from the mean
![Page 34: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/34.jpg)
IPDET 3434
Calculating Standard Deviation
• Calculating is time consuming
• Can use statistical programs:– SPSS
– Excel or other spreadsheet program
![Page 35: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/35.jpg)
IPDET 3535
Guidelines for Analyzing Quantitative Survey Results
1 Choose a standard way to analyze the data and apply it consistently
2 Do not combine the middle category with each side of the scale
3 Do not report an agree or disagree category without also reporting the strongly agree agree or strongly disagree category
4 Analyze and report percentages (or numbers)
5 Provide the number of respondents for an anchor
6 If there is little difference in the data, raise the benchmark
7 Like any art or skill, it gets easier with training and practice
![Page 36: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/36.jpg)
IPDET 3636
Common Descriptive Statistics
• Frequencies
• Percent
• Mean
• Median
• Mode
• Percent
• Ratio
• Comparisons
![Page 37: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/37.jpg)
IPDET 3737
Describing Two Variables at the Same Time
• Two variables at once
• Example: What percent were boys and what percent were girls in hands-on and traditional classes?
![Page 38: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/38.jpg)
IPDET 3838
Example Two Variables at the Same Time
Hands-on Hands-on Traditional Traditional
Boys 28 55% 34 45%
Girls 22 45% 41 55%
Total 125 N=50 100% N=75 100%
Source: Fabricated Data: 2004 Survey
![Page 39: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/39.jpg)
IPDET 3939
Two Variables with Crosstabs
• Cross tabulation (crosstab)– presented in a matrix format
– displays two or more variables simultaneously
– each cell shows number of respondents
![Page 40: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/40.jpg)
IPDET 4040
Example Crosstabs
Hands-on Traditional Total %
Boys(n=45)
45% 55% 100%
Girls(n=80)
35% 65% 100%
N=125 Source: Fabricated Data
![Page 41: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/41.jpg)
IPDET 4141
Variables
• Independent– Variable which you believe explains a
change in the dependent variable
– Program evaluation: the program
• Dependent– Variable you want to explain
– Program evaluation: the outcomes
![Page 42: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/42.jpg)
IPDET 4242
Example: Comparison of Means
Mean Income
Women 27,800 SA Rand
Men 32,400 SA Rand
-dependent variable: annual income
-independent variable: gender
![Page 43: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/43.jpg)
IPDET 4343
Measure of Relationship
• How strongly variables are related, reported differently
• Measures of association– range from zero to 1
• Measures of correlation– range from –1 to +1
![Page 44: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/44.jpg)
IPDET 4444
Interpretation of Correlation
• Measures of correlation:– perfect relationship: 1 or –1
• closer to 1 or –1: strong relationship
• .5: moderate/strong (maybe as good as it gets)
– closer to zero: no relationship• .2 - slight/weak relationship
![Page 45: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/45.jpg)
IPDET 4545
Direct Relationship
• Plus sign +– both variables change in the same
direction
– example:• as driving speed increases, death rate goes up
![Page 46: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/46.jpg)
IPDET 4646
Inverse Relationship
• Minus sign -– both variable change but in the opposite
direction
– example:• as age increases, health status decreases
![Page 47: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/47.jpg)
IPDET 4747
Inferential Statistics
• Used to analyze data from randomly selected samples
• Risk of error because your sample may be different from the population as a whole
• To make an inference, you first need to estimate the probability of that error
![Page 48: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/48.jpg)
IPDET 4848
Statistical Significance Tests
• Tools to estimate how likely the results are in error
• Called tests of statistical significance– to estimate how likely it is that you have
gotten the results you see in you analysis by chance alone
![Page 49: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/49.jpg)
IPDET 4949
Statistical Significance
• Benchmark of .5%– .05 Alpha level or P value
• It means we are 95% certain that our sample results are not due to chance– or
• The results are statistically significant at the .05 level
• Most reports do not go beyond .5
![Page 50: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/50.jpg)
IPDET 5050
Chi Square and t-Test
• One of the most popular statistics– easy to calculate and
interpret
• Used to compare two sets of nominal data (i.e marital status and religious affiliation)
• Used to compare two ordinal variables or a combination of nominal and ordinal variables
• Used to determine if one group of numerical scores is statistically higher or lower than another group of scores– two means
Chi Square t-Test
![Page 51: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/51.jpg)
IPDET 5151
Hypothesis Testing
• Research hypothesis is your best guess as to the relationship between variables– Example: there is a difference between the per
capita incomes of men and women in South Africa
• Null hypothesis is always a statement that “there is no difference” or “no impact”between our variables– Example: there is no difference between the per
capita incomes of men and women in South Africa
![Page 52: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/52.jpg)
IPDET 5252
Remember:
• A significant test is nothing more than an estimate of the probability of getting the results by chance if there really is no difference in the population
![Page 53: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/53.jpg)
IPDET 5353
Linking Qualitative and Quantitative Data
• Should qualitative and quantitative data and associated methods be linked during study design? – How?
– Why?
![Page 54: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/54.jpg)
IPDET 5454
Qualitative-Quantitative Linkages
• Confirmation or corroboration – triangulation
• Richer detail
• Initiate new lines of thinking
• Expand the scope
![Page 55: IPDET Module 10: Data Analysis and Interpretation](https://reader030.vdocuments.net/reader030/viewer/2022012705/61a69a9566c6981f88221f73/html5/thumbnails/55.jpg)
To continue on to theNext Module click here
To return to theTable of Contents click here