data collection & analysis presentation
DESCRIPTION
Data Collection & Analysis PresentationTRANSCRIPT
DATA COLLECTION AND ANALYSIS TCUP Fundamentals of Education Research Workshop Mercy Mugo October 31, 2014
Quality Education for Minorities (QEM) Network
The Research Linkages
Identify the Research Question(s)
Determine the Research Methodology
Collect the Data
Data Analysis and Report of Findings
What is Data Collection?
¨ A detailed plan of procedures that aims to gather data for the purpose of answering a research question(s)
Quantitative Data
¨ Collected in standardized manner ¨ Presented in numerical format ¨ Analyzed using statistical techniques ¨ Results more generalizable
Qualitative Data
¨ Collected in natural setting ¨ Presented in narrative format ¨ Thick in detail and description ¨ Analysis often emphasizes understanding
phenomena as they exist
Data Collection Methods
♦ Surveys ♦ Tests ♦ Rubrics ♦ Checklists
♦ Observations ♦ Interviews ♦ Focus Groups ♦ Document Review/Analysis ♦ Case Studies ♦ Photographs, Videos
Qualitative Methods Quantitative Methods
Cultural Responsive Data Collection Methods
¨ Talking Circles ¨ Visiting ¨ Performance-based Assessment ¨ Appreciative Inquiry
Pros and Cons of Quantitative Methods
Method Advantages Disadvantages
Surveys • Inexpensive • Good for gathering
descriptive data • Cover a wide range of
topics • A variety of software for
analysis
• Self-report may lead to bias
• Data lack depth • No control for
misunderstood questions, missing data
Pros and Cons of Quantitative Methods
Method Advantages Disadvantages
Tests • Objective information on what the test taker knows and can do
• Can be constructed to match level of skills
• Easy to administer/score • Provide “hard” data • Accepted by public
• Time consuming • Biased against some
groups • May be subject to
corruption via coaching or cheating
Pros and Cons of Qualitative Methods Method Advantages Disadvantage
Interviews • Yield richest data, insights
• Permit face-to-face contact
• In-depth exploration of topics
• Allow interview to explain or clarify questions
• Expensive and time consuming
• Need well trained interviewers
• Interviewee may distort information
• Large volume of information may be difficult to transcribe
Pros and Cons of Qualitative Methods Method Advantages Disadvantage
Focus Groups
• Useful to gather different viewpoints and new insight
• Less time required • Subject matter is not
sensitive
• Not suitable for generalization
• Require qualified facilitator
• Peer pressure may inhibit responses
Pros and Cons of Qualitative Methods Method Advantages Disadvantage
Observations • Provide direct information
• Permit observer to enter into and understand context
• Exist in natural, flexible setting
• Good for identifying unanticipated outcomes
• Expensive • Time consuming • Need well qualified
observers • Selective perception of
observer may distort data • Behavior observed may not
be representative of a group/situation
Pros and Cons of Qualitative Methods Method Advantages Disadvantage
Document Analysis
• Inexpensive • Available locally • Grounded in setting
and language in which they occur
• Provide information on historical trends
• May be incomplete • May be inaccurate • Challenges locating
suitable documents • Time consuming
When Choosing Methods, Consider…
¨ Purpose of study (research questions) ¨ Respondents/data sources ¨ Resources available ¨ Type of information needed ¨ Value of using multiple methods ¨ Importance of ensuring cultural appropriateness
Other Considerations………..
¨ Understand the community ¨ Involve community ¨ Allow time to establish relationships ¨ Take care in constructing and asking questions ¨ Respect cultural protocols ¨ Provide incentives
DATA ANALYSIS
What is Data Analysis?
¨ Summarizing data into manageable format to communicate its meaning
¨ Reflecting on the data and searching for patterns
¨ Seeking out the story in the results
Types of Statistics in Quantitative Research
¤ used to organize, describe, and summarize a set of data.
Descriptive Statistics Inferential Statistics
¨ used to draw inferences about characteristics of a population based on what is known about a sample drawn from that population.
Types of Statistical Analyses in Quantitative Research
¨ Measures of central tendency ¤ Mean, median, mode
¨ Measures of variability ¤ Range, variance,
standard deviation
Descriptive Statistics Inferential Statistics
¨ Parametric Tests ¤ t-tests, Analysis of variance
(ANOVA), Regression analysis
¨ Non-parametric Tests ¤ Chi-Square test; the sign
test
Data Analysis in Qualitative Research
¨ Read all data, get a sense of the whole ¨ Code data, tag items with same meaning using
using unique codes ¨ Identify patterns/themes among the codes ¨ Represent themes (writing, visual, etc.) ¨ Interpret and make meaning out of the data
Qualitative Data Analysis
Mixed Methods Designs
¨ Utilizes both quantitative and qualitative data collection methodologies
¨ Major designs: ¤ Convergent Design ¤ Explanatory design ¤ Exploratory design ¤ Embedded Design
Convergent Design
¤ Collect qualitative and quantitative data concurrently
¤ Analyze the two datasets separately ¤ Mix the two databases by merging results
during interpretation
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods
Explanatory Design ¤ Starts by collecting and analyzing quantitative
data ¤ Uses quantitative results to inform subsequent
qualitative inquiry ¤ Uses quantitative results to shape the qualitative
research questions, sampling, and data collection
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods
Exploratory Design ¤ Starts by collecting and analyzing qualitative
data ¤ Utilizes qualitative results to build the quantitative
phase ¤ Connects the phases by using qualitative results to
shape the quantitative research questions, variables, and instrument
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods
Embedded Design
Qualitative (or quantitative) Data Collection and Analysis
(before, during, or after)
Quantitative (or Qualitative) Design
Quantitative (or Qualitative) Data Collection and Analysis Interpretation
Validity of Results ¨ Validity
¤ Most important characteristic of the study/assessment results
¤ Concerned with the appropriateness of the interpretations made from assessment results
¤ Specific to the interpretation being made and to the group being assessed
Reliability of Results
¨ Reliability ¤ Concerned with how well the results can be
replicated ¤ Would a particular technique (or survey) yield
the same results each time? ¤ Reliability does not ensure accuracy
Working with Identifiable Data
¨ Maintaining confidentiality
¨ Presenting accurate information ¨ How can we reconcile these two conflicting
dynamics?
Ways to Address Small Sample Size
¨ Report results in aggregate --- across several samples to maintain confidentiality
¨ Allow access of raw data to user who may need it for e.g. evidence-based decision-making, policy-making, budgeting, proposal development, etc. ¤ IRB approval ¤ Participants permission to release information/data
Example: Impact of Suppression of Small Data Cells
¨ Negatively affect underrepresented groups (loss of information)
¨ Value of data significantly diminished ¨ Prevent access to information essential to
providing highly needed opportunities A report on the series of outreach meetings on the “Impact of the Suppression of Small Data Cells” in the Survey of Earned Doctorates (SED) Report (2009). Prepared by QEM for NSF’s Science Resources Statistics (SRS) Division.
Impact of Suppression Continued…
¨ Harm diversity-focused initiatives and minority-focused programs
¨ Difficult finding role models ¨ Difficult designing intervention strategies
A report on the series of outreach meetings on the “Impact of the Suppression of Small Data Cells” in the Survey of Earned Doctorates (SED) Report (2009). Prepared by QEM for NSF’s Science Resources Statistics (SRS) Division.
Questions
Thank you!