Download - ECNU Educational Research Methodology
ECNU Educational Research Methodology
Lecture 2 Research design in quantitative research
2016/4/18 1 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Review of last class
2016/4/18 Fang Chen ECNU 陈芳 华东师大 2
ECNU Educational Research Methodology
QUANTITATIVE RESEARCH
Quantitative research has its roots in positivism, which Gall, Gall and Borg (2003) have defined as: The epistemological doctrine that physical and social reality is independent of those who observe it, and that observations of this reality, if unbiased, constitute scientific knowledge (p.632).
2016/4/18 3 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Research Design in quantitative research
Research Design in quantitative research tends to be structured and prescriptive, much more so than in qualitative research.
The outcomes are typically expressed as
numbers and enables the researchers to make valid interpretations through comparisons and partitioning of those numbers.
2016/4/18 4 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Purpose of design
What is the purpose of research design? 1. Provide answers to questions
2. Control variance This, to a large extent, unique to quantitative
research.
2016/4/18 5 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Controlling variance
Controlling variance means creating conditions that allow the researcher to get a clear look at the variable of interest by limiting or eliminating the influence of some variables and explaining the influence of others.
2016/4/18 6 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Example
Research statement: A study of the effect of teaching method on the
performance of high school students enrolled in chemistry
Method 80%
Method 55%
2016/4/18 7 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Controlling Variance
1. Randomization
2. Building conditions into the design as independent variables
3. Holding conditions constant
4. Statistical adjustments
2016/4/18 8 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
1. Randomization
Recall that at least some of the variability may be from the differences between my groups.
If this is the case, we want it to be only because of the teaching method, NOT because of other factors Randomly pick participants from population Randomly place in groups
2016/4/18 9 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
2016/4/18 Fang Chen ECNU 陈芳 华东师大 10
ECNU Educational Research Methodology
The randomization essentially equalized the groups with respect to ability level, but it also equalized them with respect to other variables such as motivation, attitude, to name a few.
Although this process distributes the effect of ability level evenly, it does not allow the researcher to quantify the influence of ability level on the scores of the chemistry test.
Figure 4.1
2016/4/18 Fang Chen ECNU 陈芳 华东师大 11
ECNU Educational Research Methodology
2. Build in Factors
If there is something that could affect the outcome, we can include it in the design.
They also account for variability, therefore reducing what is considered random noise.
Figure 4.3 Build in the ability factor
2016/4/18 12 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
2016/4/18 Fang Chen ECNU 陈芳 华东师大 13
ECNU Educational Research Methodology
Issues
At times, adding additional factors is not possible
Need more examinees
May be too expensive
May take too much time
2016/4/18 14 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
3. Holding Factors Constant
We could constrain some factors to be constant.
Think of student ability Only use students with IQ score between 100-108.
2016/4/18 15 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
One danger is that you will lose external validity. Generalizability of research findings
Issues
2016/4/18 Fang Chen ECNU 陈芳 华东师大 16
ECNU Educational Research Methodology
4. Statistical control
Use all the data collected, Statistically control the factors that are
relevant to the study if measure of those factors exist
Statistical control would most likely account for more variance than building in student ability as a two-category, independent variable.
2016/4/18 17 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Example
2016/4/18 Fang Chen ECNU 陈芳 华东师大 18
MethodoreChemTestSc 10 ββ +=
IQMethodoreChemTestSc 210 βββ ++=
TeacherMethodoreChemTestSc 210 βββ ++=
MethodoreChemTestSc 10 ββ +=
ECNU Educational Research Methodology
Adequate scores on a control variable may not be available.
Statistical assumptions must be met; if they are not tenable, the statistical procedures should not be used.
Issues
2016/4/18 Fang Chen ECNU 陈芳 华东师大 19
ECNU Educational Research Methodology
Extending the chemistry example: In stead of using only 90 students, we used 180
within a school. Students come from different elementary schools
with various science background. The three teaching method is still the independent
variable of interest Three teachers teaching 6 classes in total, with 2
classes per teaching method IQ scores from 8th grade are available and
controlled
Using procedures for control in combination
2016/4/18 Fang Chen ECNU 陈芳 华东师大 20
ECNU Educational Research Methodology
Figure 4.7
2016/4/18 Fang Chen ECNU 陈芳 华东师大 21
1. Randomization
2. Building conditions into the design as independent variables
3. Holding conditions constant
4. Statistical adjustments
ECNU Educational Research Methodology
Characteristics of Good Research
Freedom From Bias
Freedom from Confounding
Control Extraneous Variables
Statistical Precision
2016/4/18 22 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Characteristics of Good Research
Freedom From Bias E.g. higher ability students assigned to one method Students volunteered into a program may have more
motivation
Freedom from Confounding If each teacher uses one method, method effectiveness
may be confounded with teacher effectiveness
Control Extraneous Variables Statistical Precision 2016/4/18 23 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
Characteristics of Good Research
Freedom From Bias Freedom from Confounding Control Extraneous Variables Randomization Build the factor into design Hold the factor constant Statistical control
Statistical Precision Statistical precision is increased with larger samples
and when additional independent variables are built into the research design.
2016/4/18 24 Fang Chen ECNU 陈芳 华东师大
ECNU Educational Research Methodology
When you read any article
Think about: 1. Independent and dependent variables. 2. Controlling variance. 3. Are there other factors that they might have
wanted to account for? 4. Does the study have bias or confounding
variables?
2016/4/18 25 Fang Chen ECNU 陈芳 华东师大