Mixed-method design

Chong Ho Yu

Paradigm war

• In the past qualitative and quantitative researchers could not get along.

• Qualitative researchers criticized quantitative research:

• Based on narrow-minded philosophy: positivism

• Reductionism: reduce vibrant and rich human phenomena to objects (numbers)

• Is it true?

What is positivism?

Vienna Circle

Werkmeiste

(a) Knowledge is knowledge only because of its

form. Content is non-essential.

(b) A proposition is meaningful if only if it can be verified.

(c) There is only empirical knowledge.

(d) Metaphysics are meaningless.

Vienna Circle

Werkmeiste

(e) All fields of inquiry are parts of a

unitary science: physics.

(f) The propositions of logic are tautologies.

(g) Mathematics can be reduced to logic

Logical positivism

Ian Hacking (a) an emphasis on verification, (b) pro-observation (c) anti-cause (d) downplaying explanation (e) anti-theoretical entities (f) anti-metaphysics

Verification

A statement is meaningless if verification is not possible or the criteria for verification are not clear.

e.g. God love us Humans carry original sins. He is moved by the Holy Spirit.

Pro-observation

Reality refers to experience

Science is built upon empirical laws.

Anti-cause

Logical atomist Russell (1913) explained relationships in terms of functions.

For example, Y = a + bX can be rewritten as X = (Y - a)/b. Thus, X could not be viewed as a cause of Y because the positions of X and Y could be swapped around the equation.

Russell’s quote

All philosophers imagine that causation is one of the fundamental axioms of science, yet oddly enough, in advanced sciences, the word ‘cause’ never occurs…The law of causality, I believe, is a relic of bygone age, surviving, like the monarchy, only because it is erroneously supposed to do no harm.

Downplaying explanation

The covering law model, introduced by Hempel (1965), illustrates how positivists seek explanation in terms of regularity.

In Hempel's view, to explain an event (S), a law (L) is applied and some particular facts (F) are observed to link S and L.

This deductive approach is descriptive in nature and thus does not generate any new knowledge

Anti-theoretical entities & Anti-metaphysics

Logical positivists are skeptical of unobservable and theoretical entities.

Logical positivists deny the existence of metaphysical and transcendental reality

Reductionism

Psychology → Biology → Chemistry → Physics → Math → Logic

Paradigm war is unnecessary!

• Modern quantitative research does not have MOST of the characteristics found in logical positivism!

• Verification

A statement is meaningless if verification is not possible or the criteria for verification are not clear.

Statistics is based on many unproven assumptions.

Paradigm war is unnecessary!

• Anti-cause: Quantitative researchers look for causal inferences!

• Downplaying explanation: Quantitative researchers look for theoretical explanations!

• Anti-theoretical entities: Psychologists study latent constructs.

• Reductionism: Psychologists look at mulit-dimensional constructs.

But paradigm war continues…

2004 National Research Council (NRC) released a report to suggest directions of research methodology:

• Research should inform policy and practice

• Research should be scientifically based

• Research should be supported by evidence

But paradigm war continues…

Some critics said that

• the NRC committee does not include under-represented groups in the decision process.

• The NRC committee has a narrow view of science

• Qualitative methods are not highly regarded.

• It is a continuation of No Child Left behind Act; the government wants to take more control of the citizens by setting the agenda.

Mixed-method

Why either-or? Why not both?

Cronbach, the famous statistician who introduced “Cronbach coefficient Alpha” and “construct validity.”

When he studied causal inference, he looked into the more qualitative methods of the ethnographer, historian, and journalist.

Mixed-method Taxonomy I

Parallel/simultaneous studies: employ the qualitative and quantitative methods concurrently.

Equivalent status designs: conducts the study using both the quantitative and qualitative approaches equally.

Mixed-method Taxonomy I

Dominant-less dominant studies: conducts the study "within a single dominant paradigm with a small component of the overall drawn from an alternative design" (Creswell, 1995, p.177).

Designs with multilevel use of approaches: use different methods at different levels of data, e.g. data could be analyzed qualitatively at the student level, qualitatively at the class level, quantitatively at the school level and qualitatively at the district level (Tashakkori, & Teddlie, 1998, p.18).

Mixed-method Taxonomy II (Creswell and Plano Clark’s)

• Triangulation design

• quantitative and qualitative methods are concurrently employed

• to bring together different but complementary perspectives by placing emphasis on convergence, data transformation, validation, or multi-level models.

Mixed-method Taxonomy II (Creswell and Plano Clark’s)

• Embedded design: one data set provides a supportive role in a study based primarily on the other data type.

• Explanatory design: sequential mixed-method design utilizing the insight gained from qualitative data to explain significant (or non-significant) results, outliers, or surprising findings yielded from quantitative data.

Mixed-method Taxonomy II (Creswell and Plano Clark’s)

• Exploratory design: • A sequential approach. Unlike its explanatory counterpart,

the order of employing qualitative and quantitative procedures is reversed

• When the problem is ill-defined and the variables are unknown, employ qualitative methods to understand the problem rather than imposing a theory on the phenomenon.

• This design is particularly useful when the researcher needs to develop an instrument to collect data, but is not sure about what constructs should be measured and what questions should be asked (grounded theory)