Lecture 2: Mapping Research

HEST 5001

Housekeeping

Next week (17th October) Julian Stribling will talk about research strategies.

Seminar groups: I will circulate a list for the seminar groups in weeks 7 & 8 at the end of this week. Write to me and tell me which paper you’ve picked.

Philosophies / Paradigms / DilemmasPositivism / Subjectivism / Critical realism / Postmodernism

Methodological CriteriaInternal validity / External validity / Reliability

Research Strategies / Designs Experiments / Surveys / Action research / case studies / ethnography / grounded

theory / phenomenology / mixed methods

Research MethodsQuestionnaires / interviews / observations / documents

Data AnalysisInferential statistics / descriptive statistics / content analysis / thematic analysis /

discourse analysis / interpretive phenomenological analysis

(Dyson & Brown, 2006:3)

Some research elements have been described at different conceptual levels, (e.g. phenomenology):

Philosophy (Porter, 1998)

Strategy (Denscombe, 2007)

Method (Moustakas, 1994)

Analysis (Smith et al., 1999)

Likewise, grounded theory is often referred to at different conceptual levels

Tensions and dualisms

Dyson & Brown (2006) highlight:

Some of the tensions (dualisms) that exist exist in research

That most researchers start at the level of strategy (design), depending on their research question and the context in which it it being undertaken.

The philosophical stance, data collection method and analysis are often implied by the choice of strategy

Important points:

All research necessitates “background assumptions” about the World

Whether researchers acknowledge these assumptions or not, they are important and influence the conduct of research, and what is considered as evidence and the relative importance of evidence.

Empiricism and Positivism

Empiricism This was a belief in the natural world, that was “out there” and can be seen as a reaction to medieval theological thought.

What do angels look like?

How many angels fit on a pin-head?

Dyson & Brown (2006, chapter 2)

Richard Napier1559-1634

John Napier1550-1617

Francis Bacon1561-1626

John Locke1632-1704

Christopher Wren1632-1723

Wren’s brains

Isaac Newton1642-1727

Empiricism can be viewed as a reaction to mediaeval debates - i.e. there is a world that we can observe

However, as individuals, we see can the same thing, but perceive them differently – duck or rabbit?

Adolphe Quetelet1796-1874

Positivism This is a belief that :

1 Social research is scientific in that it adopts the methods (and approaches) of natural sciences, and provides explanations in terms of laws

2 Requires that knowledge should be observable (disallowing theoretical concepts as knowledge)

3 “Competent observers” can agree on observations

Auguste Comte 1798-1857

Claude Bernard1813-1878

Lord Kelvin1824-1907

Unconscious bias We are selective in our perception, and not all external

stimuli are processed by our brains

Conscious bias This is where selective perception is deliberate, also

called political or moral bias. Refers to situations where researchers register certain data but do not report it, or are selective about what they report! In extreme cases entire data sets could be fabricated!

Observation is not theory-free

Filter through a standardised structured research instrument, e.g. a survey questionnaire

Positivism

Interpretivism (Subjectivism)

Subjectivism

If we assume that the real world is not independent of the researcher, or how the researcher interprets, i.e. that there isn’t a world “out there” that is measurable, but is an inter-play of a number of interactions

Consider 2 examples

WWI Solder (Thomas & Thomas, 1928)

If people define situations as real, then they are real in their consequences

Thomas, W.I. and Thomas, D.S. (1928) The Child in America: Behaviour Problems and Programs, New York: Knopf.

Body Dysmorphic Disorder Case of men treated by Dr Robert Smith. In these cases,

the body images of the men were so distorted that they felt they had unnecessary limbs, and each man ended up having a limb amputated. Smith operated on a man from England in 1997, and a man from Germany in 1999. Both men were healthy, aside from having Body Dysmorphic Disorder

Interpretivism (Subjectivism) Meanings, motives, perceptions

Positivism Subjectivism

Ontology (what)

Observable Meanings, motives, actions, feelings

Epistemology (how)

Observe relations between phenomena

Understand reality

Approaches Deductive Inductive

Quantitative Qualitative

Methods Comparison, statistics, structured interviews, structured observations

Naturalistic observations, participant observation, case studies. Unstructured / depth interviews From Dyson & Brown, 2006: 30

Other philosophical stances:

Critical (scientific) realism

Pragmatism

Post-modernism

Scientific Realism

0102030405060708090

Barbie Jets

BoysGirls

Influences on choice of toys

Scientific Realism

Pragmatism

Denscombe recently (2010) argued that the mixed method approach has recently emerged as a practical alternative to other approaches.

“… it is framed by a whole variety of practical issues and demands (rather than being guided by some overarching philosophy) and, almost inevitably, this means that the manner in which the elements of quantitative and qualitative methodologies get combined is liable to be fragmented and inconsistent”

(Denscombe, 2008: 280).

Michel Foucault1926-1984

Post-Modernism

Dominated by the influence of Michel Foucault; what is seen is actually the result of a set of power struggles

Dyson & Brown (2006) highlight his main topics:

Truth

Knowledge

Ethics

Philosophies / Paradigms / DilemmasPositivism / Subjectivism / Critical realism / Postmodernism

Methodological CriteriaInternal validity / External validity / Reliability

Research Strategies / Designs Experiments / Surveys / Action research / case studies / ethnography / grounded

theory / phenomenology / mixed methods

Research MethodsQuestionnaires / interviews / observations / documents

Data AnalysisInferential statistics / descriptive statistics / content analysis / thematic analysis /

discourse analysis / interpretive phenomenological analysis

(Dyson & Brown, 2006:3)

Measuring Brains

Methodological Criteria

Validity: are we measuring what we claim to measure; describing what we claim to describe?

Reliability: to what extent can the research be repeated (by another researcher and/or at a different time) and the same result found?

Generalizability (External validity): to what extend can we generalize our findings beyond the immediate sample?

Invalid

Unreliable

Internal validity and reliability

Stopped Valid (twice a day) but not reliable

Internal validity and reliability

Ten minutes slow Reliable (but reliably wrong, never valid)

Positivism Subjectivism

Internal validity*

Phrase concepts in terms that can be observed and measured in order to hypothesize about causal relationships between variables

Descriptions of how social life is “achieved”Understand meanings of social world

Reliability* Imposing structure / control on social world – reduce variability

Understand that the production of knowledge, and the effects of the researcher

Generalizability*

Sampling population to make statistical claims

Small world research.Generating, rather than testing theory

From Dyson & Brown, 2006: 30

*denotes items usually part of the Positivist paradigm

Lincoln, Y.S. and Guba, E.G. (1985)

Internal validity, which Lincoln and Guba translate as truth value, is replaced by the concept of credibility – whether or not the participants studied find the account true.

External validity, or the extent to which findings are more generally applicable, is replaced by fittingness or transferability, which is based on the idea that accounts may be transferable to other specified settings through the provision of thick description about both the sending and the receiving contexts.

Reliability, or the consistency of findings, is replaced by the notion of dependability, which is achieved through an auditing process called an ‘audit trail’, in which the researcher documents methods and decisions, and assesses the effects of research strategies, rather than being concerned about replication.

Objectivity, or a concern with neutrality, is replaced by confirmability –the extent to which findings are qualitatively confirmable through the analysis being grounded in the data and through examination of the ‘audit trail’.

Types of Validity 1 Conclusion validity The degree to which the

conclusions reached by researchers about the relationships between variables within their data are reasonable.

Consequential validity The degree to which the results are commensurate with the purposes to which the results were supposed to be applied.

Construct validity The degree to which conclusions can legitimately be made from the indicators of the study to the theoretical concepts that those indicators are held to represent.

Types of Validity 2

Content validity The extent to which the research concepts or measures incorporate all aspects that should be included and none that should not be included.

Convergent validity The degree to which concepts that should in theory be closely associated are actually observed to be closely associated.

Criterion validity The degree to which a research concept accurately reflects relevant criteria external to the original context of the research.

Types of Validity 3 Ecological validity

The extent to which results can be applied back to a context where the research has been disembedded from the context.

Types of Validity 4

External validity The extent to which the results of the study can be statistically generalized beyond the context of the original study.

Face validity The extent to which the measurement indicator ‘looks right’ or is intuitively appealing to the user or research participant. This kind of validity is treated sceptically by many researchers.

Types of Validity 5 Sample orientated validity Concerned with types

of validity that relate to whether ideas are appropriately grouped together

Sign orientated validity Concerned with types of validity that relate to whether proxy indicators are a good representation of a theoretical construct

Statistical conclusion validity The degree to which conclusions, based on your statistical choices of sample size and significance level set, are correct.

Types of Validity 6

Catalytic validity 'Catalytic validity points to the degree to

which research moves those it studies to understand the world and the way it is shaped in order for them to transform it ............Research that possesses catalytic validity will not only display the reality-altering impact of the inquiry process, it will direct this impact so that those under study will gain self-understanding and self-direction.'

(Kincheloe & McLaren, 2000: 297)

Reliability Temporal stability Under this criterion, a test is reliable if the

same form of a test on given on two or more separate occasions to the same group of participants yields the same result. Repeated measurements may require costly and time consuming visits to field settings, where it may not be easy to locate the original respondents anyway. Repeated testing is also likely to change the participants.

Form equivalence This is based on the idea that two or more different forms of test, based on the same content can be used.

Internal consistency This relates to tests, measures and questionnaires with large number of items. We can correlate the items together in various ways so as to see whether they are related. This principle underlies measures of internal reliability such as Cronbach’s Alpha, or Spilt-half.