: Experimental Studies: Experimental Studies

Lineesh.P

Associate professor

AWH special College

Threats to internal validity…Threats to internal validity…

1. history2. maturation3. testing4. instrumentation5. statistical regression6. differential selection of participants7. mortality8. selection-maturation interaction

historyhistory…the occurrence of events that are not

part of the experimental treatment but that occur during the study and affect the dependent variable

maturationmaturation…the physical, intellectual, and

emotional changes that occur naturally in a study’s participants over a period of time

testingtesting…refers to improved scores on a

posttest as a result of having taken a pretest

instrumentationinstrumentation…the unreliability or lack of

consistency in measuring instruments that can result in an invalid assessment of performance

statistical regressionstatistical regression…the tendency of participants who

score highest on a test to score lower on a second, similar test and vice versa

differential selection of participantsdifferential selection of participants…the outcome when already formed

groups are compared raising the possibility that the groups were different before a study even begins

mortalitymortality…the case in which participants drop

out of a study which changes the characteristics of the groups and may significantly affect the study’s results

selection-maturation interactionselection-maturation interaction…if already-formed groups are used in

a study, one group may profit more (or less) from a treatment or have an initial advantage because of maturation, history, or testing factors

Threats to external validity…Threats to external validity…

1. pretest-treatment interaction2. selection-treatment interaction3. multiple treatment interference4. specificity of variables5. treatment diffusion6. experimenter effects7. reactive effects

pretest-treatment interactionpretest-treatment interaction…the situation when participants

respond or react differently to a treatment because they have been pretested

multiple-treatment interferencemultiple-treatment interference…the situation when the same

participants receive more than one treatment in succession

selection-treatment interferenceselection-treatment interference…the situation when participants are

not randomly selected for treatments

specificity of variablesspecificity of variables…the situation when a study is

conducted with (1) a specific kind of participant; (2) is based on a particular operational definition of the independent variable; (3) uses specific dependent variables; (4) transpires at a specific time; and, (5) under a specific set of circumstances

treatment diffusiontreatment diffusion…the situation when different

treatment groups communicate with and learn from each other

experimenter effectsexperimenter effects…the situation when the researchers

present potential threats to the external validity of their own studies

reactive arrangementsreactive arrangements…the situation when a number of

factors associated with the way in which a study is conducted interacts with or shapes the feelings and attitudes of the participants involved

Types of reactive arrangements…Types of reactive arrangements…

…Hawthorne effectHawthorne effect: any situation in which participants’ behavior is affected not by the treatment per se but by their knowledge of participating in a study

…compensatory rivalrycompensatory rivalry: the control group is informed that they will be the control group for a new, experimental study (“John Henry effectJohn Henry effect”)

…placebo effectplacebo effect: the situation in which half of the participants receive no treatment but believe they are

…novelty effectnovelty effect: the situation in which participant interest, motivation, or engagement increases simply because they are doing something different

Controlling for extraneous Controlling for extraneous (confounding) variables…(confounding) variables…

1. randomization2. matching3. comparing homogeneous

groups or subgroups4. using participants as their own

controls5. analysis of covariance (ANCOVA)

randomizationrandomization…the process of selecting and

assigning participants in such a way that all individuals in the defined population have an equal and independent chance of being selected for the sample

matchingmatching…a technique for equating groups on

one or more variables, usually the ones highly related to performance on the dependent variable (e.g., pairwise matching)

comparing homogeneous groups or comparing homogeneous groups or subgroupssubgroups…a technique to control an extraneous

variable by comparing groups that are similar with respect to that variable (e.g., stratified sampling)

using participants as their own controlsusing participants as their own controls…exposing a single group to different

treatments one treatment at a time

analysis of covariance (ANCOVA)analysis of covariance (ANCOVA)…a statistical method for equating

randomly formed groups on one or more variables by adjusting scores on a dependent variable for initial differences on some other variable

Data analysis and interpretation…Data analysis and interpretation…

for single-subject research…a visual inspection and analysis of

graphical presentations of results…focuses upon: adequacy of the

design; an assessment of treatment effectiveness (clinicalclinical vs. statistical statistical significancesignificance)