EDPSY 500

Remember on 9/11 we will meet in Zimmerman Library

(2nd floor room 254).

CP194

???

According to Stanovich, what Makes ResearchScientific?

How does the No Child Left Behind Act of 2001 (NCLB)

impact educational research?

Who Cares? You should.

• The No Child Left Behind Act of 2001 uses the phrase “scientifically-based research” (SBR) 111 times.

• This has spawned an industry of consultants.

• It has created a very volatile atmosphere.

What Is ScientificScientific Research?(According to NCLB)

• The application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge.

• Systematic, empirical methods that draw on observation or experiment.

• Involves rigorous data analyses that are adequate to test the hypotheses.

• Is evaluated using experimental or quasi-experimental designs.

• Is reported in sufficient detail to allow replication.• Has been accepted by a peer-reviewed journal or

approved by an independent panel of experts through rigorous, objective, and scientific review.

What Paradigm Appears to Be Influencing NCLB?

Postivism?

Post-positivism?

Critical Theory?

Constructivism?

Between a Rock and a Hard Place

The rock.The rock.• Calling for scientifically based

research is good and needed.• “The recent enactment of no child

left behind, and its central principle that federal funds should support educational activities backed by “scientifically-based research,” offers an opportunity to bring rapid, evidence-driven progress – for the first time – to U.S. Elementary and secondary education.” – Coalition for evidence-based policy.

The hard place.The hard place.• Defining SBR as randomized

experimental designs is over-restrictive.

• “The requirement that research methods be restricted to group design with a preference for randomized clinical trials will significantly inhibit the development and validation of new scientific knowledge in education.” – American association on mental retardation (AAMR) board of directors.

“Council recognizes randomized trials among the sound methodologies to be used in the conduct of educational research and commends increased attention to their use as is particularly appropriate to intervention and evaluation studies. However, the council of the association expresses dismay that the department of education through its public statements and programs of funding is devoting singular attention to this one tool of science, jeopardizing a broader range of problems best addressed through other scientific methods. The council urges the department of education to expand its current conception of scientifically-based research.” – AERA council

What Is ScientificScientific Research?(According to the NRC)

• Science poses significant questions that can be investigated empirically.

• Science links research to relevant theory.

• Science uses methods that permit direct investigation of the question.

• Science provides a coherent and explicit chain of reasoning.

What Is ScientificScientific Research?(According to the NRC)

• Scientific findings replicate and generalize across studies.

• Scientists disclose research and encourage professional scrutiny and critique.

Mayer (2000)

• Let’s take a few minutes and read Mayer.

• What makes research “scientific”?

• How important is it that educational research be respected in “academia and in society in general”?

• Should “science” and “research” mean the same things in different disciplines?

• What questions/ comments do you have?

Mayer (2000)

The Big Picture

• There are many different research processes

• Each has its own:– Philosophy of inquiry– Methods of inquiry– Purposes for doing research– Processes and “rules”

• Here is one process:

A Scientific Process

Research Question

Defining the Problem

ReviewLiterature

ArticulateTheory

DefineHypothesis

Testing the Hypothesis

Subject sampling Instrumentation Research design Piloting

Collecting Data Choosing analyses Conducting analyses

Conclusions

Results or Findings

Scientific Thinking Vs. Everyday Thinking

• Everyday thinking– Biased questions

• Do you really support the war?

– Limited sampling• Your friends and family are different from my

friends and family

– Selective attention• Confirmation bias

– Inaccurate generalization• Stereotypes

Scientific Thinking Vs. Everyday Thinking (Cont.)

• Scientific thinking.– Empirical observations.

• Empirical: capable of being confirmed, verified, or disproved by observation or experiment.

– Systematic.– Objective.

• Less dependent on emotion or personal prejudices.

– Replicable.

In groups

• One study indicates that television viewing may have a negative effect on academic achievement while another indicates no relationship.

Speculate on why researchers obtain differing results when studying the same problem.

Purposes of Scientific Research

• Exploratory– What is out there?

• Descriptive– What does this group look like?

• Explanatory– Why and how are these constructs related?

• Evaluation– Does this program work?

• Prediction– Who will become depressed?

Experimental vs. Nonexperimental

• Experiments– Treatment is administered

• Students divided into two groups; one group receives new reading materials the other uses the traditional materials.

• Cancer patients are divided into two groups; one group receives a new drug the other receives the current treatment of choice.

• Nonexperiments – No manipulation of treatment present

• Individuals given a survey asking about television viewing habits.

• Classrooms observed to see if teacher’s expectations affect student performance.

Causal-Comparative Studies.

• Causal-comparative (a.k.a. ex post facto)– Two characteristics

• Observe and describe a current condition• Look to the past, or demographic characteristics,

to identify a cause.

Survey Research

• Used to describe – Attitudes– Opinions– Beliefs– Behaviors

• Usually include large numbers

Longitudinal

• Repeated measures over a period of time– High School and Beyond– 1988 NELS Cohort

Correlational

• Many of the aforementioned designs are correlational– Study Relationships

• SAT and Achievement• Income and achievement• Age and Cholesterol level

Case Studies

• Involve one unit– Individuals– Classes – Schools– Districts

Content Analysis

• Technique that enables researchers to study human behavior in an indirect way, through an analysis of our communications.

• Types of communications are:• Textbooks• Essays• Pictures• Songs

• A person or group’s conscious and unconscious beliefs, attitudes, or values are often revealed in their communication.

Ethnographic Research

• A variety of approaches is used in an attempt to obtain as holistic a picture as possible of a particular society, group, setting, etc.

• The emphasis is on documenting the everyday experiences of individuals by observing and interviewing them and relevant others.

• The key tools are in-depth interviewing and continual, ongoing participant observation of a situation.

Historical Research

• The systematic collection and evaluation of data to describe, explain, and understand actions or events that occurred sometime in the past..

• An attempt is made to reconstruct what happened during a certain period of time as completely and accurately as possible.

Action Research

• Action Research is conducted by one or more individuals or groups for the purpose of solving a problem or obtaining information in order to inform local practice.

Research Hypotheses and Questions

• Research hypotheses

• Directional– Is a prediction of a study outcome.

• First grade girls will perform better on a reading comprehension test than first grade boys.

• Children shown an adult interacting aggressively towards a doll will engage in more violent acts than children who observe an adult interact non-aggressively with the same doll.

Research Hypotheses and Questions (cont.)

• Non-directional– Girls will score differently than boys on a

measure of self esteem.– The reading achievement of students

exposed to phonics instruction will differ from students exposed to whole language instruction.

Research Hypotheses and Questions

• Research Question– Differ from hypotheses by the generality of the

question.• How do students perceive the new curriculum?• How do students of minority groups interpret that

way they are represented in the media?

Variables

Measurement

• Is the assignment of numerals to objects.– Nominal

• Examples: Gender, party affiliation, and place of birth

• Ordinal– Examples: SES, Student rank, and Place in race

• Interval– Examples: Test scores, personality and attitude scales.

• Ratio– Examples: Weight, length, reaction time, and number of

responses

Understanding Variables and Hypotheses

• Objects– Things that one does research on.

• People, districts, nations, etc.

• Properties of objects– Give us a way to talk about how objects are

alike and how they differ.

• Scores– Values on the property of interest

• Must be at least two.

Values

• Exhaustive– Must be able to assign a value to all objects.

• Mutually Exclusive– Each object can only be assigned one of a set

of values.

• A variable with only one value is not a variable.– It is a constant.

How variables are used

• Two major piles– Descriptive and causal

• Descriptive– Describes a population in relation to one or more

variables.• Sex bias in textbooks• Trends in dropout rates

• Causal– Does A cause B– Associations between A and B

• Is the observed relationship greater than would be expected by chance?

Hypothesis

• A proposed explanation for a phenomenon.– Two types

• Casual order - 'A causes B' • Empirical generalizations – ‘A is related to B’

Sorting Out Variables in a Study

• Purpose of most empirical studies in behavioral research is to identify causal relationships.– Independent variables (IV)

• Causes, determinants, predictors, factors.

– Dependent variables (DV)• Consequences, outcomes, effects

– Moderator variables• Variables that change the relationship between the IV and

DV.• Aptitude by treatment interactions

Dissecting Hypotheses

1. Identify the two variables and sort them into IV and DV.

2. Describe each variable.• Object, property, mode of variation,

elaborate on mode of variation.

3. Specify the relationship expected between the two variables.

4. Note the unit of analysis implied or actually used.

Additional comments

• Simple Hypotheses have only two variables--bivariate relations.

– H.1: Authoritarian principals are more effective than non-authoritarian principals• What are the names of the two variables?

• How do they vary?

• Complex Hypotheses have more than two variables and sometimes contain a moderating variable.

– H.1: Authoritarian principals are more effective than non-authoritarian principals when goals are clear, but non-authoritarian principals are more effective when goals are ambiguous.

• What are the names of the three variables?

• How do they vary?

Practice

• H.1: The greater the weight of a five-year old, the taller the child.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

Practice

• H.2: Authoritarian principals command more loyalty than non-authoritarian ones when they have influence, but non-authoritarian principals command greater loyalty when principals lack influence.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

• H.3. Secondary teachers are more custodial in pupil

control ideology than elementary teachers.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

H. 4. Academic achievement will be greater among students taught by autocratic teachers than those taught by permissive teachers.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

• H. 5. The larger the size of a community college’s instructional faculty the greater the degree of administrative centralization.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

• H. 6. Democratic supervisors have teachers who demonstrate more creativity in teaching methods than autocratic supervisors.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

• H. 7. When administrators have influence with their superiors, authoritarian administrators command more loyalty from subordinates than non-authoritarian ones, but when administrators have little influence, then non-authoritarian administrators command more loyalty than authoritarian ones .

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?

• H. 8. The stronger the collective efficacy of a school, the higher the level of level of student achievement.

– What is the object?– What are the variables?– What are the names of the variables?– How do they vary-categorical or continuous?– What is the moderating variable?– What is the independent variable?– What is the dependent variable?