Assessing Science Assessing Science Learning in Learning in

3 Part Harmony3 Part Harmony

Richard DuschlRichard Duschl

GSE-Rutgers UniversityGSE-Rutgers University

rduschl@rci.rutgers.edurduschl@rci.rutgers.edu

Performances - PracticesPerformances - Practices

PianoPiano Finger/hand strength Finger/hand strength

and flexibilityand flexibility Read muscial notationRead muscial notation Musical phrasing, Musical phrasing,

playing with feelingplaying with feeling Creative musicalityCreative musicality

ScienceScience Building conceptual Building conceptual

claims, meaningsclaims, meanings Evaluating conceptual Evaluating conceptual

claims, meaningclaims, meaning Seeking evidenceSeeking evidence Seeking explanationsSeeking explanations CommunicatingCommunicating

3 Ps

Psychology - LearningCognitive Science, Information-processing, Social psychology, Activity theory

Philosophy - KnowledgeEpistemology; Science Studies; Models, Argumentation; (ETHICS)

Pedagogy - Teaching Inquiry Learning; Problem-based Learning; Community of Learners; Model-based Learning; Design Principles, Preparation for Future Learning

Nature of ScienceNature of Science

Science is about testing hypotheses and Science is about testing hypotheses and reasoning deductively from experimentsreasoning deductively from experiments Hypothetico/Deductive ScienceHypothetico/Deductive Science

Science is Theory building and revisionScience is Theory building and revision Contexts of Generation and JustificationContexts of Generation and Justification

Science is Model building and revisionScience is Model building and revision Models stand between Experiment and Models stand between Experiment and

TheoryTheory

History of Thinking about History of Thinking about Human MindHuman Mind

Differential PerspectiveDifferential Perspective Individual, Mental Tests separate from academic Individual, Mental Tests separate from academic

learning - selecting and sorting learning - selecting and sorting Behavioral PerspectiveBehavioral Perspective

Stimulus/Response Associations - rewarding and Stimulus/Response Associations - rewarding and punishing punishing

Cognitive PerspectiveCognitive Perspective Prior Knowledge, expert/novice, metacognition Prior Knowledge, expert/novice, metacognition

(thinking about thinking and knowning)(thinking about thinking and knowning) Situative PerspectiveSituative Perspective

Sociocultural, language, tools, discourse Sociocultural, language, tools, discourse

Psychology & Education

Structured Knowledge Prior Knowledge Metacognition Procedural Knowledge in Meaningful ContextsSocial participation and cognition Holistic Situation for Learning: Make Thinking Overt

(Glaser, 1994)

National Science Education National Science Education Standards Content DomainsStandards Content Domains

Big CsBig Cs Life ScienceLife Science Physical SciencePhysical Science Earth/Space ScienceEarth/Space Science Inquiry Inquiry

Little CsLittle Cs Unifying Principles & Unifying Principles &

ThemesThemes Science & Science &

TechnologyTechnology Science in Personal & Science in Personal &

Social ContextsSocial Contexts Nature of ScienceNature of Science

Standards & BenchmarksStandards & Benchmarks

Too Much StuffToo Much Stuff

3 Part Harmony3 Part Harmony

ConceptualConceptual “what we need to know” “what we need to know” EpistemicEpistemic “rules for deciding what counts” “rules for deciding what counts” SocialSocial “communicating & representing “communicating & representing

ideas, evidence and explanationsideas, evidence and explanations

Goals/Units VesselsDuschl & Gitomer,

1997

Acids &Bases

Erduran, 2001

Earthquakes& Volcanoes

Smith, 1996

ConceptualFlotation,Buoyancy

Neutralization,Acid/BaseChem.

IgneousRocks, PlateTectonics

EpistemicCausalExplanation

Models,Modeling

ScientificArgument

Social

Report toCity Planner– VesselDesign,Transport

Report toHazmet – Safedisposal ofA&B inSchools

Report toCity Council– LikelihoodE&V, Emerg.Med. Plan

Learning ProgressionsLearning Progressions&&

Learning Learning PerformancesPerformances

NAEP 2009 Science NAEP 2009 Science FrameworkFramework

http://www.nagb.org/http://www.nagb.org/ A learning progression is a sequence of

successively more complex ways of reasoning about a set of ideas.

Table 14. Examples of Performance Expectations for States of Matter

Grade 4(See content statement P4.3.)

Grade 8(See content statement P8.1.)

Grade 12(See content statement P12.1.)

Identifying Science Principles

Classify samples of material assolid, liquid or gas.

Identifying Science Principles

Given an animation of moleculesin motion, identify the substancethat is being illustrated as a solid,liquid, or gas.

Identifying Science Principles

Explain why ice is harder thanliquid water in terms of thestrength of the force between themolecules.

Using Science Principles

Infer that a change of state (e.g.freezing or melting) affects theidentity of an object, but not theidentity of the material of whichit is made.

Using Science Principles

Predict how the mass of a sampleof iodine will change aftersublimation. Justify theprediction based on what occursduring sublimation at a molecularlevel.

Using Science Principles

Use the concept of moleculararrangements and bonds toexplain why graphite is very softand diamond is very hard, eventhough they are all made of purecarbon.

Using Scientific Inquiry

Collect, display, and interpretdata showing how thetemperature of a substancechanges over time as it cools andbecomes a solid.

Using Scientific Inquiry

Plan and conduct an investigationto determine the melting pointand boiling point of an unknownsubstance.

Using Scientific Inquiry

Explain the results of experimentsshowing how the volume of threedifferent liquids changes whenthey are heated by usingmolecular theory.

Using Technological Design

Propose a method for determiningfor certain if holiday chocolatesthat have been shaped bydifferent processes (melting,freezing, reshaping, or breakinginto pieces) have the sameamount of chocolate in them.

Using Technological Design

Choose the best solution forincreasing the altitude of a hot airballoon, based on anunderstanding of the macroscopicand microscopic changes thatoccur when the gas inside theballoon is heated.

Using Technological Design

Design an instrument to measuretemperature as accurately aspossible, taking into account boththe thermal properties of liquidsand solids to be used in thedevice, and structural shape anddimensions of the device.

Why Things Sink & FloatWhy Things Sink & Float

Density LP - Density LP - Floating StrawsFloating Straws

Relative DensityRelative Density DensityDensity MassMass VolumeVolume

Forces LP - Forces LP - Floating VesselsFloating Vessels

FlotationFlotation BuoyancyBuoyancy PressurePressure MassMass Surface AreaSurface Area VolumeVolume DisplacementDisplacement

Conceptual vs. Epistemic Conceptual vs. Epistemic GoalsGoals

MisconceptionMisconception Structured ProblemStructured Problem Control of VariablesControl of Variables Productive Productive

MisconceptionsMisconceptions Unconventional Unconventional

FeatureFeature Off TargetOff Target

Causal ExplanationCausal Explanation Ill structured problemIll structured problem Design ApplicationDesign Application Modeling Modeling Forecast Items Forecast Items

(Pivotal Cases, Linn)(Pivotal Cases, Linn)

Learning GoalsLearning Goals

What we knowWhat we know How we have come to know itHow we have come to know it Why we believe it over alternativesWhy we believe it over alternatives

Figure 1. Drawings of the student work that was shown during the whole class

discussion. The upper panel was the topic of the first assessment conversation and the

lower of the second

Affordances for Future Affordances for Future Learning Learning

Knowledge in UseKnowledge in Use Density - continental drift, ocean currentsDensity - continental drift, ocean currents Forces - water pressure and neutral buoyancyForces - water pressure and neutral buoyancy

Using Scientific InquiryUsing Scientific Inquiry Density - separation of liquidsDensity - separation of liquids Forces - carrying capacity/displacementForces - carrying capacity/displacement

Using Technological Design Using Technological Design Density - test of “Crown Jewels” - Eureka! Density - test of “Crown Jewels” - Eureka! Forces - retrieval of sunken shipsForces - retrieval of sunken ships

Nature of ExplanationsNature of ExplanationsLanguage of ScienceLanguage of Science

PrincipledPrincipled RelationalRelational Unclear RelationalUnclear Relational Experiential Experiential Inadequate Inadequate

ExplanationExplanation Off TargetOff Target

Evidence-ExplanationEvidence-Explanation Patterns in EvidencePatterns in Evidence Explanatory TheoryExplanatory Theory Balance of ForcesBalance of Forces

Stronger HandsStronger Hands More HandsMore Hands

AffordancesAffordances

Making Thinking visibleMaking Thinking visible Teacher Assessments of Conceptual, Epistemic, Teacher Assessments of Conceptual, Epistemic,

Social GoalsSocial Goals Identification of Productive MisconceptionsIdentification of Productive Misconceptions

Dialogic DiscourseDialogic Discourse Measures/Observations-Data-Evidence-Models-Measures/Observations-Data-Evidence-Models-

TheoryTheory Data-Warrant-Backing-Rebuttal-Qualifier-ConclusionData-Warrant-Backing-Rebuttal-Qualifier-Conclusion

Images for Nature of ScienceImages for Nature of Science Science as Experiments; as Theory-building; as Science as Experiments; as Theory-building; as

Model-buildingModel-building Preparation for Future LearningPreparation for Future Learning

Scaffolding and Assessing Argumentation Processes in Science

King’s College London/American School in LondonCollaborator Kirsten EllenbogenNSF via a seed grant from CILT (Center for Innovations in Learning Technology).

EHH Activity SequenceIntro Unit and Lab 1

Conduct prelab including demonstration of STEP test and taking a pulse. Students collect data Lab 1

2. Data Collection for Labs 2 and 3Lab 2 - Activity Level and Heart RateLab 3 - Weight and Heart Rate

3. Data Analysis for Labs 2 and 3Knowledge Forum Activity “What Matters in Getting Good

Data”Determining Trends and Patterns of DataDeveloping and Evaluating Explanations for the Patterns of

Data4. Evaluating Exercise Programs

Heartrate/min 60 sec

36495051

565759596060606062646466666767687070727375757579808181

8586

92

0 20 40 60 80 100

1

3

5

7

9

11

13

15

17

19

21

23

25

27

29

31

33

35

stu

den

t

heartrate

Group Salient Characteristics of Lab Group Reasoning1 Group 1 uses a ŌfrequencyÕ decision rule to arrive at range of 60-80. That is,

any heart beat with < 3 data entries was eliminated from calculations todetermine the upper boundry for each graph; e,g, 90 for 6 secs., 104 for 15secs., 72 for 10 secs., 75 for 60 secs. These 4 averages were then averagedto get 80 as the upper limit. All charts should be since to get an average alldata should be used. Lab book shows that the decision rule changed withthe consideration of each of the heart rate graphs.

2 Group 2 uses a ŌmajorityÕ decision rule to arrive at a range of 60-80. That is,most of the data fell between 60 and 80. The 6 sec. chart should be used.Lab books shows that the decision rules remains the same for each of theheart rate graphs.

3 Group 3 used an ŌaverageÕ decision rule based on how the data from the 4members of their lab group, and not the class data, could be used to get theaverage and then establish the range. This strategy resulted in 4 separateranges being reported for each of the 4 heart rate graphs. The selected rangewas 60-75. Final decision was based on 10 and 60 second graphs. Labbooks show that the decision strategy is common across heart rate graphsbut that the range results are different.

4 Group 4 used an Ōend pointsÕ decision rule based on the end points of thenormal range determined for each graph. Thus, a range of 70-80 for 6 sec.,66-72 for 10 sec., 60-80 for 15 sec., and 60-75 for 60 sec. 60 appears twiceas the lower boundary and 80 appears twice as the upper boundary, hencethe normal range is 60-80. All graphs used.

5 Group 5 used a ŌcalculationÕ decision rule to arrive at a range of 60-80.That is, each graph was analyzed to find out where 2/3 of the studentsÕ heartrates fell on the graph. The 60 second graph was selected as the mostaccurate.

Group Decision Rules

1 - Frequency2 - Majority3 - Average4 - Endpoints5 - Calculation

Pathways - Historical StepsPathways - Historical Steps

Rochel Gelman & Kim Rochel Gelman & Kim Brennenman - Brennenman - Pathsways for Pathsways for Learning -PreKLearning -PreK ObserveObserve MeasureMeasure Write Write

Lehrer & Schauble Lehrer & Schauble 5th-8th grades5th-8th grades VariationVariation DistributionDistribution Growth MechanismsGrowth Mechanisms Adaptive SelectionAdaptive Selection Evolution Evolution

Observation-EvidenceObservation-Evidence

There exists a continuum of what counts as scientific data, and subsequently what counts as scientific evidence. From initial sense-based descriptive observations, to tool assisted measurement observations, and to theory-driven instrument based observations. The latter most sophisticated level underscores the revision-based and theory-laden nature of science.

Evidence-based Evidence-based ArgumentationArgumentation

There exists a continuum regarding the use of evidence to support and refute scientific claims, and the structure and practice of argumentation (language of argumentation and role of consensus). Initial arguments feature a simple single claim-evidence structure, with learning arguments develop to include counter claims and counterevidence with attention to resolving alternative explanation and informing theory.

Theory-buildingTheory-building There exists a continuum of sophistication

regarding the use of evidence and explanations to develop, refine and modify scientific theories. Initially students may not discriminate between evidence and theory. With engagement and learning opportunities students can refine and deepen their understanding and practices of the relationships between evidence and explanations. Sophisticated images of the nature of science conceptualize theories as robust explanatory schemes comprised of multiple models, models that stand between evidence and explanation.

Inquiry Based LearningInquiry Based Learning

Deciding the ContentDeciding the Content Aims & GoalsAims & Goals

ConceptualConceptual Facts, Principles, Facts, Principles,

Laws & TheoriesLaws & Theories EpistemicEpistemic

Explanations, Explanations, Models, Models, ArgumentsArguments

SocialSocial Representations, Representations,

Communications Communications

Deciding the ContextDeciding the Context School ScienceSchool Science ““Real World” ScienceReal World” Science

EnvironmentEnvironment Social IssuesSocial Issues

Museum/Science Centre Museum/Science Centre ScienceScience

3 Part Harmony3 Part Harmony

Conceptual GoalsConceptual Goals

Epistemic GoalsEpistemic Goals

Social Goals Social Goals

Thank YouThank You