design-based research
TRANSCRIPT
UneApproche de
RechercheBaséesur la
Conception afin de
ComprendreL'apprentissa
geComplexe
A Design-Based Research Approach to Understanding Complex Learning
Cindy E. Hmelo-Silver
Rutgers University
VueD'Ensemble
Pourquoi DBR?
Caractéristiques
Commencer par l’expertise
Dans le laboratoire
Iterations Multiples de "Systèmes et Cycles"
Complexity of Learning
Environments
The strengths of design studies lie in testing theories in the crucible of practice; in working collegially with practitioners, co-constructing knowledge; in confronting everyday classroom, school, and community problems that influence teaching and learning and adapting instruction to these conditions; in recognizing the limits of theory; and in capturing the specifics of practice and the potential advantages from iteratively adapting and sharpening theory in its context. (Shavelson et al, 2003)
Its not just the technology….
Pedagogy
Curriculum
Participant structures
AND
Technology
Answering the question “under
what circumstances”
“What works” is underpinned by a concern for “how, when,
and why” it works, and by a detailed specification of what,
exactly, “it” is. This intimate relationship between the
development of theory and the improvement of
instructional design for bringing about new forms of
learning is a hallmark of the design experiment
methodology.
(Cobb et al, 2003, p. 13)
Design Experiments (Brown, 1992)
Supporting new forms of learning
Conducted in a limited number of settings
Example of broader class of phenomena
Embodied conjecture (Sandoval, 2004)
DBR Cycle
•Comparison of enactments
•Microcycles within enactments
•Theory
•Embodied conjectures
•Aspects of design
•Documenting context
•Attend to unexpected
•Documenting Learning
• Embodied Conjecture
• “t”theory
What? How?
Why?Revise
?
Mixing Methods
Drowning in data, e.g.,
Video
Artifacts
Pre and post tests,
Qualitative
Ethnography
Interaction Analysis
Grounded theory
Quantitative
Descriptive
Inferential
Statistical modeling
From Expertise to
Instruction in Systems and
Cycles
Theory:
Understanding Complex Systems
Difficult because: Multiple levels of organization that often depend on
local interactions(Wilensky & Resnick, 1999) Invisible, dynamic phenomena pose barriers Conflict with learners’ prior experience(Feltovich et
al., 2001)
Novice Understanding Focus on the perceptually available structures (Hmelo et al,
2000; Wood-Robinson, 1995) Favor simple explanations, central control (Jacobson, 2001) But can conceptual representations provide organizing
frameworks for learning about such systems? Examples: Emergence, Structure-behavior-function
Structure-Behavior-Function
(SBF) theory (Goel et al, 1996)
“What” Structures: elements of a system Fish
Filter
“How” Behaviors: refer to mechanisms Filters remove waste by trapping large particles, absorbing
chemicals, converting ammonia into harmless chemicals
“Why” Functions: why an element exists within a system, role, or output of the system Filter removes byproducts from the aquarium
In the Lab: Representing Expert-Novice
Differences (Hmelo-Silver et al., 2007)
Participants 20 Middle School Students
26 Preservice Teachers
9 Experts (5 hobbyists, 4 biologists)
Coding Interviews coded with SBF coding scheme for
the presence or absence of target concepts
Results:
Aquarium Systems
* *
Qualitative Analyses Expert interviews: Provided more elaborate responses
Demonstrated a more integrated understanding that cut across the SBF levels.
Novice interviews: Mentioned numerous structures
Expert-expert differences Scientists- hierarchical model
Hobbyists- pragmatic models
Proof of Concept:
Experimental Study of Hypermedia
Function
Structure
Comparing Function-centered vs.
Structure-centered hypermedia(Liu & Hmelo-Silver, 2009)
Studies with both undergraduates and seventh-grade students
Visible SBF includes macrolevel phenomena involved
with external respiration
Organ level such as airways, brain, diaphragm, heart, lungs,
muscles, ribs
F-hypermedia = S-hypermedia
Includes microlevel phenomena related to gas
exchange, transport, and internal respiration
e.g. alveoli, capillaries, cellular respiration, red blood cells
F-hypermedia > S-hypermedia
Moving into the Classroom
Function-centered Aquarium Hypermedia
Simulations and Modeling
Allow learners to experience complex systems phenomena
Simulations and models help focus learners on function and behavior
Make invisible phenomena visible and open for inspection
NetLogo as platform for simulation development (Wilensky, 1999) Agent-based modeling tool
See how local interactions contribute to system behavior
NetLogo Fishspawn Model
Nitrification model
In the Classroom
Providing scaffolding and sequencing that help establish “why” and “how” questions
Mix of hands-on activities, hypermedia resources, simulations, class discussions
Scaffolding needs to encourage mapping: Between real world and virtual world Between different levels Considering how models simplify the world
Research Context
Goal to support middle school science instruction in domain of aquarium ecosystem
Units developed with two collaborating teachers
145 middle school students in 2 public schools for about 2 weeks 70 7th grade with Teacher A
75 8th grade with Teacher B
Both classrooms had physical aquaria and 1-2 laptops for each small group
Teaching Contexts
Both teachers experienced, considered experts
Teacher A Used worksheets with open-ended questions
Expected homogeneous progress for whole class
Focus on content
Teacher B Inquiry-oriented norms for classroom
Scaffolded exploration by asking students to observe and explain, open-ended questioning
Learning Outcomes
Enactments
Although both teachers showed significant
gains, IA showed great differences between
classrooms
Three areas
Nature of learning science
Use of language
Interpretation of computer models
Beliefs about Nature of Science
Learning
Teacher A
Science learning means learning content
Judge student learning by how well they completed
teacher-assigned worksheets.
Teacher B
Science as a process of reasoning and understanding
Assessed learning through student discourse and artifacts
Teacher A: Use of Student Language
Concentration on definitions of terms
Questions required one-word response to whole class
Questions aimed at reproducing declarative knowledge
Use of student language to convey behaviors
Results suggest student understanding was scaffolded by connecting to prior knowledge to explain new concepts
Teacher B: Use of Scientific Discourse
Open-ended questions requiring explanations
Promoted argumentation in student discourse
Incorporation of new scientific terminology
Interpretation of Computer Models:Teacher A: Technology for Instruction
NetLogo as a teaching aid Reinforce content knowledge
Concern with student understanding of computer model as end in itself
Homogeneous understanding
Teacher A: Let’s go over the key. Did you figure out what this is?
Class: Yeah.
Teacher A: What is it?
Class: Plants.
Teacher A: Brilliant, that’s a plant, you got that one. [Writes it on board] Did you getthe red dots?
Class: Yeah.
Teacher A: What’s that?
Class: Ammonia.
Teacher A: Very good. OK now I’m going to make it a little harder. White dots?
Class: Nitrite.
Teacher A: Because what appeared first?
Class: Ammonia.
Teacher A: Red dots. And what appeared second?
Class: White dots.
Interpretation of Computer Models:Teacher B: Technology as a Cognitive Tool
Technology as cognitive tool Affords inquiry
Science as a model building activity
Groups notice different aspects of model
Stimulate cognitive engagement
Use of RepTools to foster deep understanding
Promotion of scientific inquiry
Co-construction of knowledge among group members
Teacher B: …how are you going to know whether the blue boxes are snails, bacteria,what’s the other stuff you said, algae, stuff like that?
Courtney: I don’t think it’s bacteria because the red is ammonia and it’s not eating,it’s not getting rid of it.
Teacher B: How do you know that?
Courtney: Because, um well, you can see the ammonia on top of it and it’s not doinganything to it.
Teacher B: Well it’s paused right now.
Courtney: Well also because the ammonia is increasing and while these things areincreasing too it’s not decreasing the amount of ammonia.
Teacher B: It’s not?
Courtney: No, well that’s what I observed. Am I wrong?
Teacher B: No, no.
Ron: Say that again, Courtney…
Courtney: I said, I think that the blue can’t be bacteria because bacteria eats ammoniaand while the blue is increasing the ammonia is still increasing too so if the bluewas bacteria…
Lessons Learned
A tale of two classrooms Different cultures
Different beliefs about learning and inquiry
Appropriation of tools consistent with beliefs
Both teachers Considered expert
Willing to take risks
Despite differences, similar outcomes
Scaffolding SBF
More explicit guidance in SBF thinking
Aquarium Construction Toolkit (ACT)
Vattam, Goel, Rugaber, Hmelo-Silver, Jordan,
Gray, Sinha, 2011)
Aquarium Construction Toolkit (ACT)
Later EnactmentsInstitutionalizing ACT
Preparing for Formal Curriculum development
Students articulate initial ideas about
aquarium with ACT
Build on prior knowledge using hypermedia
Use NetLogo simulations to explore factors
important for maintaining healthy aquarium
Macro
Micro
Create ACT models
Classroom context
54 seventh grade students
Pre and post tests on aquatic ecosystems
Coded for:
Structures, behaviors, and functions and relations among
them
Connections between micro and macro level phenomena
Methods
Results
Learning Scientific Practices (Eberbach& Hmelo-Silver, 2010)
Scientific observation is a complex practice
(Eberbach& Crowley, 2009)
Learning to observe scientifically is rarely
the focus of learning research but should be
(Duschl, 2008)
Computer-supported tools can support
inquiry, but may be insufficient for learning
(Hmelo-Silver, 2006; Tabak, 2004)
Case Study Design
Fish Spawn:
Observing the Macro Level
“What are all those yellow
things?”Shruti: They (fish) get a lot of food. Is the yellow the food?
Ms G: Is what-
Shruti: //yellow the food?
Erica: Is the yellow the eggs?
Ms G: No.
Shruti: Is it the food?
Ms G: No.
Mary: Then what is it? The algae?
…
“Why are the fish dying?”Shruti: The number (of fish) eaten keeps going up!
Erica: Now the total fish is 76.
Shruti: Who’s eating the other fish?
Erica: 92! Oh! We just got more. 112!
Shruti: Why are so many fish getting eaten? It says poor water deaths.
…
Shruti: All the fish are dying from poor water too.
Mary: Poor water? We have good water! We changed it like a million times!
Shruti: No, I made the water quality 100—1,000%
“Why is the water so disgusting?”
The girls make independent observations without
making connections:
Mary: …the yellow in ganging up in places where the fish
aren’t.
Erica: None of the fish are dying from no food
Starve the Fish!
Shruti: I want the fish to die from food
death.
Observing the simulation and data outputs:
Mary: But the yellow goes away when
there’s less food. But she (the teacher)
said it wasn’t the food!”
Eureka!
Ms G: …the yellow patches, I think they are the food. I was thinking they were the waste. But I think they’re the food. But if you think about it, the waste would probably be similar. They don’t show the waste apparently. But it would be a similar amount if they’re eating that food. It’s gonna be converted into waste.
Shruti: I thought it wasn’t waste because when the baby fish went over it, it started disappearing.
Ms G: It did? Ok. So, good.
Observing the Micro Level:
“Why are the fish dying?”
“What are the red dots?”
Shruti: What are the red dots?
Mary: I don’t see any fish. [Pause=4 s] Are the red
dots the ammonia?
Ms G: Well um why did you say that?
Mary: Because the ammonia was going up and there
was more red dots.
Ms G: Did you hear what Mary said? She noticed that
as she saw red dots the ammonia level in the graph
was going up. (points to graph)...
Coordinating Observational
Practice
Shruti: I have no idea—Maybe they’re the bacteria? Maybe
they’re the different types of bacteria?
Ms G: How could you figure that out?
Shruti: I don’t know. (laughs)
Ms G: What if you started again and really looked at when
those patches start to appear, the timing of the
patches, then look at the other data (points to screen).
What’s happening when those chemicals start to build?
How does the appearance of those patches um relate to
the chemicals?
Summary
Computer & social mediation are synergistic
Simple identification questions can filter
complexity & stimulate productive observations of
complex behaviors.
Design should facilitate learning in complex inquiry
learning environments in ways that enable learners
to observe the effects of their actions and to
observe multiple dimensions of complex systems.
Current Work
Analysis of 2010-2011 enactments and implementation
Qualitative analysis of student engagement (Sinha)
(because self-report measures didn’t help)
curriculum revision
Studying relation of engagement to transfer
Analysis of transfer of ecosystem principles (Yu)
Demonstrated significant gains from pre-test to post-test on use ofphotosynthesis, cellular respiration, decompositionin aquatic ecosystem AND rainforest
Microgenetic analysis of drawings (Eberbach, Hmelo-Silver, Jordan)
Trajectories of Change
Challenges in DBR
By their very nature, design studies are complex, multivariate, multilevel, and interventionist, making warrants particularly difficult to establish (Shavelson et al, 2003)
Data overload and responding to emergent questions while trying to stay systematic and focused
Resource challenges
Documenting design decisions and rationales in the heat of the moments
Staying accountable to both theory and practice
Acknowledgements
Ashok Goel
Spencer Rugaber
Swaroop Vattam
David Joyner
National Science Foundation
Institute for Education Sciences
Rebecca Jordan
Lei Liu
Steven Gray
Surabhi Marathe
Suparna Sinha
Catherine Eberbach