developing and assessing scientific reasoning in introductory
TRANSCRIPT
Krista E. Wood
Department of Math, Physics
& Computer Science
May 16, 2015
Kathy Koenig
Department of Physics &
Department of Science Education
Ohio-Project Kaleidoscope (OH-PKAL)
Scientific Reasoning
Scientific Reasoning (SR) is a set of skills
necessary in carrying out scientific practices
related to collection and analysis of evidence, and
generation of evidence-based arguments6
SR skills positively correlate to:
• ability to learn concepts1,2
• develop higher levels of problem solving skills3,4
K.E. Wood & K. Koenig 2 May 16, 2015
How to Develop SR Skills Curriculum should:
• Explicitly target SR skills
• Use a process of inquiry1,5,6
3
Guided Inquiry • Engage & Explore with simulations and physical equipment
• Design experiment to answer research question
• Conduct experiment and analyze data to develop a mathematical model
• Synthesize & Evaluate to develop evidence-based claim.
K.E. Wood & K. Koenig May 16, 2015
How to Assess SR Skills
• Lawson Classroom Test of Scientific
Reasoning7 is commonly used 2,5,6,8
has ceiling effect with college students9
• Inquiry for Scientific Thinking and
Reasoning (iSTAR)10 used for finer
grain analysis of Control of Variables
(COV) sub skill11
K.E. Wood & K. Koenig 4 May 16, 2015
Purpose of Study
•Redesign General Physics II labs to use process of guided inquiry
target specific SR skills
•Evaluate effect on student development
of SR skills.
•Evaluate level of COV sub skills to
further refine lab curriculum.
K.E. Wood & K. Koenig 5 May 16, 2015
Study Context
•Spring 2014 semester at UC Blue Ash, an open
access two-year regional college of UC
•20 students in algebra/trig-based General
Physics II Lab
Lab Curriculum Redesign
–Original labs traditional, prescriptive labs
–Redesigned to be guided, inquiry-based labs
–Focus: Design controlled experiments &
make evidence-based claims
K.E. Wood & K. Koenig 6 May 16, 2015
Objective Tasks Task Goals,
SR Skills Targeted Engagement
&
Exploration
Guided inquiry activities using PhET
simulations and lab equipment.
Explore concepts and
potential factors involved
Design
Experiments
Identify possible factors. Narrow to
Independent Variable (IV), Dependent Variable
(DV), and Control Variables (CVs) involved to
address research question.
Control of Variables
Hypothetical Deductive
Reasoning
Implement
Experiments
Collect and analyze data. Develop conceptual
or mathematical model.
Quantitative Linear
Correlation
Elaborate Analyze “student reasoning” scenario. Reveal common naïve
conceptions.
Extend Apply model to another scenario. Integrated Hypothesis
Explanation Students explain during instructor checkpoints
interspersed throughout lab.
Check for
understanding.
Communicate & explain
results.
Synthesis &
Evaluation
Whole class Whiteboard meetings to present
Claims, Evidence, and Reasoning and
collaboratively evaluate results.
Develop team work and
evaluation skills.
Methods Lab Curriculum Redesign Instructional Framework12
Methods Lab Curriculum Redesign for General Physics II
K.E. Wood & K. Koenig 8 May 16, 2015
Lab Content Focus Simulations Used
Labs 1, 2 – Properties of
Ideal Gas
Explore properties of ideal gas using PhET Gas
Properties. http://phet.colorado.edu/ *
Labs 3, 4, 5 – Behavior of
Light (Snell’s Law, Lens
ray optics, Diffraction)
Explore behavior of light using PhET Bending
Light, Wave Interference, and Optics Simulation.
http://physics.bu.edu/~duffy/java/Opticsa1.html
Labs 6, 7 – Electric Fields
and Electric Potential
Explore electric fields and equipotential surfaces
using PhET Charges and Fields.
Labs 8, 9, 10, 11 – DC
Circuits
Explore using PhET Circuit Construction Kit (DC
only)
Lab 12 – Magnetic Force
on a Wire
No simulation used for this topic.
* All PhET simulations available at http://phet.colorado.edu
Methods Data Collection
iSTAR administered pre-test and post-test
• 29-question multiple choice test
• 5 paired questions. Students chose a statement to
explain their reasoning for a previous question
• Assessed 9 SR sub skills:
quantitative linear basic probability
COV statistical probabilistic
integrated hypothesis causation correlation
hypothetical deductive conditional logic reasoning
correlation
K.E. Wood & K. Koenig 9 May 16, 2015
iSTAR Sample Questions – Quantitative Linear
10
1. A twelfth grade class has 9 students. The teacher brings in 6 bottles
of water that fully fill all students’ glasses (no water is left). How
many glasses could be filled with 8 bottles of water?
a. 5 ⅓ glasses c. 8 glasses e. 11 glasses g. 14 glasses
b. 6 ¾ glasses d. 9 glasses f. 12 glasses h. 16 glasses
i. none of the above
2. From the previous question, how many bottles of water are needed
to fully fill 6 glasses?
a. 2 bottles c. 4 bottles e. 8 bottles g. 12 bottles
b. 3 bottles d. 7 bottles f. 9 bottles h. 18 bottles
i. none of the above
May 16, 2015 K.E. Wood & K. Koenig
iSTAR Question – Control of Variables (Basic w/o data)
4. A group of students completed a project which involved making soy milk
ice cream. In the process, the soy milk had to be heated to boiling first. Most of
the students cooled their hot soy milk to room temperature before placing it in
Type of Soy
Milk
Amount of Soy
Milk
Temperature of Soy
Milk before Placing in
Freezer
Experiment 1 sugar-free soy
milk (2) 300C
Experiment 2 (1) half cup of soy milk 700C
a. sugar-free soy milk; (2) half cup of soy milk
b. soy milk with sugar in it; (2) half cup of soy milk
c. soy milk with sugar in it; (2) one cup of soy milk
d. sugar-free soy milk; (2) one cup of soy milk
11
the freezer. However, Jessy placed her hot soy milk directly in the freezer
before it was cooled. Later it was observed that Jessy’s soy milk took less time
to completely freeze compared to the others. The students are puzzled and
wonder “do hot liquids freeze faster than cold liquids?” The table below
provides the conditions for various experiments that enable this question to be
answered. However, there are two items missing from the table (labeled (1) and
(2)). Determine what these items (1) and (2) need to be in order to address the
students’ question.
iSTAR Question – Control of Variables (Basic w/ data) 5. As shown below, a string hangs from a bar and has a small ball attached to its end.
The string (and the attached ball) can be made to swing back and forth, and the number
of complete swings during a certain time interval can be counted. A student wants to
know whether or not the number of swings in 10 seconds is affected by the length
of the string, the mass of the ball, and/or the angle the string is pulled away from the
vertical at the time of release.
The student carried out several experiments to investigate what factors affected the
number of swings in 10 seconds. The conditions and results are shown in the table below.
Trial 1 Trial 2 Trial 3
Variables
length of string 10cm 10cm 40cm
mass of ball 20g 30g 30g
angle at release 15° 30° 15°
Number of swings in 10 seconds 16 16 8
Ignoring all other variables, which variable or variables do you think
can be tested using the information shown in the table above?
a. the length of the string
b. the mass of the ball
c. the angle at release
d. “a” and “b”
e. “b” and “c”
f. “a” and “c”
g. “a”, “b”, and “c”
h. No variable can be tested using the
information provided in the table.
Methods Data Analysis
Analyzed iSTAR class averages for pre-
test, post-test, and change in SR skills for:
• Overall SR
• SR sub skills
• COV sub skills11
K.E. Wood & K. Koenig 13 May 16, 2015
Results and Discussion Overall Scientific Reasoning Skills
Shift in overall
iSTAR scores
from
48.6% to 55.0%
for a change of
6.4%
K.E. Wood & K. Koenig 14 May 16, 2015
Overall
Avg Pre 48.6%
Avg Post 55.0%
Change 6.4%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
% C
orr
ec
t
SS14 iSTAR Overall SR Skills (n=20)
Results and Discussion Scientific Reasoning Sub Skills
15
QuantitativeLinear
Control ofVariables
BasicProbability
IntegratedHypothesis
HypotheticalDeductive
CorrelationStatistical
ProbabilisticCausationCorrelation
ConditionalLogic
Pre SS14 63.5% 38.6% 71.4% 61.0% 85.0% 76.2% 28.6% 31.0% 19.0%
Post SS14 72.7% 48.6% 75.0% 67.3% 86.4% 81.8% 29.5% 34.1% 18.2%
Change 9.2% 10.1% 3.6% 6.3% 1.4% 5.6% 1.0% 3.1% -0.9%
-10.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
SS14 iSTAR SR Sub Skill Class Means (n=20)
• 5-10% increase in most targeted SR sub skills: quantitative linear, COV,
integrated hypothesis, and correlation, but not in HD reasoning
• Below 5% for SR sub skills not targeted: basic probability , statistical
probabilistic, causation correlation and conditional logic.
Results and Discussion Control of Variables Sub Skills
K.E. Wood & K. Koenig 16 May 16, 2015
COV(Basic,withoutdata)
COV(Basic, with
data)
COV(Causation)
COV(HiddenRelation)
Pre SS14 58.7% 42.9% 16.7% 23.8%
Post SS14 68.2% 59.1% 18.2% 34.1%
Change 9.5% 16.2% 1.5% 10.3%
0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%
SS14 iSTAR COV Class Means (n=20)
• Student
post-test was
highest on
COV, basic
without data11
• Increase was
greatest for
COV, basic
with data.
Conclusion
• Increase of 5-10% considered reasonable for a
15-week course13 that explicitly targets scientific
reasoning.
•Little change occurred in SR sub skills not
targeted.
Next Steps Make improvements based on results.
Increase sample to all UCBA lab students SS15.
K.E. Wood & K. Koenig 17 May 16, 2015
1. H. She and Y. Liao, J. Res. Sci. Teach. 47(1), 91-119 (2010).
2. V.P. Coletta and J. A. Phillips, Am. J. Phys. 73(12), 1172-1182 (2005).
3. C. Fabby and K. Koenig, Physics Education Research Conference Proceedings, Portland, OR (2013).
4. S. Ates and E. Cataloglu, European J. Phys. 28(6), 1161-1171 (2007).
5. K. Koenig, M. Schen, M. Edwards, and L. Bao, J. College Sci. Teach. 41(4), 23-29 (2012).
6. K. Koenig, M. Schen, and L. Bao, Sci. Educator. 21(2), 1-9 (2012).
7. A.E. Lawson, J. Res. Sci. Teach. 15(1), 11-24 (2000).
8. J.C. Moore, and L.J. Rubbo, Phys. Rev. ST – Phys. Educ. Res. 8, 010106 (2012).
9. Bao, L., et al., Science, 323, 586-587 (2009).
10. L. Bao and K. Koenig, Inquiry into Scientific Thinking and Reasoning (private communication, 2013).
11. S. Zhou, L. Bao, K. Koenig, A. Raplinger, J. Han, Y. Pi, and H. Xiao, Assessment of student reasoning in
control of variables. Am. J. Phys. (under review).
12. L. Bao and K. Koenig, (2012). TI21: A Technology Enhanced Inquiry Framework for Developing and
Assessing 21st Century Skills, iSTARAssessment.org
13. L. Bao (private communication, 2014). 18
References
Thank You!
For more information, feel free to contact
Krista Wood Kathy Koenig [email protected] [email protected]