skipping the first step: physical process understanding in ...€¦ · see perc poster p1-53 ....
TRANSCRIPT
![Page 1: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/1.jpg)
Skipping the First Step: Physical Process
Understanding in Problem Solving
Katie Ansell
![Page 2: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/2.jpg)
New Conceptual Assessment at Illinois
In previous clinical studies:
Hands-on experiences weaken the attractiveness
of distractor answers in conceptual assessments
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Pe
rce
nt
of
stu
de
nts
Sample Answer Choice Distribution
from Kinematics Study
Control
IOLab
correct distractor other
New assessment for
longer-term hands-on
programs
![Page 3: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/3.jpg)
Conceptual Assessment - Goals
For Introductory Calculus-based Mechanics:
• Cover full scope of course
• Challenging conceptual questions
• No math*
• Strong distractors
Given in Fall 2013 and Spring 2014 as exam review online homework
![Page 4: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/4.jpg)
Conceptual Assessment -
Implementation
• 4 sets of 20 questions chosen based on observed student difficulties
• Given as exam review online homework in Physics 211
• Data collected in Fall 2013 (N = 919) and Spring 2014 (N = 1211)
Data from the Fall 2013 semester will be presented.
![Page 5: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/5.jpg)
Example Question
Question Source: The Ohio State University
![Page 6: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/6.jpg)
Example Question
With what force do you have to pull…
Low Physics High Physics
Low High Low High
Math Score
Question Scores by Math and Physics Ability
For more on math skills analysis: William Evans, session GA04
![Page 7: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/7.jpg)
Example Question
With what force do you have to pull…
Student Answer Choices by Percentile
Correct answer
Distractor 1
Distractor 2
![Page 8: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/8.jpg)
Evaluation of Questions by Distractor
Strength
Strong distractor(s) Weak/no distractor(s) Weak question
![Page 9: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/9.jpg)
Conceptual Questions as diagnosis of
student problem-solving behavior
Assessment questions:
Do students activate the
appropriate physics resources?
Possible expert-like
behavior
Equation
huntingMemorization
These approaches break down for
more difficult problems
no yes
![Page 10: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/10.jpg)
Conceptual Assessment to characterize
student problem-solving behavior
![Page 11: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/11.jpg)
Grouping by Assessment Score
Average = 61 ± 1 % Average = 75 ± 1 %
![Page 12: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/12.jpg)
Relationship to Calculation Problems
Average = 54 ± 1 % Average = 71 ± 1 %
![Page 13: Skipping the First Step: Physical Process Understanding in ...€¦ · see PERC poster P1-53 . Title: Microsoft PowerPoint - AAPT Talk - Assessment.pptx Author: crimmin1 Created Date:](https://reader034.vdocuments.net/reader034/viewer/2022052616/60a174ac585d865ae65d06c1/html5/thumbnails/13.jpg)
Next Steps
• Further examine relationship between student
answer choices and exam performance
• Apply questions as long-term assessment for
Hands-on Prelecture supplement program (in
development)
For more on the pilot program,
see PERC poster P1-53