analysis of human to human tutorial dialogues: insights for teaching analytics
TRANSCRIPT
“Analysis of Human-to-Human Tutorial Dialogues:
Insights for Teaching Analytics”
Irene-Angelica Chounta, Bruce M. McLarenCarnegie Mellon University
Patricia Albacete, Pamela Jordan, Sandra KatzUniversity of Pittsburg
Dialogue as a means for learning
• We aim to develop an adaptive tutorial dialogue system, guided by a student model that will support students in learning physics
Research questions• RQ1: What makes tutorial dialogue successful?– Teachers’ adapt the level of discussion to the
student’s “zone of proximal development” (Vygotsky)
• RQ2: How tutorial dialogues adapt to different student characteristics and prior knowledge?– Level of Control/ Level of Specificity (van de
Pol)– Contingent Tutoring (Pino-Pasternak)– Cognitive complexity (Nystrand, Graesser)
Research questions• RQ1: What makes tutorial dialogue successful?– Teachers’ adapt the level of discussion to the
student’s “zone of proximal development” (Vygotsky)
• RQ2: How tutorial dialogues adapt to different student characteristics and prior knowledge?– Level of Control/ Level of Specificity (van de
Pol)– Contingent Tutoring (Pino-Pasternak)– Cognitive complexity (Nystrand, Graesser)
Research Objective: What makes some tutor’s help generous or stingy, easy or challenging, straightforward or “cognitively complex”? Level of Support
(LOS)
An example would be nice….RQ: What minimum acceleration must the climber have in order for the rope not to break while she is rappelling down the cliff? (You do not have to come up with a numerical answer. Just solve for "a" without any substitution of numbers.)Chip: a = f / mT: what's f ?Chip : f = mgT: just mg ? how many forces act ont he climber ?Chip : mg + TT: is mg down or up ?Chip : down and T is upT: ok so now solve for a again plugging in T and mg
RQ: What minimum acceleration must the climber …….Dale: 500/55 kg=a m/s^2T: I don't agree - that's the acceleration that just the pull from the rope would produce (well once the units are straightened out it would be). Think a little more. What is the general rule for finding acceleration from forces?Dale : F/m=aT: and what is the F there?Dale : tension?T: No.. the F in F=ma is always the net force on the object (or group of objects). The vector sum of all the forces on the object. I prefer to say "Sum of F= ma" because it's easier to get it right. So.. if she is sliding down and the rope is just short of breaking, what is the *net* force on her?
High performer
[How] can we group the features of dialogic discourse to differentiate and operationalize the “levels of support” (LOS)?
• Analyze human-to-human tutorial dialogues• Build a coding scheme to operationalize Level of
Support
The mechanics of tutorial discussions
Method of the study
3 human-to-human dialogues on Physics / 1 per overall learning gains level [low/medium/high]
Level of Control [3-step scale]
Question Categories[18 types]
Level of Specificity[3-step scale]
Contingent Tutoring[binary]
LOSCoding Scheme
Coding scheme - Application
• 4 coders• 3 dialogues [low/medium/high]• 19 tutor turns• Introduction to the coding scheme• Rating handbook & template
Coding scheme - ResultsDimension Fleiss’ Kappa p-value
Level of control 0.404 4.13e-11Question category 0.395 0Level of specificity 0.141 0.0245
Contingency 0.0764 0.415
Lessons learned:
Still unclear how teachers effectively regulate the level of support
Coding scheme: Lessons learned
• Not easy to interpret the goal of the intervention• One intervention, multiple goals• Crucial features: – New content– Feedback Information / Information meant to push student
forward– Degree of detail
Coding scheme Adaptation and Evaluation
Before After
Level of Control Information related to student’s answer (Backward/Forward)Hints Provision
Question Category Question Category
Level of Specificity Feedback on Correctness
Information related to feedback
Contingency Contingency
Application and Evaluation
• 10 human-to-human tutorial dialogues (Physics) – 3 High, 3 Low, 4 Medium
• 2 raters per dialogue• The raters were given a tutorial on the coding
scheme and detailed instructionsDimensions Cohen's kappa
D1. Information Provision (B) 0.871D1. Information Provision (F) 0.843
D2 –Hints Provision 0.843D3-Feedback on correctness 0.826
D4-Information related to feedback 0.764
So you coded it. Now what?
• Provide appropriate, adaptive dialogic support: What is appropriate for whom?– guidelines for feedback provision
• Use dialogue-support mechanisms to inform teachers:– Concepts-coverage and content-contribution– Provide hints vs. provide information– Give away answer vs. set challenge
Chip & Dale: a use case exampleDale
Not so high performer…Loves to ride his bike out in the sun!Not good background knowledge in Physics
Chip
High Performer!Loves to study!Good background knowledge in Physics
Teacher: During the arrow`s flight, how does its horizontal velocity change (increases, decreases, remains the same, etc.)? Remember that you can ignore air resistance."
Student: decreases
Chip & Dale: a use case exampleDale
Not so high performer…Loves to ride his bike out in the sun!Not good background knowledge in Physics
Chip
High Performer!Loves to study!Good background knowledge in Physics
Teacher: During the arrow`s flight, how does its horizontal velocity change (increases, decreases, remains the same, etc.)? Remember that you can ignore air resistance."
Student: decreases
High Performers: Is there something that could cause velocity to change? What can this be?
Teacher’s Support: - do not give away the answer, give some time to the student to construct the correct answer- do not provide concrete information- provide hints
Low Performers : No, this is not right. If there is nothing to cause velocity to change (for example, some force), then the velocity will remain the same. Please list all forces that are applied on the arrow while it is in flight.
Teacher’s Support: - provide explicit information regarding background knowledge- provide explicit instruction on next steps-use simple language
Future work
• Let the experts tell! Great time for a study – author dialogic support for various student types– Provide alternatives to existing tutorial dialogues– Ask for teachers’ input: what do you prefer?
what would you say?why?
The end.If you want to know more, get in touch!
(plus cat pictures!)