carolyn penstein rosé language technologies institute human-computer interaction institute school...
TRANSCRIPT
![Page 1: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/1.jpg)
1
Carolyn Penstein RoséLanguage Technologies Institute
Human-Computer Interaction Institute
School of Computer Science
With funding from the National Science Foundation and the Office of Naval Research
Souflé: A Three Dimensional Framework for Analysis of Social Positioning
in Dyadic and Group Discussions
![Page 2: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/2.jpg)
2
Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…
Human learning
Health
Wellbeing
![Page 3: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/3.jpg)
3
Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…
Human learning
Health
Wellbeing
Human learning
![Page 4: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/4.jpg)
4
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
![Page 5: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/5.jpg)
5
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
![Page 6: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/6.jpg)
Purpose of Interaction in Learning Contexts
Reward structures encourage students to focus on performance over learning
Well crafted instruction provides opportunities for learning
Opportunities only help if students take them
Take Home Message:
Introducing reflection points provides opportunities for students to take advantage of learning resources
![Page 7: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/7.jpg)
CarefullyStructuredConceptualKnowledge
Important Ingredients for Learning
7
Reflectionthrough
RichInteraction
![Page 8: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/8.jpg)
End of Fall Semester: Students learn about Rankine Cycles 1 Week of lectures Homework assignment on analysis of
Rankine Cycles Tutorial on using CyclePad software
package (Developed at Northwestern University (Forbes et. al. 1999) Allows students to construct and
analyze a variety of Thermodynamic Cycles)
Instructed on Effects of Changing System Variables (Temperature, Pressure) on System Output (Power, Waste Heat)
Second-Year Thermodynamics
![Page 9: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/9.jpg)
Learning Goal: Encourage students to reflect on interactions between cycle parameters
Collaborative Task
Reduction in Steam Quality
Power
Waste Heat
Increasing heat increases power but also waste heat
Increasing pressure increases efficiency
Design Goal: Design a power plant based on the Rankine Cycle paradigm
Competing Student Goals: Power: Design a power plant that achieves
maximum power output Motivated by economic concerns
Green: Design a power plant that has the minimum impact on the environment Motivated by environmental concerns
Each pair turns in exactly one design
![Page 10: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/10.jpg)
Computer Supported Collaborative Learning
![Page 11: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/11.jpg)
11
![Page 12: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/12.jpg)
12
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
![Page 13: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/13.jpg)
Souflé Framework(Howley et al., in press)
13
Person Person
3 Dimensions: Transactivity Engagement Authoritativeness
Souflé Framework(Howley et al., in press)
![Page 14: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/14.jpg)
14
Sociolinguistics
Discourse Analysis
LanguageAnd
Identity
LanguageUse
MachineLearning
Multi-Level
Modeling
AppliedStatistics
ComputationalModels
OfDiscourseAnalysis
![Page 15: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/15.jpg)
Souflé Framework(Howley et al., in press)
15
Transactive Knowledge Integration
Person Person
![Page 16: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/16.jpg)
16
i
• Definition of Transactivity• building on an idea expressed earlier in a conversation • using a reasoning statement
We don't want tmax to be at 570 both for the material and [the Environment]
well, for power and efficiency, we want a high tmax, but environmentally, we want a lower one.
![Page 17: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/17.jpg)
17
![Page 18: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/18.jpg)
18
![Page 19: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/19.jpg)
19
![Page 20: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/20.jpg)
Findings
Moderating effect on learning (Joshi & Rosé, 2007; Russell, 2005; Kruger & Tomasello, 1986; Teasley, 1995)
Moderating effect on knowledge sharing in working groups (Gweon et al., 2011)
Computational Work
Can be automatically detected in:
Threaded group discussions (Kappa .69) (Rosé et al., 2008)
Transcribed classroom discussions (Kappa .69) (Ai et al., 2010)
Speech from dyadic discussions (R = .37) (Gweon et al., 2012)
Predictable from a measure of speech style accommodation computed by an unsupervised Dynamic Bayesian Network (Jain et al., 2012)
Transactivity (Berkowitz & Gibbs, 1983)
![Page 21: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/21.jpg)
Engagement Engagement
Souflé Framework(Howley et al., in press)
21
Transactive Knowledge Integration
Person Person
![Page 22: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/22.jpg)
Engagement(Martin & White, 2005, p117)
System of Engagement Showing openness to the
existence of other perspectives
Less final / Invites more discussion
Example: [M] Nuclear is a good choice [HE] I consider nuclear to be
a good choice [HC] There’s no denying that
nuclear is a superior choice [NA] Is nuclear a good
choice? 22
![Page 23: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/23.jpg)
23
![Page 24: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/24.jpg)
Findings
Correlational analysis: Strong correlation between displayed openness of group members and articulation of reasoning (R = .72) (Dyke et al., in press)
Intervention study: Causal effect on propensity to articulate ideas in group chats (effect size .6 standard deviations) (Kumar et al., 2011)
Mediating effect of idea contribution on learning in scientific inquiry (Wang et al., 2011)
Engagement (Martin & White, 2005)
![Page 25: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/25.jpg)
Authority
Authority
Engagement Engagement
Souflé Framework(Howley et al., in press)
25
Transactive Knowledge Integration
Person Person
![Page 26: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/26.jpg)
Analysis of Authoritativess
26
Water pipe analogy: Water = Knowledge or Action Source = Authoritative speaker Sink = Non-authoritative Speaker
Authoritativeness Ratio = Source Actions Actions
![Page 27: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/27.jpg)
27
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary
Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
Additionally…ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
![Page 28: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/28.jpg)
28
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary
Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
![Page 29: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/29.jpg)
29
Source
or Sink?
Primary
Se
condary
Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
The Negotiation Framework(Martin & Rose, 2003)
![Page 30: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/30.jpg)
Source
or Sink?
Primary
Se
condary
Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
The Negotiation Framework(Martin & Rose, 2003)
![Page 31: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/31.jpg)
31
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary
Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
Additionally…ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
![Page 32: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/32.jpg)
32
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary
Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
Additionally…ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
K1 + A2
K1 + K2 + A1 + A2
Authoritativeness:
![Page 33: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/33.jpg)
33
K2?
![Page 34: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/34.jpg)
34
Set up!
K1
K2
![Page 35: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/35.jpg)
35
![Page 36: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/36.jpg)
Findings Authoritativeness measures display how students respond to
aggressive behavior in groups (Howley et al., in press) Authoritativeness predicts learning (R = .64) and self-efficacy (R
= .35) (Howley et al., 2011) Authoritativeness predicts trust in doctor-patient interactions (R
values between .25 and .35) (Mayfield et al., under review)Computational Work
Detectable in collaborative learning chat logs (R = .86)
Detectable in transcribed dyadic discussions in a knowledge sharing task (R = .95) (Mayfield & Rosé, 2011)
Detectable in transcribed doctor-patient interactions (R = .96) (Mayfield et al., under review)
Authoritativeness (Martin & Rose, 2003)
![Page 37: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/37.jpg)
37
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
![Page 38: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/38.jpg)
Computer Supported Collaborative Learning
![Page 39: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/39.jpg)
39
AutomaticAnalysis
Of Conversation
ConversationalInterventions
PositiveLearning
Outcomes
![Page 40: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/40.jpg)
6 Years of Positive Results
Foundational study: students work with a partner and dialogue agent for support Learn 1.24 s.d. more than individuals without support (Kumar et al., 2007a)
Results inform iterative design of agent behavior Personalized agents increase supportiveness and help exchange between students (Kumar et al.,
2007b) Agents are more effective when students have control over timing of the interaction (Chaudhuri
et al., 2008; Chaudhuri et al., 2009) Agents that employ Balesian social strategies are more effective than those that do not (Kumar
et al., 2010; Ai et al., 2010) Students are sensitive to agent rhetorical strategies such as displayed bias (Ai et al., 2010),
displayed openness to alternative perspectives (Kumar et al., 2011), and targeted elicitation (Howley et al., 2012)
Bazaar architecture enables efficient, principle based agent development (Kumar & Rosé, 2011; Adamson & Rosé, 2012)
![Page 41: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/41.jpg)
41
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
![Page 42: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/42.jpg)
42
• Transactivity is a conversational behavior that is important for learning
• Authoritativeness and Engagement are dimensions of conversation that play a supporting role
• Positioning students to exchange Transactive contributions
• We have made progress at automating analysis of Transactivity and Authoritativeness
•Automated analysis enables dynamic triggering of supportive interventions for online group learning
Conclusions and Current Directions
![Page 43: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/43.jpg)
43
•In the future, CSCL activities will be part of societies of online learners
• Vision began with Virtual Math Teams/ The Math Forum• Now we’re already seeing the shift through companies
like Coursera and Udacity
•We are taking steps towards this future• Fully distance learning studies (UCSB, Drexel)• Sustainable CSCL (online office hours agent)
•As we look to the future: we must understand the emergent effects of our local interventions in order to maximize positive benefit on a grand scale
Conclusions and Current Directions
![Page 44: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/44.jpg)
44
Thank You!
Questions?
![Page 45: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/45.jpg)
Revoicing Agent
45
![Page 46: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/46.jpg)
46
Experimental Paradigm Day 1: Pretest on conceptual questions related to the unit (Diffusion or
Punnett Squares) Day 2: Online collaborative activity + Immediate Posttest isomorphic to
pretest Day 3: Whole class teacher led discussion + Delayed Posttest isomorphic to
pretestParticipants: consenting 9th grade biology students, randomly
assigned to groups of 3Experimental Design: Simple between subjects design
Groups randomly assigned to Revoicing condition or Control Condition
Revoicing Agent Studies
![Page 47: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/47.jpg)
Revoicing Agent
47
![Page 48: Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute School of Computer Science With funding from the National Science](https://reader035.vdocuments.net/reader035/viewer/2022062308/56649d9e5503460f94a88cc7/html5/thumbnails/48.jpg)
48
Study 1: Year 1, Diffusion Lab (50 students) Students learn more on explanation questions in supported conditions
(F(1,46) = 4.3, p < .05, effect size 1 standard deviation) Students in supported conditions more active in whole group discussion
(F(2,26) = 4.2, p < .05, effect size .75 standard deviations)
Study 2: Year 2, Diffusion Lab (78 students) Students learn more on immediate post test in Revoicing Agent
condition (F(1,74) = 4.3, p < .05, effect size .51 standard deviations)
Study 3: Year 2, Punnett Square Lab (78 students) Students learned more on delayed post-test in Revoicing Agent
condition
Results of CSCL Studies