Integrated Collaborative Learning Environments with Dynamic Support

Carolyn Penstein Rosé

Language Technologies Institute/

Human-Computer Interaction Institute

Design Principle

Know what problem you are trying to

solve!!

Passivity in Class Discussions

Accountable Talk A codified set of moves that

Ts and Ss can be taught that build effective language in the classroom

Press individuals to explain and elaborate Require individuals to listen and respond to

each other Reveal misunderstandings, incomplete ideas.. Let Ss know how others are “hearing them” Spread participation through the group

Transactivity

Students explicitly display their reasoning

Students orient their contributions towards previous contributions May be competitive or

non-competitive May be representational

operational May be oriented

towards self or other

Student 1: Alright, we put that the negative and positive whole numbers.

Teacher: Alright, other ideas you want to add on, yes Mario.

Student 2: I put a um numbers that doesn't include decimal points or fractions but they include, they can include negative signs or positive signs.

Accountable Talk Move Corresponding Transactivity Move

Revoicing Clarification, Competitive Clarification, Refinement

Restate Request Paraphrase, Competitive Paraphrase

Reasoning Application Request

Extension, Competitive Extension, Contradiction, Reasoning Critique

Prompting for Further Participation.

Extension, Completion, Clarification

Request for Elaboration Extension, Completion

Challenge orRequest for Counter-Example

Competitive Paraphrase, Contradiction, Competitive Extension

Accountable Talk elicits Transactivity in group discussion

Accountable Talk elicits Transactivity in group discussion

Eddie: Well, i don't think it matters what order the numbers are in. You still get the same answer. But three times four and four times three seem like they could be talking about different things.

Teacher: Rebecca, do you agree or disagree with what Eddie is saying?Rebecca: Well, I agree that it doesn't matter which number is first,

because they both give you twelve. But I don't get what Eddie means about them saying different things.

Teacher: Eddie, would you explain what you mean?Eddie: Well, I just think that like three times four can mean three groups

of four things, like three bags of four apples. And four times three means four bags of three apples, and those don't seem like the same thing.

Tiffany: But you still have the same number of apples, so they are the same!

Teacher: OK, so we have two different ideas here to talk about. Eddie says the order does matter, because the two orders can be used to describe different situations. So Tiffany, are you saying that three times four and four times three can't be used to describe two different situations?

Reasoning Application RequestJustification Request

Request for ElaborationClarification

Reasoning Critique

Revoicing

Unequal Participation

Wizard-of-Oz Setup

Tutor

Student Student

Problem Solving Interface

Interface allowsstudents to submitIncorrect solutions

Reflection Interface

Scaffolding for Collaboration

• Prompts to encourage the instructionally beneficial behavior of students– Explanations– Equal contributions

Explanation Prompt

(Student 1 corrects the answer for the confederate learner without offering any explanation)

Student1: c is 1Auto-Prompt (to student 1): Help the student understand your correction. Student2 : what does c mean? Student1 : constant Student2 : okStudent1 : the constant in front of the variable which in our case is variable t Student2 : okStudent1 : now for the second part we use the constant function

Equal Contribution Prompt

(After the participant makes 5 correct problem actions without confederate student’s contribution)

Auto-Prompt: It seems like the other student has not contributed lately. Why don’t you see if they need help? Student1: you want to do the other half?Student2: ok

Learning effect

Significant benefit for prompting on student learning with pretest as a covariate F(1,39) = 4.47, p < .05, effect size .58

Process Losses

Another Domain: Earth Sciences

Collaborative idea generation in the Earth Sciences domain

Chinese TagHelper learns hand-coded topic analysis Human agreement .84

Kappa TagHelper

performance .7 Kappa Trained models used in

follow-up study to trigger interventions and facilitate data analysis

Example Dialogue

SpeakerSpeaker TextText

Student 1Student 1 People stole sand and stones to use for construction.People stole sand and stones to use for construction.

VIBRANTVIBRANT

Yes, steeling sand and stones may destroy the balance and Yes, steeling sand and stones may destroy the balance and thus make mountain areas unstable. Thinking about thus make mountain areas unstable. Thinking about development of mountain areas, can you think of a kind of development of mountain areas, can you think of a kind of development that may cause a problem? development that may cause a problem?

Student 2Student 2 Development of mountain areas often causes problems.Development of mountain areas often causes problems.

Student 1Student 1 It is okay to develop, but there must be some constraints.It is okay to develop, but there must be some constraints.

* Feedback during idea generation increases learning and idea generation productivity (except during the first 5 minutes) (Wang et al., 2007)

Pairs+Feedback

Individuals+NoFeedback

Pairs+NoFeedback

Individuals+Feedback

Process Analysis

Process loss Pairs vs Individuals:F(1,24)=12.22, p<.005, 1 sigma Process loss Pairs vs Individuals:

F(1,24)=4.61, p<.05, .61 sigmaNegativeNegative effect of Feedback: effect of Feedback:F(1,24)= 7.23, p<.05, -1.03 F(1,24)= 7.23, p<.05, -1.03

sigmasigmaPositivePositive effect of feedback: effect of feedback:F(1,24)=16.43, p<.0005, 1.37 sigmaF(1,24)=16.43, p<.0005, 1.37 sigma

0 5 10 15 20 25 30

02

46

810

12

Time Stamp

#Uni

que

Idea

s

Unique Ideas

Nom+NNom+FReal+NReal+F Pairs+Feedback

Individuals+NoFeedback

Pairs+NoFeedback

Individuals+Feedback

Pairs+Feedback

Individuals+NoFeedback

Pairs+NoFeedback

Individuals+Feedback

Deficient Help Exchange

Why study help?

Offering of deep help and elaborated explanations predicts post test performance (e.g., Webb et al., 2002)

Help behavior mediates learning (e.g., Gweon et al., 2006; Gweon et al., 2007)

Exchanging help in the context of collaborative learning increases identification with a learning community and motivation, and improves race relations (Sharan, 1980)

Providing help leads to feelings empowerment for “low status” students (Elbers & Hann, 2004)

How can we prompt helping behavior? DIRECT: Explicit prompts (Gweon et al., 2006) LESS DIRECT: Manipulating availability of help

from problem solving environment (Gweon et al., 2007) Girls offered more help with delayed feedback from

environment Boys offered more help with immediate help from the

environment SUBTLE: Conversation agents inject humor

(Kumar et al., 2007) Positive effects on perceived help exchange (p<.05),

effective help exchange (p<.1), and Learning (p=.06)

Collaborative Problem Solving Environment

Jan packed several books to amuse herself on along car ride to visit her grandma. After 1/5 ofthe trip she had already finished 6/8 of the booksshe brought. How many times more books shouldshe have brought than what she packed?

Extraneous Entertainment?

Coding Scheme (Gweon et al., 2007) Help provision mediates learning (Gweon et al.,

2006; Gweon et al., 2007)

(R) Help Requests: “Help me”, “I’m stuck”, “I don’t get it.”

(P) Help Provisions: “Find the common denominator”, “Try the flip and multiply strategy”

(C) Can’t help: “I don’t know”, “I’m stuck too” (D) Deny help: “ask the teacher”, “you’re an idiot”,

“press the help button” (N) Nothing

Examples[R] Student 1: What do we put on top of the fraction?[P] Student 2: Did you find a common denominator?<student 1 correctly finds the common denominator>

[R] Student 1: I don’t get it[D] Student 2: hold on<then student 1 tried something and got negative feedback from the problem solving

environment.><finally student 1 tried something else, which was correct, and got positive feedback

from the problem solving environment>

[R] Student 1: Why 16?[C] Student 2: I don’t know.

[R] Student 1: I don’t get it<student 2 tries something and gets negative feedback from the problem solving

environment><student 2 tries something else and gets negative feedback from the problem solving

environment><student 2 clicks on the help button><student 1 tries something that is correct and gets positive feedback from the problem

solving environment>

Big findings:

More help related episodes in experimental condition

More episodes where people got help, and then managed to solve

the problem themselves in the experimental condition

Vision

Vision Support for collaborative

learning is like training wheels

Effective support allows learners to achieve better collaboration

Unnecessary support can be demotivating

Fading support is ideal But too little support can be

detrimental as well Trained human facilitators

are able to achieve the right balance

Vision Support for collaborative

learning is like training wheels

Effective support allows learners to achieve better collaboration

Unnecessary support can be demotivating

Fading support is ideal But too little support can be

detrimental as well Trained human facilitators

are able to achieve the right balance

Vision Support for collaborative

learning is like training wheels

Effective support allows learners to achieve better collaboration

Unnecessary support can be demotivating

Fading support is ideal But too little support can be

detrimental as well Trained human facilitators

are able to achieve the right balance

Vision Support for collaborative

learning is like training wheels

Effective support allows learners to achieve better collaboration

Unnecessary support can be demotivating

Fading support is ideal But too little support can be

detrimental as well Trained human facilitators

are able to achieve the right balance

Vision Support for collaborative

learning is like training wheels

Effective support allows learners to achieve better collaboration

Unnecessary support can be demotivating

Fading support is ideal But too little support can be

detrimental as well Trained human facilitators

are able to achieve the right balance

Support for collaborative learning is like training wheels

Effective support allows learners to achieve better collaboration

Unnecessary support can be demotivating

Fading support is ideal But too little support can be

detrimental as well Trained human facilitators

are able to achieve the right balance

Vision

Architecture for Dynamic Collaboration Support

VMT-Basilica

** Students learn up to 1.25 standard deviations more when interactive support is provided in the environment.

Basilica Architecture

A channel filter collects all of the

events that occur at the interface from

all students involved in the conversation

Basilica Architecture

Events are then channeled to

special purpose filters such as a text

processing filter

Other filters may keep track of time

or other factors not related to student

behavior

Basilica Architecture

Filters that are related to decision making for specific

types of interventions

receive notifications from the analysis filters and pass

notifications on to the Actors, which are responsible to

launching interventions

Basilica Architecture

Actors of different types produce

events that are sent to the Outgoing

Message Spooling Filter

From there, events are sent through the

Presence Actor to the interface.

Issues to consider What problem are you trying to solve?

Formulate analysis scheme

When should you intervene?Use technologies like TagHelper and SIDE to

track interaction and trigger support

What should the intervention be?Technologies like TuTalk can be used to offer

support

Questions?