Tutoring and Help Systems

Tell me and I forget.

Show me and I remember.

Involve me and I understand.

- Chinese proverb

Previous Approach

• Used for over 20 years– Computer-based training (CBT)– Computer aided instruction (CAI)

• Effective in helping learners, but do not provide the same attention a human tutor can provide– Approach to a solution– Individual problem solving style

A New Approach

• Previous approaches focused on scripted information about the domain

• New approach must reason about both the domain and the learner

• Allowing greater versatility in systems interaction with students

Intelligent Tutoring System

Possesses two intelligent properties:

1. Generate problem solution– Flexibility within problem domain– Able to explain errors

2. Adapt to user needs– User knowledge models

Model of Traditional ITS

Components in Tutoring System

Domain Knowledge

Expert Model

Pedagogical Module Communication

Model

Student Model

Student Model• Used to tailor instruction for each student• Must represent the student’s knowledge with

respect to domain– Choice of representation

• Must store pedagogical information about the student– Student’s preferences– Problem solving style

• General information– Acquisition and retention

Student Model Representation

• Typically represented with overlays

• Student’s knowledge as a subset of expert’s knowledge

Overlay Student Model

Student’s

Knowledge

Expert’s Knowledge

Overlay With Buggy Extensions

Expert’s Knowledge

Student’s Buggy

Knowledge

Shared Knowledge

Pedagogical Module

• Uses information from the student model to determine what to present to learner– New Material from the Domain– Review of Previous Topic– Feedback on Current Topic

• Teaching Meta-Strategy vs. Low Level Issues

Low Level

• Topic Selection– Examine student model for areas of focus

• Problem Generation– Difficulty based on student’s ability level (taken from

student model)

– Size of question depend on granularity of domain

• Feedback– How much & What kind

Meta-Strategy• Implementing strategy has been a

formidable problem• Ideal to have many strategies to choose

from based on student model– Realistically many ITSs only have one

• Difficulty in representing knowledge impedes some methods– Socratic method requires “Common Sense”

…Enter CBR

• Individualizing will depend on two issues1. Information about how learner solved tasks

2. Using this information in subsequent tutorial decisions

• Storing this information builds cases• Cases from other learners• Pre-stored cases - Pitfalls domain experts

have foreseen

Two Goals of CBR Tutoring

• Case-based Adaptation– Adapt interface components to the user’s needs

– CBR that not only uses pre-stored cases but also stores new cases can be adapted

– CHEF: Recipe and Taste

• Case-based Teaching– Provide user with cases that help solve current problem

– Observe user solving problem – cases can be used as a reminder

Two Ways to Store a Case

1. Case is stored as a whole– Most systems use this approach– Show examples or give advice

2. Case is stored as a snippet– Describes sub goals of problems within

particular context– Used to find problem solving path– Application used in ELM

Episodic Learner ModelELM

• Analyzes solutions (or partial solutions) to programming problems in LISP

• Looks for problem solving errors and returns feedback

• Used in diagnostic process

• Able to return examples and remindings– EBR: Explanation-based retrieval

ELM

• Stores user model in a collection of episodes (cases)

• User code is analyzed to create a derivation tree consisting of concepts and rules

• These concepts and rules are instantiations of units from the knowledge base

ELM Knowledge Representation

• Represented in hierarchically organized frames• Concepts

– Knowledge about the language (concrete procedures and semantic concepts)

– Schemata of common algorithmic and problem solving knowledge (eg recursion)

• Additional information– Plan transformations for semantically equivalent

solutions• Bug rules for derivations which may result from

confusion

Bug Rule•Bug Code

•Ideal Code

Append:(APPEND “a” “bcd”)(APPEND (a) (bcd))

Append:(APPEND ‘(a) ‘(bcd))

ELM Diagnostic

• Code is at least syntactically correct• Starts with task description related to higher

concepts in the knowledge base• Most concepts have transformations describing

semantically equivalent variations– Ordering of clauses or sequence of arguments

• The sequence of testing transformations is determined by the student model

ELM Diagnostic Cont.

• A set of rules is indexed by concepts describing different ways to solve the goal– Good– Bad– Buggy

• Applying a rule results in comparison between plan and student’s code

• Diagnostic process is called recursively on further concepts– Results in derivation tree

ELM Derivation Tree

• Information in tree added to episodic model– Instances of concepts and rules

• Context

• Transformations and argument bindings

• Each concept (level) in tree creates a frame• The set of episodic frames of a particular

episode constitutes a case– Can later be indexed by first frame in case to

rebuild tree

Partial Derivation Tree:

(NIL-TEST(FIRST-ELEMENT(PARAMETER?LIST)))

NIL-TEST

Empty-List-Nil-Test-Rule

(NULLTEST(FIRST-ELEMENT(PARAMETER?LIST)))

NULLTEST

Unary-Func-Rule

(NULL-OP) (FIRST-ELEMENT(PARAMETER?LIST))

NULL-OP FIRST-ELEMENT

Correct-Coding-Rule Unary-Func-Rule

null (FIRST-ELEM-OP) (PARAMETER ?LIST)

FIRST-ELEM-OP PARAMTER

Correct-Coding-Rule Correct-Param-Rule

car li

Student CodeSimple And:(defun simple-and(li) (cond ((null li) t) ((null (car li)) nil) (t (simple-and (cdr li)))))

Derivation Tree

ELM

LISP Code

Diagnosis(Explanation)

Derivation Tree(Explanation Structure)

Task Description

Domain Knowledge

Learner Model Generalization

Explanation-Based Retrieval

• System generates a solution based on concepts and rules and temporarily stores this solution in case base

• All episodic frames that are neighbors contribute to computing weights for similarity

• Most similar case is retrieved (based on previous explanations) and temporary solution is deleted

ELM-Programming Environment

• Intelligent analysis of task solutions– Diagnostic tool based on ELM– Gives user feedback on purposed solution– Directs user with hints

• Example-based Programming– Can reuse code from pre-installed cases or the user’s

own previous experience

• Example-based Explanation– Shows examples based on matching of expected

solution with previous cases already in learner model

ELM-Adaptive Remote Tutor

• HTML Implementation of ELM-PE• Conceptual network of topics

– Red light, green light

• Example-based programming– Can find the most relevant example from case history

• Demonstration of ELM-ART– http://apsymac33.uni-trier.de:8080/elm-art/login-e

Static vs Dynamic CB Teaching• Static

– Problem design facilitates the diagnosis of failure– Cases (failures) are associated with supporting case to

help overcome failure– Limited by case-base

• Dynamic– Problems solved twice, by learner and system– System solution used as an index for supporting cases

• Model Tracing– Similar to dynamic, but solution used for direct

feedback (could limit multiple solution paths)

Case-based Chess Endgame Tutor

• Dynamic Teaching• Chess heuristics are not given, instead must be

inferred– First given examples to watch

– Next examples to solve

• CACHET structures this learning by recognizing sub optimal moves and providing hints that lead in the right direction

CACHET Case Libraries

• Pre-defined cases– Prototypical informative games

• Cases generated on demand– Able to generate scenarios for learner

• Cases produced by the learners themselves– Self-generated cases are very successful for

remindings– Useful to system as point of intervention

Roger Schank

• President and CEO of Socratic Arts

• Founder of Institute for the Learning Sciences

• Research on AI and cognitive learning theory

• Focus on e-learning

Cognitive Learning Theory

• A general approach that views learning as an active mental process of acquiring, remembering, and using knowledge.

• Learning is evident by a change in knowledge which makes a change in behavior possible.

• Learning itself is not directly observable.

Schank’s Criticisms• Schools act as if learning can be disassociated

from doing• Schools believe they have the job of assessment

as part of their natural role• Schools believe they have an obligation to create

standard curricula• Schools believe studying is an important part of

learning • Schools believe students have a basic interest in

learning whatever it is schools decide to teach them

Schank’s Idea

• Case-based reasoning: Understand the universe by matching incoming events to past experiences– The Steak and the Haircut

• Knowledge is built on the ability to index and make sense of cases– It is not a set of facts!

• You must question to learn

Conclusion

• CBR can effectively be applied to enhance tutoring systems

• Cases can be complete or snippets• Cases can include buggy information• Cases are applied either diagnostically or

adaptively• ELM-PE and ELM-ART use cases

diagnostically forming a derivation tree