cognitive modeling & intelligent tutors
DESCRIPTION
A review of cognitive modeling and intelligent tutors. Presentation based on three papers, summarized below.The base paper reports on an experiment of intelligent tutoring in three urban high schools in Pittsburgh. An intelligent tutor has been made a part of 9th grade algebra, accompanying a new algebra curriculum focused on mathematical analysis of real world situations and the use of computations tools. The 470 students in experimental classes outperformed students in comparison classes by 15% on standardized tests and 100% on tests targeting the PUMP objectives. The first auxiliary paper by Anderson describes the cognitive basis for intelligent tutors, from theory to model-tracing methodology, to issues that arise in implementation. The second auxiliary paper by VanLehn describes the lessons learned in developing and testing a cognitive tutor for physics at the U.S. Naval Academy. In particular, this system was designed to run as part of a course with minimal invasion of curricular design. Interestingly, the intelligent tutors for both algebra and physics, based on different models and designed for different educational contexts, had almost identical results. It was amazing to see the long history of work on intelligent tutors, the scientific progress and implementation in schools across the country. The cognitive basis for such models is fascinating, tracing students' cognitive states in real time and modeling their knowledge as they learn new material. Yet, interaction with the tutor is simple: the tutor silently observes the students strategy, until the student asks for help or makes a mistake, and provides immediate feedback. This helps increase the quality and speed of learning as well as positively reinforce the joy (rather than the struggle) involved, keeping students motivated and moving in the right direction as they develop their problem-solving skills. However, its clear that there is a lot of work still remaining. Despite having a long history, the number of researchers in this area remains relatively small and the challenges ahead of them are large (including technical and political/social challenges).TRANSCRIPT
1
Cognitive Modeling and Intelligent Tutors
Cody A. Ray
Base slides adopted from Ken Koedinger’s presentation for 2011 Franklin Award for John R. Anderson
2
Goals of Intelligent Tutoring
• Automate education. Private tutors=$$$
• Explore epistemological issues related to the knowledge being tutored and how it can be learned.*
Anderson, Boyle, Corbett, & Lewis. Cognitive modeling and intelligent tutoring. Artificial Intelligence. 42 (1990) 7-49
* Intelligent tutoring is used to test cognitive theories, such as ACT-R
3
Cognitive Modeling
• Performance models of executing skills– Correct & incorrect rules to perform skills– Model tracing: follow in real-time the
cognitive states the student goes through in solving the problem
• Learning models of how skills acquired– Assumptions about how knowledge state
changes after each step in solving problem– Knowledge tracing: track changes in
student’s knowledge across problems
Real World Impact of Cognitive Science (PAT)
Algebra Cognitive Tutor• Based on computer
model of student learning
• Used in 2600 schools500,000 students
• Spin-off:
Koedinger, Anderson, Hadley, & Mark (1997). Intelligent tutoring goes to school in the big city.
5
Cognitive Tutor Algebra: Problems that engage intuition & interest
Health Care
Extinction
Smoking Risks
Importance of Math Education
6
Algebra Cognitive Tutor SampleAnalyze real world Analyze real world problem scenariosproblem scenarios Use graphs, graphics calculatorUse graphs, graphics calculator
Use table, Use table, spreadsheetspreadsheet
Use equations, Use equations, symbolic symbolic calculatorcalculator
Tracked by Tracked by knowledge tracingknowledge tracing
Model tracing to provide Model tracing to provide context-sensitive context-sensitive InstructionInstruction
7
ACT-R: A Cognitive Theory of Learning and Performance
• Big theory … key tenets:– Learning by doing
in addition to watching & listening
– Production rules represent performance knowledge:
These units are: Instruction implications:• modular • context specific
isolate skills, concepts, strategiesaddress "when" as well as "how"
Anderson, J.R., & Lebiere, C. (1998). Atomic Components of Thought. Erlbaum.
8
Learning in ACT-R• Declarative Knowledge and Productions• Productions
– by-product of interpretive use of declarative knowledge. Highly efficient, use-specific
• Knowledge Compilation– learning process which creates productions
• Tutoring– create experiences to acquire production
rules of a competent problem solver
9
• Cognitive Model: A system that can solve problems in the various ways students can
Strategy 1: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + ac = dStrategy 2: IF the goal is to solve a(bx+c) = d
THEN rewrite this as bx + c = d/a
Misconception: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + c = d
Cognitive Tutor TechnologyApply ACT-R to individualize instruction
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4 (2) 167-207
10
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor TechnologyUse cognitive model to individualize instruction• Cognitive Model: A system that can solve problems in
the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
If goal is solve a(bx+c) = dThen rewrite as bx+c = d/a
• Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4 (2) 167-207
11
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor TechnologyUse cognitive model to individualize instruction• Cognitive Model: A system that can solve problems in
the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
• Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
Hint message: “Distribute a across the parentheses.”
Bug message: “You need tomultiply c by a also.”
• Knowledge Tracing: Assesses student's knowledge growth -> individualized activity selection and pacing
Known? = 85% chance Known? = 45%
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4 (2) 167-207
Cognitive Tutor Algebra course yields significantly better learning
Course includes text, tutor, teacher professional development
8 of 10 full-year controlled studies demonstrate significantly better student learning
0
10
20
30
40
50
60
Iowa SAT subset ProblemSolving
Represent-ations
Traditional Algebra Course
Cognitive Tutor Algebra
Koedinger, Anderson, Hadley, & Mark (1997). Intelligent tutoring goes to school in the big city.
13
Andes Physics Tutoring
Upgrading homework support only– Same problems,
exams, lectures, etc.– other existing physics
courses can use it– Different cognitive
models, tutoring system, and context
– same results!
14
Koedinger vs Andes• Conceptual understanding*
• Multiple-choice standardized tests*
* effect size
Yet Koedinger changed curriculum and tutoring, while Andes only changed the way students completed homework.
Koedinger 1.2 0.7
Andes 1.21 0.69
Koedinger 0.3 0.3
Andes 0.25 -
15
Summary
• Intelligent tutors provide evidence for underlying cognitive theory (eg, ACT-R)
• Personalized tutors enhance learning– Examples: Algebra Tutor, Physics Tutor
• Cognitive tutoring might help independently of curricular reform – Can be used more widely, helping students
in both reformed and traditional courses
16
Thank you! Acknowledgements• Koedinger, Anderson, Hadley, & Mark (1997). Intelligent
tutoring goes to school in the big city. Artificial Intelligence in Education. 8 (1997) 30-43
• Anderson, Boyle, Corbett, & Lewis. Cognitive modeling and intelligent tutoring. Artificial Intelligence. 42 (1990) 7-49
• VanLehn, Lynch, Schulze, Shapiro, Shelby, Taylor, Treacy, Weinstein, & Wintersgill. The Andes Physics Tutoring System: Lessons Learned. Artificial Intelligence in Education. 15 (2005) 147-204