in intelligent tutoring and ap dept i of stan-cs-87-i175 ... · 7 -ri8 53 using and evaluating...

41
7 -Ri8 53 USING AND EVALUATING DIFFERENTIAL MODELING IN INTELLIGENT TUTORING AND AP (U) STANFORD UNIV CA DEPT I OF COMPUTER SCIENCE D C WILKINS ET AL JAN 87 UNCLAS;SIFIED STAN-CS-87-i175-REV-1 N88844-85-K-03:85 F/G 5,'6 IU. 7

Upload: others

Post on 10-Aug-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

7 -Ri8 53 USING AND EVALUATING DIFFERENTIAL MODELING IN

INTELLIGENT TUTORING AND AP (U) STANFORD UNIV CA DEPTI OF COMPUTER SCIENCE D C WILKINS ET AL JAN 87UNCLAS;SIFIED STAN-CS-87-i175-REV-1 N88844-85-K-03:85 F/G 5,'6 IU.

7 •

Page 2: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

11.8

11111l.25~~111 .4 "I 16

MICROCOPY RESOLUTION TEST CHARTN4ATIONAL BUREAU OF STANDARDS 1963-A

Page 3: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

r1iC FILE r Rpot o.STN-S-7-115

Januairy 1987ReotN.s-NC8715Also numbered KSIL-86-62

* Using and Evaluatd~ng D~c~ta 'odeling inIntelligent Tutorn an

Apprentice Learning Systems

by0

1). C. Wilkins, WV. J. Clancei. ard 13. G. Buchanan

00

Department of Computer Science

Stanford UpivmrsityStanford, CA 94305

DTICDE 1419870

9,-) 76,0

DISTRIBUTfONSTfATEMNT A

Approved for P'ibUc reI.=*e:

Page 4: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

SECURITY CLASSIFICATION OF 'HiS PAGE

REPORT DOCUMENTATION PAGEla REPORT SECURITY CLASSIFICATION lb RESTRICTIVE MARKINGS

UNCLASSIFIED2a SECURITY CLASSIFICATION AUTHORITY 3 DISTRIBUTION/AVAILABILITY OF REPORT

APPROVED FOR PUBLIC RELEASE,2b. DECLASSIFICATION / DOWNGRADING SCHEDULE DISTRIBUTION UNLIMITED

4 PERFORMING ORGANIZATION REPORT NUMBER(S) S. MONITORING ORGANIZATION REPORT NUMBER(S)

STAN-CS-87-1175 or KSL-86-62 ONR TECHNICAL REPORT # 23

6a NAME OF PERFORMING ORGANIZATION 6b. OFFICE SYMBOL 7a NAME OF MONITORING ORGANIZATION

STANFORD KNOWLEDGE SYSTEMS (if applicable) PERSONNEL AND TRAINING RESEARCH PROGRAMSLABORATORY I

6c. ADDRESS (City, State, and ZIPCode) 7b ADDRESS (City, State, and ZIP Code)COMPUTER SCIENCE DEPARTMENT OFFICE OF NAVAL RESEARCH (CODE I142PT)701 WELCH ROAD, BUILDING C 800 NORTH QUINCY STREETPALO ALTO, CA 94304 ARLINGTON, VA 22217-5000

8a. NAME OF FUNDING/SPONSORING 8b. OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBERORGANIZATION (If applicable)

I_ N00014-85K-0305

8c ADDRESS (City, State, and ZIP Code) 10 SOURCE OF FUNDING NUMBERS

PROGRAM PROJECT TASK WORK JNITELEMENT NO NO. NO ACCESSION NO

I I TITLE (Include Security Classification)

.'USING AND EVALUATING DIFFERENTIAL MODELING IN INTELLIGENT TUTORING AND APPRENTICELEARNING SYSTEMS

12 PERSONAL AUTHOR(S)DAVID C. WILKINS, WILLIAM J. CLANCEY AND BRUCE G. BUCHANAN

13a TYPE OF REPORT 13b TIME COVERED 14. DATE OF REPORT (Year, Month, Day) 15 PAGE COUNTTECHNICAL FROM TO JAN 1987 37

'6 SUPPLEMENTARY NOTATIONKSL o86-62; TO APPEAR IN INTELLIGENT TUTORING SYSTEMS: LESSONS LEARNED, (J. PSOTKA,

D. MASSEY AND S. MUTTER, EDITORS) LAWRENCE ERLBAUM, 198717 COSATI CODES 10 SUBJECT TERMS (Continue on reverse if necessary and identify by block number)

PIELD GROUP SUB-GROUP LEARNING, KNOWLEDGE ACQUISITION, TUTORING, DEGUGGING,U9.

DIFFERENTIAL MODELING.

'9 ABSTRACT (COntIrl. on reverse if necessary and identify by block number)

A powerful approach to debugging and refining the knowledge structures of a prob-

S leni solving agent is to differentially model the actions of the agent against a gold

standard. This paper proposes a framework for exploring the inherent limitations of

such an approach when a problem solver is differentially modeled against an expert

system. A procedure is described for determining a performance upper bound for de-

bugging via differential modeling, called the synthetic agent method. The synthetic

agent method systematically explores the space of near miss training instances and

expresses the limits of debugging in terms of the knowledge representation and

control language constructs of the expert system. Vi,, , :

20 DISTRIBUTION/AvAILABILITY OF ABSTRACT 21 ABSTRACT SECURITY CLASSIFICATIONNCLASSFIED/UNLIMITED 6 SAME AS RPT 0 DTC USERS UNCLASSIFIED

22a NAME OF RESPONSIBLE NDIVIDUAL 22b TELEPHONE (Include Area Code) 22c OFFIC LDR. SUSAN CHIPMAN (202) 696-4318 ONR 1142PT

% DO FORM 1473,84 MAR 83 APR ed,tion may be used until exhausted SECURITY CLASSIFICATION OF THIS PAGEAll other edtions are obsolete

O0

IT1rMama"

Page 5: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Knowledge Systems Laboratory October 1986KSL Report No. KSL-86-62 Rev. 1, January 1987

Using and Evaluating Differential Modeling in

Intelligent Tutoring and Apprentice Learning Systems

David C. Wilkins, William J. Clancey and Bruce G. Buchanan

Knowledge Systems LaboratoryDepartment of Computer Science

Stanford UniversityStanford, CA 94305

' aslon ~For

TAB El

*To appear in: iouncedif Icat ion

Intelligent Tutoring Systems: Lessons Learned, J. Psotka, D. Massey--and S. Mutter, editors, Lawrence Erlbaum Publishers, 1987

LD1tribut 1on/.. . .AvailabilitY Codes

IAvail and/or-

D at Special

044

Page 6: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Contents

1 Introduction 2

2 The Process of Differential Modeling 4

2.1 Previous work in differential modeling ..................... 5

2.2 Assumptions and issues in differential modeling ............... 7

3 Performance Evaluation Issues 8

3.1 Performance evaluation and the synthetic agent method ......... 9

3.2 Knowledge-oriented vs. performance-oriented validation ....... .. 10

3.3 Capability-oriented vs. limitation-oriented validation ............ 12

4 Synthetic Agent Method of Validation 13

4.1 The synthetic agent method ............................ 17

4.2 Discussion of synthetic agent method ....................... 18

4.3 Categories of errors .................................. 19

5 Application of Synthetic Expert Method 19

' 6 Summary 22

7 Acknowledgments 23

8 References 24

.4.

Oq"

Page 7: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Using and Evaluating Differential Modeling in

Intelligent Tutoring and Apprentice Learning Systems

David C. Wilkins, William J. Clancey and Bruce G. Buchanan

Knowledge Systems LaboratoryDepartment of Computer Science

Stanford UniversityStanford, CA 94305

Abstract

A powerful approach to debugging and refining the knowledge structures of a

problem-solving agent is to differentially model the actions of the agent against

a gold standard. This paper proposes a framework for exploring the inherent limi-

tations of such an approach when a problem solver is differentially modeled against

an expert system. A procedure is described for determining a performance upper

bound for debugging via differential modeling, called the synthetic agent method.

The synthetic agent method systematically explores the space of near miss training

instances and expresses the limits of debugging in terms of the knowledge represen-

tation and control language constructs of the expert system.

1 Introduction

Artificial Intelligence has long been interested in methods to automatically refine

0 and debug an intelligent agent. This is a central concern in machine learning and

automatic programming, where the agent to be improved is a program. It is also

a central concern in intelligent tutoring, where the agent to be improved is a hu-

man problem solver. Many Al systems for improving an intelligent agent involve

2

Page 8: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

differential modeling of the agent against the observable problem-solving behavior

of another agent. We focus on the situation where one of the agents is a knowledge-

based expert system and the knowledge structures to be improved encode factual

information that is declaratively represented'.

This paper describes the synthetic agent method, which allows calculation of

a performance upper bound on improvement to an intelligent agent attainable by

differential modeling of the agent against an expert system. A performance up-

per bound identifies missing or erroneous knowledge in an intelligent agent that a

particular differential modeling system is inherently incapable of identifying. By

contrast, most performance evaluation procedures aim to determine a performance

lower bound; they experimentally demonstrate that a particular differential model-

ing system can successfully identify some missing or erroneous knowledge.

The synthetic agent method involves replacing the human problem solver in a

Rdifferential modeling scenario with a synthetic agent that is another expert system.

The knowledge in the synthetic agent expert system is systematically modified to be

slightly different than the knowledge in the original expert system. The knowledge

in the synthetic agent is modified to be slightly 'better' in an apprenticeship learning

scenario and slightly 'worse' in an intelligent tutoring scenario.

This paper is organized as follows. Section 2 surveys previous and current

work on improving an intelligent agent via differential modeling. Section 3 identi-

fies important performance evaluation issues related to evaluation of a differential

modeler. Section 4 presents and discusses the synthetic agent method. Finally,

Section 5 describes an application of the synthetic agent method that is currently

underway.

This paper presents our framework for evaluating a differential modeling sys-

tem. No experimental results are given. A future paper will describe the use of the

framework to evaluate the ODYSSEUS modeling program (described in Section 5) in

the context of intelligent tutoring and apprenticeship learning.

'As much domain-specific knowledge as possible is declaratively represented in a well designed

knowledge-intensive expert system. Domain-specific procedural knowledge is contained in an expertsystem shell for the generic problem class (Clancey, 1984).

3

N . . .

Page 9: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

2 The Process of Differential Modeling

Many AI systems that debug and refine an intelligent agent employ a method called

differential modeling; this is the process of identifying differences between the ob-

served behavior of a problem-solving agent and the behavior that would be expected

in accordance with an explicit model of problem solving.

Statement of Problem

PSDME

Problem Differential ExpertSolver Modeler System

Answer Answer

Knowledge DifferencesBetween PS and ES

Figure 1: A general model of the differential modeling process.PS solves a problem, and DM finds differences between knowledgestructures of PS and ES. In this paper, equal attention is given tothe situation of apprenticeship learning where PS is a human ex-pert and the goal is to improve ES; and the situation of intelligenttutoring where PS is a student and the goal is to improve PS.

The differential modeling process is illustrated in Figure 1. The three major

elements are a problem solver (PS), a differential modeler (DM), and a knowledge-

based expert system (ES). The task of the DM is to identify differences between theknowledge structures of PS and ES in the course of watching PS solve a problem, for

example a medical diagnosis problem. In the figure, Answer consists of all observable

behavior of the respective problem solver. The DM can be quite complex and can

easily exceed the complexity of the ES.

4

@41

Page 10: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Two major tasks that confront a DM are global and local credit assignment,

which are performed by a global and local learning critic, respectively. The global

critic determines when the observable behavior of PS suggests a difference between

the knowledge structures of ES and PS. In such a situation, the local critic is

summoned to identify possible knowledge differences between ES and PS that are

suggested by the actions of PS. A complete learning system consists of a global critic,

local critic and a repair component (Dietterich and Buchanan, 1981); discussion of

the repair stage is beyond the scope of this paper.

2.1 Previous work in differential modeling

AI systems that employ a differential modeling approach to debugging and refining

a problem-solving agent are found in the areas of machine learning, automatic

programming, and intelligent tutoring. We first describe systems that do not employ

a knowledge-based expert system as the explicit model of problem solving and then

describe systems that do.

The earliest such systems were in the rea of machine learning, notably,

Samuel's checker player and Waterman's poker player (Samuel, 1963; Waterman,

1970). The PS used by Samuel's DM program was a book of championship checker

games. The DM global critic task was accomplished by comparing the move of PS

to the move that Samuel's program made in the same situation. The local critic task

was accomplished by adjusting the coefficients of a polynomial evaluation function

for selecting moves so that the action of the program equaled the action of PS. A

recent example of machine learning research that uses a differential modeling ap-

proach is the PRE system for theory-directed data interpretation (Dietterich, 1984).

PRE learns programs for Unix commands from examples of the use of the commands.,14.

The DM employs constraint propagation to identify differences between the PS and

the programs for commands.

In automatic programming, the synthesis of LISP and PROLOG functions from

example traces falls under the rubric of debugging via differential modeling (Bier-

mann, 1978; Shapiro, 1983). The PS consists of the input/output behavior of a

5. I

Page 11: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

correct program. The DM modifies the program being synthesized whenever it

does not give the same output as PS when given the same input.

In intelligent tutoring the goal is to 'debug' a human problem solver. Many

intelligent tutoring systems contain an expert system and use a differential model-

ing technique, including the WVEST program in the domain of games (Burton and

Brown, 1982), SOPHIE III and GUIDON in the domain of diagnosis (Brown et al.,

19S2: Clancey, 1979), and the MACSYMA-ADVISOR in the domain of symbolic inte-

gration (Genesereth. 19S2). SOPHIIE III uses an expert system for circuit diagnosis

as an aid in isolating hypothesis errors in the behavior of students who are perform-

ing electronic troubleshooting. GUIDON is built over the MYCIN expert system for

medical diagnosis (Buchanan and Shortliffe, 1984); student hypothesis errors are

discovered in the process of conducting a Socratic dialogue.

Recent research within machine learning also uses an expert system as the

explicit model of problem solving, especially within the subarea of apprenticeship

learning. Apprenticeship learning is defined as a form of learning that occurs in

the context of normal problem solving and uses underlying theories of the problem

solving domain to accomplish learning. Examples of apprenticeship learning systems

are LEAP and ODYSSEUS. The LEAP program refines knowledge bases for the VEXED

expert system for VLSI circuit design (Mitchell et al., 1985). PS is a circuit designer

who is using the VEXED circuit design aid and the underlying theory used by the DM

is circuit theory. ODYSSEUS refines and debugs knowledge bases for the IIERACLES

.4 expert system shell, which solves problems using the heuristic classification method.: .- (Wilkins, 1986). When the ODYSSEUS problem domain is medical diagnosis, PS

is a physician diagnosing a patient. The DM uses two underlying theories, the

* principal one being a strategy theory of the problem-solving method. ODYSSEUS is

also applicable to intelligent tutoring; it functions as a student modeling program

for the GUIDON2 intelligent tutoring system (Clancey, 1986a).

,0,1

S,.

6

04

,#0

wGw p* ~-

Page 12: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

2.2 Assumptions and issues in differential modeling

Much of the power of an expert system derives from the quantity and quality of its

domain-specific knowledge. For the purposes of this paper, the principal function

of the differential modeler is to find factual domain knowledge differences between

the problem solver and the expert system. Our work assumes that the expert sys-

tem represents domain knowledge declaratively, including domain-specific control

knowledge. Further, as much as possible, the knowledge is represented indepen-

dently of how it will be used by problem-solving programs. This practice facilitates

use of the same domain knowledge for different purposes, such as problem solving,

explanation, tutoring and learning.

The framework provided by this paper for understanding the limits of de-

* 'bugging via differential modeling has been fashioned with the following assump-

"' tions in mind. First, we assume that an agent is differentially modeled against aV' knowledge-based expert system that is capable of solving the problems presented

to the human PS. Second, we assume that the observed actions of the agent consist

of normal problem-solving behavior in a domain. And third, we assume that the

goal of the differential modeling system is to discover factual domain knowledge

differences between the agent and the expert system's knowledge base, as opposed

to the discovery of procedural control knowledge differences; procedural knowledge

involves sequencing constructs such as looping and recursion.

There are many open questions regarding debugging via differential modeling

against an expert system. For instance, what are the types of knowledge in the PS

that can and cannot be debugged using a differential modeling approach? What

characteristics and organization of an ES facilitate differential modeling? What

characteristics and organization impose inherent limitations? How can the strengths

and weaknesses of a particular DM be best described? The evaluation methodology

proposed in this paper, called the synthetic agent method, provides a framework

for the exploration of these questions.

7

Page 13: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

3 Performance Evaluation Issues

DM performance evaluation is intimately related to ES performance evaluation. The

function of a DM is to improve the performance of an ES and so DM performance

evaluation requires ES performance evaluation. Although ES evaluation is a difficult

and time consuming task, there is agreement on the general approach that should be

taken when ES is an expert system program. Examles of performance evaluation

studies based on a sound methodology are the evaluations of the MYCIN, INTERNIST

and RL expert systems (Yu et al., 1979; Miller et al., 1984; Fu and Buchanan, 1985).

Two major functions of a DM are global and local credit assignment 2. Thegeneral problem of assessing the limits of a DM consists of finding performance

upper bounds on a DM's global and local critics. The difficulty of these functions

is very domain dependent. In the domain used to develop repair theory, the globalcritic merely has to determine whether a student's answer to a subtraction problem

is correct (Brown and VanLehn, 1980). Sometimes a DM has a person perform

the global credit assignment, for example in LEAP and MACSYMA-ADVISOR. In very

difficult domains a DM might have a person perform both global and local credit

assignment; TEIRESIAS takes this approach when debugging MYCIN (Davis, 1982).

TEIRESIAS can be viewed as an intelligent editor that allows an expert to perform

global and local credit assignment while watching MYCIN solve problems.

In domains where expertise involves heuristic problem solving, having a pro-

gram perform global credit assignment is often very difficult. In a medical appren-

ticeship, a student may recognize that his or her knowledge is deficient when he orshe can no longer make sense of the sequence of questions that the physician asks

the patient. Since a weakly plausible explanation for any sequence of questions of-ten exists, this can be very difficult to implement in a computer program. A similar

situation exists in complex games such as chess or checkers. There is usually no way

to know that a given move is necessarily bad; it depends on what follows. Samuel's

checker player solved the global critic problem by declaring a discrepancy to exist

'Recall from section 2 that the global critic notices that something is wrong and the local criticdetermines which part of the knowledge base is responsible for the error. A learning program consistsof a global and local critic and a repair component.

8

. .

Page 14: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

whenever thc expert (e.g., the book move) and the checker program recommended

different moves at a particular board configuration.

There are often many different changes the local critic can make to effect an

improvement in the performance element. The selection process is usually based

on which modification leads to the best improvement in the performance element.

Selection is very much affected by how 'improvement' in defined. This is further

discussed in Section 3.2.

3.1 Performance evaluation and the synthetic agent

method

* The synthetic agent method proposed in this paper is considerably different from

standard performance evaluation methods in two fundamental ways. The purpose

of the remainder of Section 3 is to explain and justify these aspects of the synthetic

agent method. In Section 3.2 we argue that a fruitful evaluation criteria for a

knowledge-based system should be quality of the individual knowledge elements,not the quality of the problem solving performance of a particular problem-solving

program. These metrics only partially overlap and certainly conflict in the shortterm. In section 3.3, we describe how the focus of the proposed synthetic agent

method is to delineate a performance upper bound. A performance upper bound

describes where and under what conditions a debugging system for a problem solver

must fail. By contrast, a standard evaluation approach aims at showing the extent* to which a debugging system can succeed. Further, instead of characterizing the

limits of debugging in terms of a percentage of problems that cannot be solved, the

synthetic agent method characterizes the performance upper bound in terms of the

knowledge representation language and the inference constructs used in the expertU system.

9

04l

Page 15: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

3.2 Knowledge-oriented vs. performance-oriented valida-

tion

The ultimate goal of a DM is to improve the performance of a PS or ES. The

architecture of knowledge-based systems requires a shift in our concept of improved

performance. Ve refer to the type of validation technique we advocate as knowledge-

oriented validation and distinguish it from the traditional practice of performance-

oriented validation.

Performance-oriented validation requires that modifications to a particular

problem-solving program improve problem-solving performance. Because this type

of validation has traditionally focused on improved performance with respect to asingle problem-solving program, the veracity of the underlying knowledge has not

been of overriding concern. A system designed exclusively to maximize problem-solving performance of a particular problem-solving program may use a method of

knowledge representation in which the semantics of the domain knowledge cannot

be represented easily, if at all. A polynomial evaluation function for rating checker

positions, for example, captures none of the meaning of its terms.

Knowledge-oriented validation might be defined as performance-oriented vali-

dation that prohibits lessening the truth of individual knowledge elements solely forthe sake of problem-solving performance. The advent of large declarative knowledge

bases used by multiple problem solving programs makes this perspective important.

Examples of multiple problem-solving programs that might use the same medicalknowledge base are programs to accomplish medical diagnosis, knowledge acquisi-

tion, intelligent tutoring, and explanation. When multiple programs use the same

declaratively-specified factual knowledge base, it is helpful to specify knowledge

in a manner that is independent, so far as possible, of its use. Knowledge-based

validation accomplishes this by requiring that changes to the knowledge base be

semantically meaningful.

Suppose we wish to be faithful to the traditional performance-oriented vali-. dation paradigm when using multiple-purpose knowledge bases. This requires that%j Ievery time a learning program finds a change to the knowledge base that will improve

10

04

Page 16: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

one problem-solving program, before that change can be recorded, the validation

method must insure that the aggregate performance of all programs is improved.

This policy wvill be expensive and computationally overwhelming. Further, pro-

grams for all the intended uses of a knowledge base are not necessarily in existence

at the time learning is taking place.

Another rationale for knowledge-oriented validation is our belief that perfor-

mance in the long term will be more correct and robust if the knowledge structures

are carefully developed. Moreover, when PS is a person, it is unrealistic, proba-

bly even unwise, to attempt to replace semantically-rich knowledge structures with

others that deviate radically from them merely to improve short-term performance.

It should be noted that to some extent all programs for improving an in-

telligent agent aim at both good performance and good knowledge; nevertheless,

almost all past research in machine learning, intelligent tutoring and automatic

programming has adopted a pure performance-oriented validation approach. This

is especially true in automatic programming, where any mutation to the program

to be debugged is judged to be acceptable if it causes the program to produce the

correct output when given a correct input/output training instance (Shapiro, 1983).

In machine learning, one of the best systems for refining an expert system

knowledge base is the SEEK2 program for the EXPERT expert system shell (Gins-

berg et al., 1985). This learning system takes a performance-oriented validation

approach. One possible input to SEEK2 is a representative set of past solved cases

and an initial knowledge base of rules. Given this input, SEEK2 attempts to modify

elements of the knowledge base so as to maximize the problem-solving performance

of the EXPERT expert system on the given representative set of solved problem cases.

In EXPERT, the strengths of inexact rules in the knowledge base are represented us-

ing certainty factors (CFs). Examples of modification operators used by SEEK2 to

improve performance are LOWER-CF and RAISE-CF (Ginsberg, 1986). When a repre-

*Q sentative set of past cases is present, the strengths of inexact rules are determined,

since certainty factors can be given a strict probabilistic interpretation (Heckerman,

1986). We strongly believe that an arbitrary change to the strength of a rule just

to improve performance is unjustifiable and unnecessary (Wilkins and Buchanan,

* 11

Page 17: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

1986). The cost of this improved performance is a knowledge base that may con-tain incorrect knowledge. The SEEK2 refinement approach is not an instance of

knowledge-oriented learning; it does not use knowledge-oriented validation.

A good example of hnowledge-oriented learning is repair theory in the domainof subtraction problems (Brown and VanLehn, 1980). Repair theory is concerned

with detecting underlying bugs, given the observable problem solving behavior ofstudents. Repair theory has a procedural model of problem solving that claims to bea plausible model of the associated human skill; bugs of students are correlated with

possible bugs in the problem-solving procedure for subtraction. Repair theory issimilar in spirit to the synthetic agent method we propose for assessing a differential

modeling system. Repair theory generates most of the significant possible bugs bydeleting parts of the procedural knowledge; likewise we expect our approach togenerate most of the significant possible types of bugs in the declarative domain

knowledge base, mainly by deleting parts of the knowledge base, as we shall describein Section 4. The main difference is that in the repair theory model of subtraction

the PS and ES knowledge is almost completely procedural, whereas we are interestedin factual knowledge is declaratively represented.

• .3.3 Capability-oriented vs. limitation-oriented validation

A typical way of validating that a DM improves an ES involves using a disjoint set

of validation and training problem sets. The ES solves the validation problem set

and its performance is recorded. Then the DM improves the ES while watching

a human expert PS solve a training problem set. Finally ES solves the validation

problems again; the amount of improvement in performance provides a measure of

the quality of the DM.

'.:

This scenario establishes a lower bound on the quality of a DM. By increasing

the size of the training problem set, DM might improve ES even more. We refer to

4.. validation methods that establish a lower bound on the quality of a DM as capability-

oriented. For a given set of training and validation problems, capability-oriented

validation shows that the DM is responsible for a more capable ES.

" 12

V

Page 18: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Another method of validating a DM is to have the DM watch a student solve a

training problem set. Let us assume that the student exhibits a representative set of

the types of domain knowledge errors that could be made in the problem domain.

A domain expert can manually identify the domain knowledge errors connected

with each training problem. This manual analysis provides a performance upper

bound with respect to this training set for the DM, and the DM modeling program

is measured against this standard. The goals of this type of manual analysis and

our proposed automated analysis using the synthetic agent method are identical, in

the case where the student and the problem set have been both constructed so as

to allow all possible types of domain knowledge errors to be made.

We desire to know those types of differences in an expert system knowledge

base that cannot be detected or corrected via differential modeling. In contrast

to the capability-oriented approach, our validation approach aims at determining

when the differential modeler must fail - we are limitation-oriented. For example,

a limit of a program for inducing LISP functions from examples might be that the

program can't induce cases that require certain types of loop constructs. In our

work, we have focused on showing certain conditions that force the differential

modeling approach to fail under the most favorable of conditions, the single fault

assumption. The multiple fault assumption would allow determination of a broader

performance upper bound.

4 Synthetic Agent Method of Validation

%The apprenticeship learning and tutoring scenarios shown in Figures 2 and 3 involve

two agents: a person and an expert system. The person serves as an expert and stu-

dent, in the context of apprenticeship learning and intelligent tutoring, respectively.

The synthetic agent method consists of replacing the person with a synthetic agent,

which is another expert system, in order to experiment with and validate the dif-

ferential modeling system objectively. The knowledge in the synthetic agent expert

system is modified to be slightly different from the knowledge in the original ex-

pert system. The knowledge is modified to be slightly 'better' in an apprenticeship

13

Page 19: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

learning scenario and slightly 'worse' in an intelligent tutoring scenario.

Problem Statement

PS DM ES

Human Differential ApprenticeExpert Modeler Expert System

Answer Differences Between AnswerKnowledge Structures

of PS and ES

Figure 3: Apprentice learning scenario: Apprentice expert systemwatches human expert through the differential modeling program,with the goal of improving the apprentice program's knowledge.

An advantage of the synthetic agent method is control over interpersonal vari-

ables involved in differential modeling. An example of an interpersonal variable is

the problem-solving style of a PS, as exemplified by the set of strategic diagnostic op-

erators used by the PS. Diagnostic operators specify the permissible task procedures

that can be applied to a problem as well as the allowable methods for achieving the

task procedures. Examples of problem-solving operators in the domain of diagnosis

include: ask general questions, ask clarifying questio:.s, refine hypotheses, differen-

tiate between hypotheses, and test hypothesis. Another interpersonal variable is

the quantity of domain-specific knowledge that the PS possesses.

While control of interpersonal variables almost always leads to an incorrect

DM performance lower bound, conclusions reached concerning a performance upper

bound are sound when interpersonal variables are controlled. If a system is inher-

ently limited under the most optimal assumptions possible for differential modeling,

it will still be inherently limited in those settings that involve a less optimal differ-

ential modeling setting.

14

0.

• '** 14

Page 20: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Problem Statement

PS DM ES

Student Differential Expert SystemModeler

Answer Differences Between AnswerKnowledge Structures

of PS and ES

Figure 4: Intelligent tutoring scenario: Expert system watchesstudent through the differential modeling program, with the goalof improving the student's knowledge.

In the learning and tutoring scenarios, the synthetic agent method treats the

original expert system knowledge base as a "gold standard". The apprentice ES

and the student PS always have a deficiency with respect to this gold standard. In

this paper we restrict our analysis to the situation where the apprentice's knowledge

differs from the gold standard by a single element of knowledge; hence we have a

single fault assumption. Two types of knowledge base discrepancies are possible:

missing knowledge and erroneous knowledge. The synthetic agent method proce-

dure described in section 4.1 shows how deletion of knowledge can represent the

space of missing and erroneous knowledge. Other methods for creating erroneous

knowledge are described in section 4.3.

For a given problem statement, a distinction is made between referenced, ob-

servable, and essential knowledge in the ES's knowledge base. The relation between

these categories is illustrated in Figure 4. Referenced knowledge is simply knowledge

that is accessed during a problem solving case. Observable knowledge is knowledge

whose removal leads to different external observable behavior of a PS, either in the

sequence of actions that the PS exhibits or the final answer. Essential knowledge is

15

SV

Page 21: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Domain Knowledge Base

Referenced Knowledge

Observable Knowledge

Essential Knowledge

Figure 5: The relation between different categories of knowledge,with respect to a particular problem case.

knowledge whose removal leads to a significantly different final answer.

Of most concern is the apprentice's ability to acquire the essential knowledgeelements connected with a problem statement. These are the relations most im-

portant for solving a given case. For plausible reasoning systems, what comprises a

significantly different answer needs to be specified. For instance, if there are multi-

ple diagnoses, the significance of the order in which the hypotheses are ranked needsto be determined. Acquisition of elements that are observable but not essential are

also of interest, since they can be essential elements with respect to another problem

statement.

The procedure for calculating a performance upper bound on a differential

modeling system is now presented.

16

04

W

Page 22: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

4.1 The synthetic agent method

Step 1. Create synthetic agent. Replace PS with a synthetic agent: a copy of ES

with initially the same domain knowledge.

Step 2. Solve problem case. Solve a problem using PS and save the solution trace,

i.e., the observable actions of PS and the final answer.

Step 3. Identify observable knowledge. For a particular problem case, collect all

elements in the knowledge base that were referenced by PS during problem

solving. Identify the observable knowledge: the subset of the referenced knowl-

edge whose removal would lead to a different solution trace or a different final

answer.

Step 4. For each observable kno, ledge element:

Step 4a. Remove the element from ES. In an apprenticeship learning sce-

nario this creates an apprentice expert ES with missing knowledge. In

an intelligent tutoring scenario the element removed from the ES is de-

clared to be erroneous3 . Since the element is still present in PS, the

synthetic student PS has erroneous knowledge.

Step 4b. Detect and localize knowledge discrepancy. Have the PS solve the

problem case. See if DM can detect (the global critic problem) and

localize (the local critic problem) the knowledge difference.

Step 5. For each observable knowledge element:

Step 5a. Remove the element from PS. In an intelligent tutoring scenario

this creates a synthetic student PS with missing knowledge. In an appren-

ticeship learning scenario the element removed from the PS is declared

to be erroneous3 . Since the element is still present in ES, the apprentice

expert ES has erroneous knowledge.

Step 5b. Detect and localize knowledge discrepancy. Have the PS solve the

problem case. See if DM can detect (the global critic problem) and

localize (the local critic problem) the knowledge difference.

aN.B. This element of knowledge is treated as erroneous for purposes of validation. In reality,the element is true knowledge.

17

L St

Ai4

Page 23: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

4.2 Discussion of synthetic agent miethod

An expert systei m's explanation facility can be li(,l)ful in locating the observable

knowledge with respect to a given problem case. One of the hallmarks of a good

expert system is its ability to explain its own reasoning. So it is not too much to

ask for those pieces of knowledge used on a problem case, and a good explanation

systeni might even be able to identify the essential knowiedge. At worst, given

the pieces of knowledge that were used to solve a particular problem, the essential

pieces of knowledge can be determined by experimentation. Usually, only a smal.amiouint of an expert system's domain knowledge is observable with respect to a

given problemm, and our experiences in the medical diagnosis domain have shown usthat only a small atlo)uilt )f the observable knowledge is essential knowledge.

Some knowledge that is referenced by the expert system may not have ob-

s(ervable co(nse(luences, even if it is used by the problem solver, since the removal

S"of knowl(lge does not always effect the external behavior of a problem solver. For

instan'e. in MYCIN and NEOMYCIN, terms that represent medical symptoms and

measurements, such as patient weight, have an ASKFIRST property. The expert

system uses the value of this property to decide whether the value '-f a variable is

first determined by asking the user or first determined by derivation by some othermethod, such as from first principles. However, if the system does not possess tech-

niques for deriving the information from other principles, then the external behavior

of the system is the saine regardless of the value of the ASKFIRST property.

\When testing the global critic in steps 4b and 5b of the synthetic agent

imethodl, part of the assessment must relate to whether the apprentice detects

• knowledge base differences close to the point in the problem-solving session where

the different knowledge was used. This temporal proximity is important, since the

problemn-solving context at this point in the problem-solving session strongly focuses

tie search for missing or erroneous knowledge.

0.1UP

Page 24: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

4.3 Categories of errors

The knowledge organization that we focus upon specifies all factual domain knowl-

edge in a declarative fashion. In such a knowledge base. there are two main cat-

egories of errors: missing and erroneous knowledge. Missing knowledge is absent

from the knowledge base, and erroneous knowledge is factually incorrect knowledge

that is present in the knowledge base.

The space of missing knowledge is easy to generate, especially with the single

fault assumption. Recall that the original expert system serves as our gold stan-

dard and the domain knowledge in the expert system is declaratively represented.

Hence, the number of single faults from missing knowledge is equal to the number

of elements in the declarative knowledge base.

The space of erroneous knowledge is much more difficult to describe. The

synthetic agent method takes a novel approach to the problem in steps 5a and

6a. An erroneous element is created by declaring a correct knowledge element to

be erroneous for purposes of validation. We are also considering other approaches.

Much of the knowledge is represented declaratively and typed. Therefore, erroneous

knowledge can be generated by substituting different values for the knowledge in

the range of the type, as long as the assumption can be made that the erroneous

knowledge is at least correctly typed by the problem solver. The space of possible

variations of declarative associational rule knowledge is significantly reduced by the

practice used in the HERACLES' expert system shell of factoring different types of

knowledge from the domain knowledge, such as causal, definitional, and control

knowledge (Clancey, 1986b).

5 Application of Synthetic Expert Method

Our investigations of a performance upper bound for a differential modeler are be-

ing performed in the context of the HIERACLES and ODYSSEUS systems. IIERACLES

is an expert system shell that solves classification-type prol)e(:ms using the heuris-

19

04,

Page 25: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

tic classification method (Clancey, 1985). The ODYSSEUS program differentiallymodels a PS against any ES implemented using the ItERACLES expert system shell(Wilkins, 1986). When PS is a human expert, ODYSSEUS functions as a knowledgeacquisition program for the TIERACLES expert system shell. When PS is a student,ODYSSEUS functions as a student modeling program for the GUIDON2 intelligenttutoring system, which is built over HERACLES.

Problem Statement

PS DM ES

"Expert" ApprenticeExpert System LifR i Expert System

Generator

Global CriticAnswer Expert System Answer

Local CriticExpert System

Differences BetweenKnowledge Structures

a' of PS and ESS

Figure 6: Synthetic agent validation situation for apprenticeship... learning in which the role of the PS has been filled by a synthetic

expert system. In apprenticeship learning, the DM watches PS toimprove ES's knowledge structures.

V: In HERACLES, domain knowledge is encoded using a relational language andMYCIN-type rules (Clancey, 1986b). The knowledge relations of the relational lan-

guage are predicate calculus representations of the domain knowledge, written using

20

Page 26: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

the logic programming language MRS. For example, an instantiation of the propo-

sition (SUGGESTS $PARM $HYP) represents the fact that if a particular parameter

is true then this suggests that a particular hypothesis is true. An instantiation of

the template (ASKFIRST $FINDING $FLAG-VALUE) specifies whether the system

should first ask the user for the value of a finding, or derive the information from

existing information. The major domain knowledge base for HERACLES at this time

is the NEOMYCIN knowledge base for diagnosing meningitis and neurological prob-

lems (Clancey, 1984). A second effort in the sand casting domain is called CASTER

(Thompson and Clancey, 1986).

Three aspects of the HERACLES expert system shell facilitate the task of

differential modeling faced by ODYSSEUS. First, distinctions are made between the

different types of knowledge in IIERACLES' knowledge base, such as heuristic, def-

initional, causal, and control knowledge. Second, the method of reasoning, called

hypothesis-directed reasoning, approximates that used by human experts (Clancey,

1984). Hence, HERACLES can be viewed as a simulation of an expert's process of

diagnosis. Third, the control knowledge is explicitly represented as a procedural

network of subroutines and metarules that are both free of domain knowledge; the

subroutines and metarules use variables rather than specific domain terms (Clancey,

1986b). By contrast, the heuristic rules in MYCIN have a great deal of control knowl-

edge imbedded in the premises of the rules (Clancey, 1983; Buchanan and Shortliffe,

1984).

Figure 5 shows the place of the ODYSSEUS DM in the context of debugging an

apprentice expert system. The DM tracks the problem-solving actions of the PS step

by step. For each observable step of the problem solver, ODYSSEUS generates and

scores the alternative lines of reasoning that can explain the reasoning step. If the

*global critic does not find any plausible reasoning path, or all found paths have a low

plausibility, ODYSSEUS assumes that there is a difference in knowledge between the

human problem solver and the expert system. The local critic attempts to locate the0.1I knowledge difference either automatically or by asking the expert specific questions.

* ODYSSEUS' analysis of problem-solving steps uses two underlying domain theories: a

strategy theory of the problem-solving method called hypothesis-directed reasoning

using the heuristic classification method, and an inductive predictive theory for

'V 21

%0.

Page 27: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

I,'II

heuristic rules that uses a library of previously solved problen cases.

The ODYSSEUS global and local critics are themselves being implemented

as two IlERACLES-based expert systems. There are three reasons wvhy we choose to

' implement the critics as expert systems. First, the task that confronts the learning

critics is a knowledge-intensive task (Dietterich and Buchanan, 19S1), and expert

syst em techniques are useful for representing large amounts of knowledge. Second.

* with an expert system architecture, the reasoning method used by the critics can'S'

be made explicit and easily evaluated, since the domain knowledge is declaratively

enico(led using IER ACLES' knowledge relations and simple heuristic rules. Third,

since ODYSSEUS is designed to improve any IIERACLES-based expert system, it can

th,,trticmallv improve itself in an apprenticeship learning setting.

Approximately sixty different knowledge relations in IIERACLES specify the

declarative domeain knowledge. It would be useful to know how successful the global

-1and local critics are at detecting discrepancies in the different knowledge relations

of the knowledge representation language. Are there certain types of knowledge re-lations whose absence is always noticeable? Are there particular types of knowledge

whose absence is very hard to recognize? For example, ItERACLES represents final

diagnoses in a hierarchical tree structure; determining that a problem is caused by

a missing link in this structure may be very difficult for the apprentice to discover.By contrast, it may be very easy to discover whether a trigger property of a rule is

missig. A trigger property causes the conclusion of a rule to treated as an active

hypothesis if particular clauses of the rule premise are satisfied. Clearly global and

local crcdit assignment are greatly affected by the complexity of the procedural

control !%iowledge ,,sed in the expert system shell.

6 Summary

With the proliferation of expert systems. methods of intelligent tutoring and ap-

prenticeship learning that are based on differential modeling of the normal problem-

solving behavior of a student or expert against, a knowledge-intensive expert system

9')

Sk

Page 28: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

should become increasingly common. The synthetic agent method is proposed as

an objective means of assessing the limits of a particular differential modeling pro-

gram in the context of intelligent tutoring and apprenticeship learning. The powerof a differential modeler is crucialhy dependent upon the expert system's method

of knowledge representation and control. The synthetic agent method provides a4

means of expressing the limitations of a differential modeler in terms of the knowi-

edge representation and control vocabulary.

The synthetic agent method involves a systematic perturbation of a progranm

that takes the place of the student or expert. Traditionally, methods of evaluating

a differential modeler have focused on a performance lower bound. The described

synthetic agent method focuses on establishing a performance upper bound. It

provides a means of exploring the extent that a differential modeling system is able

to detect and isolate an arbitrary difference between a knowledge base of an expert

0 system and the problem-solving knowledge of a tudent or expert. Our work to

(late confirms our belief that the task of differential modeling is easier the more anexpert system represents factual domain knowledge in a declarative fashion.

The validation framework described in this paper is being used to assess

the limits of the ODYSSEUS modeling program in the context of intelligent tutoring

and apprenticeship learning. Students and experts are being differentially modeled

against knowledge bases for the tIERACLES' expert system shell. This should lead

to a better understanding of the synthetic agent method, the ODYSSEUS modeling

program, and the extent to which t!ERACLES' method of knowledge representation

and control facilitates differential modeling.

7 Acknowledgments

\ e are grateful for very substantive critiques of draft versions of this paper byMarianne Winslett, Haym Hirsh and Devika Subramanian.

This work was supported in part by NSF grant MCS-83-1214S, ONR/ARIcontract N00014-79C-0302, Advanced Research Project Agency (Contract D.-\PPA

N00039-S3-C-0136). the National Institute of Health (Grant NIH RR-007S. I1)

04

r23

------------------------------------- 2 ..

Page 29: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

National Aeronautics and Space Administration (Grant NAG-5-261), and Boeing

(Grant W266S75). We are grateful for the computer time provided by SUMEX-AIM

and the Intelligent Systems Lab of Xerox PARC.

8 References

Bierinann, A. W. (1978). The inference of regular LISP programs from examples. IEEE

Transactions on Systeins. Alan, and Cybernetics, SMC-8(8):585-600.

Brown, J. S., Burton, R., and DeKleer, J. (1982). Pedagogical and knowledge engineering

techniques in SOPHIE I, II and III. In Sleeman, D. II. and Brown, J. S., editors,

Intelligent Tutoring Systems, London: Academic Press.

Brown, J. S. and VanLehn, K. (1980). Repair theory: a generative theory of bugs in

*procedural skills. Cognitive Science, 4:479-426.

., Buchanan. B. G. and Shortliffe, E. It. (1984). Rule-Based Expert Systems: The MYCINExpcrimenits of the Stanford Heuristic Programming Project. Reading, Mass.: Addison-

Wesley.

Burton, I. and Brown, J. S. (1982). An investigation of computer coaching for informal

learning activities. In Sleeman, D. H. and Brown, J. S., editors, Intelligent Tutoring

Systems, London: Academic Press.

Clancey, W. J. (1979). Transfer of Rule-Based Expertise Through a Tutorial Dialogue. PhD

thesis, Stanford University. Stanford Technical Report STAN-CS-79-769.

A Clancey, W. J. (1983). The epistemology of a rule-based system: a framework for explana-

tion. Artificial Intelligence, 20:215-251.

S -Clancey, NV. J. (1984). NEOMYCIN: reconfiguring a rule-based system with application to

teaching. In Clancey, W. J. and Shortliffe, E. II., editors, Readings in Medical Artificial

Intelligence, chapter 15, pages 361-381, Reading, Mass.: Addison-Wesley.

Clancey, W. J. (1985). Heuristic classification. Artificial Intelligence, 27:289-350.

Clancey, W. J. (1986a). From GUIDON to NEOMYCIN to HERACLES in twenty short

lessons. A I Magazine, 7:40-60.

24

"n F

O,

0 , a, '', , , ,o,, didd,~" --'ili ddd IHidIl ikl . . I

Page 30: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Clancey, NV. J. (1986b). Representing control knowledge as abstract tasks and metarules.

In Coombs, M. and Bolc, L., editors, Computer Expert Systems, Springer Verlag. Also,

Knowledge Systems Lab Report KSL-85-16, Stanford University, April 1985.

Davis, R. (1982). Application of meta level knowledge in the construction, maintenance anduse of large knowledge bases. In Davis, R. and Lenat, D. B., editors, Knowledge-Based

Systems in Artificial Intelligence, New York: McGraw-Hill.

Dietterich, T. G. (1984). Constraint Propagation Techniques for Theory-Driven Data In-terpretations. PhD thesis, Stanford University. Stanford Technical Report STAN-CS-

79-xxx.

Dietterich, T. G. and Buchanan, B. G. (1981). The Role of the Critic in Learning Systems.

Report HPP-81-19, Stanford University. Reprinted in Selfridge, 0., Rissland, E. andArbib, M. eds: Adaptive control of Ill-defined systems.

Fu, L. and Buchanan, B. G. (1985). Inductive knowledge acquisition for rule based expertsystems. Technical Report KSL 85-42, Stanford University, Computer Science Dept.

Genesereth, M. R. (1982). Diagnosis using hierarchical design models. In Proceedings ofthe Second National Conference on Artificial Intelligence, pages 278-283, American

Association for Artificial Intelligence.

Ginsberg, A. (1986). A metalinguistic approach to the construction of knowledge base

refinement systems. In Proceedings of the 1986 National Conference on Artificial In-

telligence, pages 436-441.

Ginsberg. A., Weis-, S., and Politakis, P. (1985). SEEK2: a generalized approach to auto-

matic knowledge base refinement. In Proceedings of the 1985 IJCAI, pages 367-374.

Ileckerman, D. (1986). Probabilistic interpretations for Mycin's certainty factors. In Kanal,

L. and Lemmar, J., editors, Uncertainty in Artificial Intelligence, North Holland.

* Miller, R. A., Pople, H. E., and Myers, J. D. (1984). INTERNIST-i: an experimental

computer-based diagnostic consultant for general internal medicine. In Clancey, W. J.

and Shortliffe, E. I., editors, Readings in Medical Artificial Intelligence, chapter 8,

pages 190-209, Reading, Mass.: Addison-Wesley.

Mitchell, T. M., Mahadevan, S., and Steinberg, L. 1. (1985). LEAP: a learning apprentice

for VLSI design. In Proceedings of the 1985 IJCA I, pages 573-580.

Samuel, A. L. (1963). Some studies in machine learning using the game of checkers.

In Feigenbaum, E. and Feldman, D., editors, New York: Computers and Thought,

McGraw- Hill.

25

01

Page 31: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Shapiro, f. 11. (1983). Algorithmic Program Understanding. C(ambridge: MIT Press.

Thompson, T. and Clancey, W. J. (1986). A qualitative modeling shell for process diagnosis.

IEEE Software, 3(2):6-15.

Waterman, D. (1970). Generalization learning techniques for automating thle learning of

lheuristics. Artificial Intelligence, 1:121-170.

Wilkins, D. C. (1986). Knowledge base debugging using apprenticeship learning techniques.

In Knowledge Acquisition for Knowledge-Bascd Systems lWorkshop, pages 40.0-4I0.14.

Wilkins, D. C. and Buchanan, B. G. (1986). On debugging rule sets when reasoning under

uncertainty. In Proceedings of the 1986 National Conference on Artificial Intelligence,

pages 448-454.

Yu, V. L., Fagan, L. M., Wraith, S. M., and Clancey, IV. J. (1979). Evaluating the perfor-, mance of a computer-based consultant. J. Amer. Med. Assoc., 242(12):1279-1282.

i "

0,1

26

W,~~ ~~ e C--

Page 32: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Personnel Analysis Division, D.r..urron H. Bower

AF/MPXA Dr. Eva L. Baker DEpartment )f Psychology5C360, The Pentagon UCLA Center for the Study Stanford 1niversityWashington, DC 20330 of Evaluation Stanford, CA )4306

145 Moroe HallAir Force Human Resources Lab University of California Pr. Richard BrabyAFHRL/MPD Los Angeles, CA 90024 NTSC Code LOBrooks AFB, TX 78235 Orlando, FL 32-51

Dr. James D. BakerAFOSR, Director of Automation Dr. Robert Breaux

Life Sciences Directorate Allen Corporation of America iode N-95RBolling Air Force Base 401 Wythe Street Naval Training Systems -enteWashington, DC 20332 Alexandria, VA 22314 Orlando, FL 32813

Dr. Robert Ahlers Dr. Meryl S. Baker Dr. John S. BrownCode N7ll Navy Personnel R&D Center XEROX Palo Alto ResearchHuman Factors Laboratory San Diego, CA 92152-6800 CenterNaval Training Systems Center 3333 Coyote RoadOrlando, FL 32813 Dr. Donald E. Bamber Palo Alto, CA 94304

Code 71Dr. Ed Aiken Navy Personnel R & D Center Dr. Bruce BuchananNavy Personnel R&D Center San Diego, CA 92152-6800 Computer Science DepartmentSan Diego, CA 92152-6800 Mr. J. Barber Stanford UniversityDr. William E. Alley HQS, Department of the Army Stanford, CA 94305

AFHRL/MOT DAPE-ZBRBrooks AFB, TX 78235 Washington, DC 20310 Dr. Howard BurrowsNational Institute of Health

Capt. J. Jean Belanger Bldg. 10, Room 5C-106

Dr. Earl A. Alluisi Training Development Division Bethesda, MD 20892HQ, AFHRL (AFSC) Canadian Forces Training System Dr. Patricia A. ButlerBrooks AFB, TX 78235 CFTSHQ, CFB Trenton

Astra, Ontario, KOK OERIAsta, ntaioKOK555 New Jersey Ave. NW- Dr. John R. Anderson CANADA 55-e esyAe, N

Department of Psychology Washington, DC 20208Carnegie-Mellon University CDR Robert J. Biersner, USNPittsburgh, PA 15213 Naval Biodynamics Laboratory Dr. Tom CaffertyP . O . Box 9407Dept . o f Psycho logyDr. Nancy S. Anderson P. 0. Box 29407 University of South CarolinaDr ac .Adro New Orleans, LA 70189 ClmiS 90Department of Psychology Columbia, SC 29208University of Maryland Dr. Menucha Birenbaum Dr. Robert CalfeeCollege Park, MD 20742 School of Education School of Education

Tel Aviv University Stanford U'niversityDr. Steve Andriole Tel Aviv, Ramat Aviv 69978 Stanford CA 94305George Mason University ISRAELSchool of Information

.. Technology & Engineering Dr. Werner P. Birke Joanne Capper4400 University Drive Personalstammamt der Bundeswehr Center for Research into PractiFairfax, VA 22030 Kolner Strasse 262 1718 Connecticut Ave., N.W.

D-5000 Koeln 90 Washington, DC 20009Dr. John Annett FEDERAL REPUBLIC OF GERMANYUniversity of Warwick Dr. Jaime CarbonellDepartment of Psychology Dr. Gautam Biswas Carniegie-Mellon UniversityCoventry CV4 7AJ Department of Computer Science Department oi PsychologyENGLAND University of South Carolina Pittsburgh, FA 15213~Columbia, SC 29208Dr. Phipps Arable ColMr. James W. CareyUniversity of Illinois Dr. John Black Commandant (G-PTE)Department of Psychology College, Columbia Univ. U.S. Coast Guard603 E. Daniel St. 525 West 121st Street 2100 Second Street, S.W.

-. 4 Champaign, IL 61820 New York, NY 10027 Washington, DC 20593Technical Director, ARI Dr. Arthur S. Blaiwes Dr. Susan Carey

5001 Eisenhower Avenue Code N711 Harvard Graduate School ofAlexandria, VA 22333 Naval Training Systems Center EducationOrlando, FL 32813 337 Gutman LibrarySpecial Assistant for Projec Appian Way

OASN(M&RA) Dr. Robert Blanchard Cambridge, MA 021385D800, The Pentagon Navy Personnel R&D CenterWashington, DC 20350 San Diego, CA 92152-6800 Dr. Pat Carpenter

Carnegie-Mellon UniversityDr. Michael Atwood Dr. Jeff Bonar Department of PsychologyITT - Programming Learning R&D Center Pittsburgh, PA 152131000 Oronoque Lane University of PittsburghStratford, CT 06497 Pittsburgh, PA 15260 Prof. John W. Carr III

Department of Computer ScienceDr. Patricia Baggett Dr. J. C. Boudreaux Moore School ofUniversity of Colorado Center for Manufacturing Electrical EngineeringDepartment of Psychology Engineering University of PennsylvaniaBox 345 National Bureau of Standards Philadelphia, PA 19104Boulder, CO 80309 Gaithersburg, MD 20899

Page 33: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Dr. Robert Carroll Dr. Stanley Collyer Goery DelacoteOP 01B7 Office of Naval Technology Directeur de L'informatiqueWashington, DC 20370 Code 222 Scientifique et Technique

800 N. Quincy Street CNRSLCDR Robert Carter Arlington, VA 22217-5000 15, Quai Anatole FranceOffice of the Chief -75700 Paris FRANCE

of Naval Operations Dr. Lynn A. CooperOP-OIB Learning R&D Center Mr. Robert DentonPentagon University of Pittsburgh AFMPC/MPCYPRWashington, DC 20350-2000 3939 O'Hara Street Randolph AFB, TX 78150D r hPittsburgh, PA 15213

.. Dr. Fred Chang1'.,, Navy Personnel R&D Center Dr. Meredith P. Crawford Mr. Paul DiRenzo

Code 51 American Psychological Association Commandant of the Marine Corps,,,. San Diego, CA 92152-6800 Office of Educational Affairs Code LBC-4

De1200 17th Street, N.W. Washington, DC 20380Dr. Alphonse Chapanis Washington, DC 200361415 Bellona Lane

Suit 210Dr. R. K. DismukesSuite 210 Dr. Hans Crombag Associate Director for Life SciencesBuxton Towers University of Leyden AFOSRBaltimore, Yj 21204 Education Research Center Boiling AFB

Boerhaavelaan 2 Washington, DC 20332Dr. Davida Charney 2334 EN LeydenDepartment of Psychology The NETHERLANDSCarnegie-Mellon University Defense TechnicalSchenley Park LT Judy Crookshanks Cameron Station, Bldg 5Pittsburgh. PA 15213 Chief of Naval Operations Alexandria, VA 22314

OP-112G5 Attn: TCDr. Eugene Charniak Washington, DC 20370-2000 (12 Copies)[ Brown University(1Cois

Computer Science Department Dr. Kenneth B. Cross Dr. Thomas M. DuffyProvidence, RI 02912 Anacapa Sciences, Inc. Communications Design Center

* P.O. Drawer Q Carnegie-Mellon UniversityDr. Paul R. Chatelier Santa Bdrbara, CA 93102 Schenley ParkOUSDRE Pittsburgh, PA 15213

." PentagonWashington DC 20350-2000 Dr. Mary Cross Barbara Eason

Department of Education military Educator's

" Dr. Michelene Chi Adult Literacy Initiative Resource NetworkLearning R & D Center Poom 4145 InterAmerica Research AssociatesUniversity of Pittsburgh 400 Maryland Avenue, SW 1555 Wilson Blvd3939 O'Hara Street Washington, DC 20202 Arlington, VA 22209

.. Pittsburgh. PA 15213Edward E. Eddowes

%Mr. Raymond E. Christal CTB/McGraw-Hill Library CNATRA N301N: AFHRL/MOE 2500 Garden Road Naval Air Station

'.,. Brooks AFB, rX 18235 Monterey, CA 93940 Corpus Christi, TX 78419

Dr. Yee-Yeen Chu Dr. John EllisPerceptronics, Inc. CAPT P. Michael Curran Navy Personnel R&D Center21111 Erwin Street Office of Naval Research San Diego, CA 92252loodiand Hills, CA 91367-371 800 N. Quincy St.

Code 125 Dr. Richard Elster)r. William Clancey Arlington, VA 22217-5000 Deputy Assistant Secretarytanford Uni-,ersity of the Navy (Manpower)'nowledge Systems Laboratory Dr. Cary Czichon OASN (M&RA)'01 Welch Road, Bldg. C Intelligent Instructional Sy Department of the Navylalo Altc, CA 94304 Texas Instruments Al Lab Washington, DC 20350-1000

P.O. Box 660245. :hief of Naval Education Dallas, TX 75266

Y.- and Training Dr. Susan EmbretsonLiaison Office Bryan Dallman University of KansasAir Force Human Resource Lab AFHRL,'LRT Psychology DepartmentOperations Training Division Lowry AFB, CO 80230 426 Fraser

0,1 Williams AFB, AZ 85224 Lawrence, KS 66045~LT John Dea ton

Assistant Chief of Staff ONR Code 125for Research, Development, 800 N. Quincy Street Dr. Randy EngleTest, and Evaluation Arlington, VA 22217-5000 Department of Psychology

Naval Education and University of South CarolinaTraining Command (N-5) Dr. Natalie Dehn Colombia, SC 29208

NAS Pensacola, FL 32508 Department of Computer andInformation Science Lt. Col Rich Entlich

j Dr. Allan M. Collins University of Oregon HQ, Department of the ArmyBolt Beranek & Newman, Inc. Eugene, OR 97403 OCSA(DACS-DPM)

' 50 Moulton Street Washington, DC 20310%A Cambridge, MA 02138 Dr. Gerald F. DeJong

Artificial Intelligence Grou Dr. Susan EpsteinOr. john J. Collins Coordinated Science Labor3to Hunter CollegeDirector, Field Research University of Illinois 144 S. Mountain Avenue

0 Office, Orlando Urbana, IL 61801 Montclair, NJ 07042NPRDC Liaison OfficerNTSC Orlando, FL 32813

Page 34: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Dr. William Epstein Dr. Carl H. Frederiksen:niversity of Wisconsin McGill University Dr. Joseph GoguenW. J. r-qden Psychology Bld 37O0 McTavish Street Computer Science Laboratory1202 4. Johnson Street Montreal, Quebec H3A IY2 SRI InternationalMadison, Wi 53706 CANADA 333 Ravenswood Avenue

Menlo Park, CA 94C25ERIC Facility-Acquisitions Dr. John R. Frederiksen4813 Rugby Avenue Bolt Beranek & Newman Dr. Sherrie GottBethesda, MD 20014 50 Moulton Street AFHRLMODJ

Cambridge, MA 02138 Brooks AFB, TX 78235Dr. K. Anders EricssonUniversity of Colorado Dr. Norman Frederiksen Jordan Grafman, Ph.D.Department of Psychology Educational Testing Service 2021 Lyttonsville Road

4 Boulder, CO 80309 Princeton, NJ 08541 Silver Spring, MD 20910

Dr. Edward Esty Dr. Alfred R. Fregly Dr. Richard H. GrangerDepartment of Education, OEP AFOSR'NL Department of Computer ScienceRoom -'D Boiling AFB, DC 20332 University of California, Irvine1200 19th St., NW Irvine, CA 92717Washington. D- 20208 Dr. Bob Frey

Dr. Beatrice J. Farr Commandant (G-P-I/2) Dr. Wayne GrayUSCG HQ Army Research Institute

Aimy Research Institute Washington, DC 205933201 Eisenhower Avenue 5001 Eisenhower Avenue

Alexandria, VA 22333 Dr. Alinda Friedman Alexandria, VA 22333

Farr Department of Psychology Dr. James G. GreenoDr. Marshall J. FarUniversity of Alberta Uiest fClfri'52-0 'orth Vernon Street EdUnesto Alberta University of California

2520Morh Vrno SteetEdmonton, Alberta Berkeley, CA 94720Arlington, VA 2220' CANADA T6G 2E9

Dr. Pat Federico Julie A. Gadsden H. William GreenupDe P11 Information Technology Education Advisor (E031)Code 511 Applications Division Education Center, MCDECSan Diego, CA 92152-6800 Admiralty Research Establish Quantico, VA 22134

Portsdown, Portsmouth P06 4A Dipl. Pad. Michael W. HabonDr. Jerome A. Feldman UNITED KINGDOM Universitat DusseldorfUniversity of Rochester ErziehungswissenschaftlichesComputer Science Department Dr. R. Edward Geiselman Univensitatsstr. 1

Rochester, NY 14627 Department of Psychology D-4000 Dusseldorf 1University of California WEST GERMANY

Dr. Paul Feltovich Los Angeles, CA 90024

Southern Illinois University Prof. Edward HaertelSchool of Medicine Dr. Michael Genesereth School of EducationMedical Education Department Stanford University Stanford UniversityP.O. Box 3926 Computer Science Department Stanford, CA 94305Springfield, IL 62708 Stanford, CA 94305

Mr. Wallace Feurzeig Dr. Dedre Gentner Dr. Henry M. Haiff

Educational Technology University of Illinois 4918 33rd Road, North

Bolt Beranek & Newman Department of Psychology Arlington, VA 2220710 Moulton St. 603 E. Daniel St.

Cambridge, MA 02238 Champaign, IL 61820Dr ragI.FelsDr. Ronald K. HambletonDr. Craig I. Fields Dr. Robert Glaser Prof. of Education & Psychol

ARPALearning Research University of Massachusetts1400 Wilson Blvd. & Development Center at AmherstArlington, VA 22209 University of Pittsburgh Hills House

3939 O'Hara Street Amherst, MA 01003

Dr. Gerhard Fischer Pittsburgh, PA 15260University of Colorado Dr. Cheryl HamelDepartment of Computer Science Dr. Arthur M. ''-nberg NTSCBoulder, CO 80309 University oj Wt.;consin Orlando, FL 32813

W. J. Brogdei Psychology Bldg.J. D. Fl-t-her 1202 W. Johnson Street Dr. Bruce W. Hamill) 931 r - :a Street Madison, WI 53706 Johns Hopkins University

f- Vienna 22180 Applied Ph-sics LaboratoryDr. Marvin D. Glock Johns Hopkins Road

Dr. Linda Flower 13 Stone Hall Laurel, MD 20'07" Carnegie-Mellon University Cornell University

Department of English Ithaca, NY 14851 Dr. Ray HannapelPittsburgh, PA 15213 Scientific and Engineering

Dr. Gene L. Gloye Personnel and EducationDr. Kenneth D. Forbus Office of Naval Research National Science FoundationUniversity of Illinois Detachment Washington, DC 20550

* Department of Computer Science 1030 E. Green Street1304 West Springfield Avenue Pasadena, CA 91106-2485

Urbana, IL 61801 Stevan Harnad

Dr. Sam Glucksberg Editor, The Behavioral andDr. Barbara A. Fox Department ot Psychology Brain Sciences

University of Colorado Princeton Unjiversity 20 Nassau Street, Suite 240Department of Linguistics Princeton, NJ 08540 Princeton, NJ 08540

* Boulder, CO 80309

LA I

Page 35: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Dr. James Howard Dr. Richard JohnsonMr. William Hartung Dept. of Psychology Boise State UniversityPEAM Product Manager Human Performance Laboratory Simplot/MicronArmy Research Institute Catholic University of Technology Center5001 Eisenhower Avenue America Boise, ID 83725Alexandria, VA 22333 Washington, DC 20064

CDR Tom JonesDr. Wayne Harvey Dr. Steven Hunka ONR Code 125SRI International Department of Education 800 N. Quincy Street333 Ravenswood Ave. University of Alberta Arlington, VA 22217-5000Room B-S324 Edmonton, AlbertaMenlo Park, CA 94025 CANADA Alberta, C NADADr. Douglas A. Jones

Dr. Reid Hastie P.O. Box 6640Northwestern University Dr. Earl Hunt LawrencevilleDepartment of Psychology Department of Psychology NJ 08648Evanston, IL 60201 University of Washington

Seattle, WA 98105 Col. Dominique Jouslin de NoProf. John R. Hayes Etat-Major de l'Armee de TerCarnegie-Mellon University Dr. Ed Hutchins Centre de Relations HumainesDepartment of Psychology Intelligent Systems Group 3 Avenue Octave GreardSchenley Park Institute for 75007 ParisPittsburgh, PA 15213 Cognitive Science (C-015) FRANCE

UCSDDr. Barbara Hayes-Roth La Jolla, CA 92093 Dr. Marcel JustDepartment of Computer Science Carnegie-Mellon UniversityStanford University Dr. Dillon Inouye Department of PsychologyStanford, CA 95305 WICAT Education Institute Schenley ParkXProvo, UT 84057

Dr. Frederick Hayes-Roth Pittsburgh, PA 15213Teknowledge Dr. Alice Isen525 University Ave. Dcf wtment of Psychology Dr. Milton S. KatzPalo Alto, CA 94301 Uzni :rsity of Maryland Army Research Institute

Catunsville, MD 21228 5001 Eisenhower AvenueDr. Joan I. Heller Alexandria, VA 22333Dr. Jaoa n R.oHade Dr. Zachary JacobsonDr Sc t Ke s

r-.. Oakland, CA 94606 Bureau of Management Consulting Dr. Scott Kelso365 Laurier Avenue West Haskins Laboratories,

' Dr. James Hendler Ottawa, Ontario KIA 0S5 270 Crown Street/.NewrHavensCTe06510* Dept. of Computer Science CANADA New Haven, CT 06510

University of MarylandCollege Park, MD 20742 Dr. Robert Jannarone Dr. Dennis Kibler

Department of Psychology University of CaliforniaUniversity of South Carolina Department of Information

5,' Dr. Jim Hollan Columbia, SC 29208 and Computer Science: Intelligent Systems Group Irvine, CA 92717Institute for Dr. Claude Janvier

. cognitive Science (C-015) Directeur, CIRADE Dr. David KierasUCSD Universite' du Quebec a Montreal University of MichiganLa Jolla, CA 92093 Montreal, Quebec H3C 3P8 Technical Communication

CANADA College of Engineering,% Dr. Melissa Holland 1223 E. Engineering Building., Army Research Institute for COL Dennis W. Jarvi Ann Arbor, MI 48109

Behavioral and Social Sci Commander% 5001 Eisenhower Avenue AFHRL Dr. Peter Kincaid

Alexandria, VA 22333 Brooks AFB, TX 78235-5601 Training Analysis& Evaluation Group

* Dr. Robert W. Holt Dr. Robin Jeffries Department of the NavyDepartment of Psychology Hewlett-Packard Laboratories Orlando, FL 32813

. eorge Mason Unive-sity P.O. Box 104904400 University Drive Palo Alto, CA 94303-0971 Dr. Walter KintschFairfax, VA 22030 Department of Psychology

" D e HDr. Robert Jernigan University of Colorado[* Dr. Keith Holyoak Decision Resource Systems Campus Box 345

• *niversity of Michigan 5595 Vantage Point Road Boulder, CO 80302*O F,.nan Performance Center Columbia, '.D 21044

330 Packard Road Dr. Paula KirkAnn Arbor, MI 48109 Oakridge Associated Universities

Margaret Jerome University Programs DivisionFrof. Lutz F. Hornke c/o Dr. Peter Chandler P.O. Box 117Insti 'ut fur Psychologie 83, The Drive Oakridge, TN 37831-0117pWTH Aachen HoveJaeqerStrasse 17/19 Sussex Dr. David KlahrD-5100 Aachen UNITED KINGDOM Carnegie-Mellon UniversityWEST ;ERMANY Department of Psychology

Dr. Joseph E. Johnson Schenley ParkMr. Dick Hoshaw Assistant Dean for()P-115 Graduate Studies Pittsburgh, PA 15213

Arlington Annex College of Science and MatheRoom 2834 University of South Carolinawa3hinqton. DC 20350 Columbia, SC 29208

Page 36: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Dr. Mazie Knerr Dr. Jim LevinProqram Manager sUniversity of California Dr. Pichard E. MayerTraining Research Division Laboratory for Comparative Department ot Psychology

% HumRRO Human Cognition University of California1100 S. Washington DOO3A Santa Barbara, CA f3106

V Alexandria, VA 22314 La Jolla, CA 92093

Dr. Tar-Let L. Kolodner Dr. John Levine;(orpia Institute of Technology Learning R&D Center Dr. James McBrideSchool ot Information University of Pittsburgh Psychological Corporation

& 'omputer Science Pittsburgh, PA 15260 c/o Harcourt. Brace,Atlanta, .A 30332 Javaznovich Inc.

1250 West 6th StreetDr. Stephen Kosslyn Dr. Michael Levine San Diego. -A 12 1!IHarvard University Educational Psychology1236 William James Hall 210 Education Bldg. Dr. David J. McGu.niiess33 Kirkland St. University of Illinois Gallaudet College-.bridge, MA 02138 Champaign, IL 61801 800 Florida Avenje, N.E

Washington, DC* Dr. Kenneth Kotovsky Dr. Charles Lewis

Department ot Psychology Faculteit Sociale Wetenschappen Dr. Kathleen McKeuwnCommunity College of Rijksuniversiteit Groningen Columbia U'nLversity

Alleqheny County Oude Boteringestraat 23 Department ot r'Dmpjjt,.r ze.i'k1Q

) Alleq, .eny Avenue 9112GC Groningen New York, NY i,1L)2-Pi ttLurih, PA [5233 lhe NETHERLANDS

Dr. Joe McLachlan.r. David H. Krantz Dr. Clayton Lewis Navy Personnel R&D Cnter

a washington Square Village University of Colorado San Diego, CA 92152-6890A 1DJ Department of Computer Science

w York, NY 10012 Campus Box 430 Dr. James McMichaelBoulder, CO 80309 Assistant for MPT Pesearch,

Dr- Benjamin Kuipers Development,r'nlversity of Texas at Austi Library OP me1t7 andDepartment of Computer Scien Naval War College o n 023-r-S. Painter Hall 3.28 Newport, RI 02940 Washington, DC 203r0AIstln, T-xas 8l

e, Library Dr. Barbara Meansi'- LibaryHuman Resources7Dr L avit R. Lambert Naval Training Systems Cente Research Organization

:aval Ocean Systems Center Orlando, FL 32813 1100 South WashingtonCole 441T

ton

:- Catalina Boulevard Dr. Charlotte Linde Alexandria, VA 22314

,an Diego, CA 92152-6800 SRI International Dr. Douglas L. Medin333 Ravenswood Avenue Department of Psychology

:r Pat Langley Menlo Park, CA 94025--. , "University of Illinois'niversity of California 603 E. Daniel StreetDepartment of Information Dr. Marcia C. Linn Champaign, IL b1820y' and Computer Science Lawrence Hall of Science

-U,. Irvlne, 'CA 9217 University of California Dr. Arthur Melm,,d

M. .Liane Langston Berkeley, CA 94720 U.S. Department of Education)rniainCetl724 Brown.jinunications Design Center Dr. Robert Linn Washington, DC 20208'.irnegie-Mellon University College of Education

S -henley Park University of Illinois Dr. Jose MestrePit-sb r h, PA 15213 Urbana, IL 61801 Department of Physics

D- %iDHasbrouck LaboratoryA; Jilr ieLaKin Dr. Don Lyon University of Massachusetts0niversity P.O. Box 44 Amherst, MA 01003

-.partmpnt o Psychology Higley, AZ 85236-P-ujbu,h PA 15213

-. Pbert Lawler Dr. Jane Malin Dr. Al Meyrowitz

- .n o rton Scines FRL Mail Code SR 111 Office of Naval Researchi m ' NASA Johnson Space Center Code 1133;TE Laboratories, Inc. Houston, TX 77058 800 N. Quincy;-j4 Sylvan Road Arlinqton, VA 222175000waltham, YA 02254 Dr. William L. Maloy

Dr. Paul E. Lehner Chief of Naval Education Dr. Ryszard S. MichalskiPr TeaoloE Conrp. and Trann EdcUniversity of IllinoisPAR Technology Corp. and Training Department of Computer Sciev926 Jones Branch Drive Naval Air Station 1304 West Springfield AvenuF

Suite 170 Pensacola, FL 32508 Ur0a IL pi e0nS--------_, McLean, V/A 22102 Urbana, IL 61801

Dr. Sandra P. Marshall Prof. D. MichieDr. Alan M. Lesgold Dept. of Psychology The Turing InstituteLearning R&D Center San Diego State University 36 North Hanover StreetUniversity of Pittsburgh San Diego, CA 92182 Glasgow GI 2AD, ScotlandPittsburgh, PA 15260 UNITED GDM

% Dr. Manton M. Matthews UNITED KINGDOM% Dr. Alan Leshner Department of Computer Scien

Deputy Division Director University of South Carolina Dr. George A. Mller

Behavioral and Neural Sciences Col .mbia, SC 29208 Department of Psycholoy! National Science Foundation Green Hall1800 G Street Princeton University

L o Washington, Dr 20550 Princeton, NJ 28540

p.* p-

Page 37: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Leadershi' Mandgement Educat office of N".' i es idh

Dr. James R. Mjiler and Tralning Project Offi (ode 110

MCC Naval Medical Command 800 N. Quincy Street

9430 Research Blvd. Code 05C Arlington, VA 2221--000Echelon Building 01, Suite 2 Washington, DC 1-0372

Austin, TX 78759 Office of Naval Research,

r MTeciniral Director, Code 1142

Dr. mark Miller Navy Health Research Ctr. 800 N. Quincy St

C jmputer*!hought Corporation P.O. Box 85122 Arlington, VA 22217-50C0

121 West Plano Parkway San Piego, CA 92138

Piano, x -50o5 office of Naval Research,

Dr T. Niblett Code l142EP

" Dr. Andrew R. Molnar The Turing Institute 800 N. Quincy Street

. Scientific and Engineering 36 North Hanover Street Arlington, VA 2221- 5'000

Personnel and Education Glasgow GI 2AD, Scotland office of Naval Research,

National Science Foundation UNITED KINGDOM Code 1142PT

Washington, DC 20550 800 N, Quincy Street

Dr. William Montague Dr. Richard E. Nisbett Arlington, VA 2221- JO00

NPRDC Code 13 University Of Michigan (6 copies)

San Dieqo, CA 92152-6800 Institute for Social Resear(

Room 5261 Director, rechnolcgy Prorams,

Dr. Tom Moran Ann Arbor, MI 48109 Office of Naval Research

Xerox PARC Code 123333 Coyote Hill Road Dr. Mary .10 Nissen 800 North Quincy Street

Palo Alto, CA 94304 UniversitY of Minnesota Arlington, VA 2221--5000

N218 Elliott Hall

jr. Mellryn Yoy Minneapolis, M.N 55455

Navry Personnel R & D Center Special Assistant for Marinc

San Diego, CA 92152-6800 Dr. Donald A. Norman Corps Matters,Institute for Cognitive SciE OR C-ode OOMCUniversity of California

800 N. Quincy St.

Dr. Allen Munro La Jolla, CA 92093 Arlington, VA 2221-5000

Behavioral Technology

Laboratories - USC Uitector, Training Laborator Assistant for Long Range

1845 S. Elena Ave., 4th Floor 'PRDC (Code 05)

Redondo Beach, CA 90277 Sn Diego, CA 92152-6800 RequirementS,'r CNO Executive Panel

, Directorn Director, Manpower and Perso Op 00Kuupport Laboratory, 2000 North Beauregard Street

Systems Division, NMPC NPRDC (Code 06) Alexandria, VA 22311

aval MiJ.itary Personnel Command San Diego, CA 92152-6800 Assistant for MPT Research," Assistamnt an Stdiesrh

Wahunit:n. DC 1 Director, Human Factors Development and Studies

& Organizational Systems Washington, DC 20370" Olr~~~ctsr, NPRDC- (Code 07) WsigoD 07

Distribution Department, NMPC San Diego, CA 92152-6800 Dr. Judith Orasanu

Washinoton, DC 20370 Fleet Support office, Army Research InstituteNiPRDC (Code 301) 5001 Eisenhower Avenue

Director, San Diego, CA )-2152-6800 Alexandria, VA 22333

Overseas Duty SupportProgam. MPCDr. Jesse Orlansky

Program, '4APC Institute for Defense Analys- h- t C 3Library NPPDC 1801 N, Beauregard St.

Coshington, DC 203-0 Code p2ClL Alexandria. VA .2311

Head. HR.M Operations Branch, San Dielo, "A 42152-6800H *RIr Prof. Seymour Papert

-,.,Nal R seajch Laboratory 20C-109D a LMassachusetts Institute

WashingtoDn, DC 203-0 coDde 262*7washing'Iton, D(- 2of Technology

Assistant for Evaluation, . Cambridge, MA 02139

Analysis, and MIS, NNPC r. Harold F. O'Neil, Jr. T. Parlette

N-CDCSchool of Education - ;qPH 01 CDR R.TWashington, F 00Dlrtment f Educational Chief of Naval OperationsWa hi gt n DC 0 7 ) - rt ent f E u a i alOP- 12G

P;.-y--hol y & Technology Wsing

Spoc. Asst. for Research, Experi- t niversitY )t Southern California washington, DC 20370-2000

mental & Academic Programs, Los Angeles, CA 90Dr. James Paulson

3TTC (Code 016) Department of Psychology

NAS Memphis (75) Dr. Stellan ohlsson Portland State University

Millington, TN 38054 Learning R & DP.O. Box 751

D University of Pittsburgh Portland, OR 97207Director. 1 j9 O'Hara Street P r ln ,O 70

Research & Analysis Div., pittsburgh, PA 15213 Dr. Douglas Pearse

;AVCRU'ITCOM Code 22 D Do a r

4015 Wilson Blvd. Director, Research Programs, oCI20Arlington. "/A 22203 Office of Naval Pesearch Box 2000

800 Nolth Quincy Street Downsvie OntarioT~chnlcal Director ArlingtOn, VA 22217-5000 CANADANavy Health Research CenterP,O. Box 85122San Di-qo, CA 32138

Page 38: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

flr. 'irginia E. Perilergrass Dr. Mary C. Potter Dr. Donald Pubis,Co de -11 Department of Psychology Statistics LeVpaitmont

Naval Training Systtems C7enter MIT (E-10-032) Science Center, Room D028

Orlando, FL 32813--.00 Cambridge, MA 021139 1 Oxford StreetHarvard University

Dr. Joseph Psotka Cambridge, M1A 02138Dr. Robert Penn ATTN: PERL-IC7SPR PC Army Research Institute Dr. DaviJ Rulmelhbart

San Diego, CA 2l 6805001 Eisenhower Ave. centerfo HuaAlexandria, VA 22333 Information Pr 'essinq

Dr. Nancy Pennilnlt~Dn ftniv. ot C:alifornia

1university Of ChlcaloO Dr. Roy Rada La Jolla, CA 92093

;raduate School of Business National Library of Medic-int

1101 E. 58th St. Bethesda, MD 20894 Ms. Riitta Ruotsalainen

Ihco lL t0 t3' General Headquartersad Dr. Lynne Reder Training Section

MilItAry Aosistant for Training an eatetO scooyMilitary Psychology OfficeP(-sne TcnlgCarnegie-Mellon ICniversity PL 919

AJSD (R & E) Schenley Park SF-00101 Helsinki i0. FINLAI

Poom 3D129, The Pentagon Pittsburgh, PA 15213r~.gfltcn, EC 20301-3080 Dr. Michael J. Samet

: r.P37P,,-z ria ReserPerceptrolics, Inc

r.P -11 :Bia eie 6271 Variel Avenue..s F-RI Dfepartment of P~vcholigy Woodland Hills, C-A 91364

-c's~ Avenue Princeton University

x Ila,3 VA 33 Princeton, NJ 08544 Dr. Roger Schank

iv -I ;.P -r~nsYale University

: rirl '- hn >IyQ -enter Dr. James A. Reggia Computer Science Department

i t rA' r r3 a ry L' niversity of Maryland P.O. Box 2158

A~ ,t'n I' ri School of Medicine New Haven, CT 06520

,!A _,L59 Department of Neurology22 South Greene Street Dr. W. L. Scherlis

SNBltmre D 10 Dept. Of Computer ScienceCarnegie-Mellon ('niversity

'.ARA ~ 3b Ylq ICDR Karen peider yPr

* . x ~~~4l9 Naval School of Health Sclenc-,-s Ptsug.P 51

National Naval Medical CenterMr. igtt Shniebc

',Aip s irilae Scho DWarshninBl g on 141081 Forsvarets Center for Lederg

~ ~Ot~i iuaP ~hoo Wasingon, C 2814Christianshavns Voldgade 8

Dr Fre Aeif 1424 Kobenhavn K

.i *in'-'! cit -cimputer Science, Physics DepartmentDEMR

i' p'>r riuate SchOOl University of California D.Wle cnie

* '* A ~BerkleyCA 9'20Learning R&D Center

peain Research, Dr arnRsikUniversity of Pittsburgh

r _ tiraduate School Learning R & D Center 33 'aaSre

' A4DUniversity of Pittsburgh Pittsburgh, PA 152603939 O'Hara Street

P!, p PitsbrghPA 1213Dr. Alan H. Schoenfeld

P.tsurr PA151 University of California

;1 y -it Technol Dr. Mary S. Riley Department of Education

-I c~ in Program in Cognitive Science Berkeley, CA 94720

Center for Human InformationEProcessingDr Jae Scoil

U-niversity of California Le.anng R&DofCete

La; 7olCA'29 University of Pittsburgh

William Rizzo Pittsburgh, PA 15260

*I CoeI Dr. Marc SebrechtsF-i' ilcqyNaval Training Systems Center Department of Psychology

u rlando, FL 32813 Wesleyan University

r~i.IoMiddletown, CT 06475Dr. Lindla (S. Roberts

*Science. Education. anG Dr. Judith SegalTransportation Program GERI

* ct i Office of Technology Assessment 555 New Jersey Ave., NW

* cii~~ . jqy c-ongress of the United States Washington, DC 20208j ,fWashiniton, DC 20510

P') IDr. Robert J. SeidelPolro'-kDr. Andrew M. Rose US Army Research Institute

41-,IAmerican Institutes 5001 Eisenhower Ave.Ifor Research Alexandria, VA 22333

- i~1; 10 g *11055 Thomas Jefferson St., NW~ ~ 3 '3Washington, DC 20007 Dr. Ramsay W. Selden

4,1 , P r leAsscsment Center

r 5rr: P ~Dr. William B. Rouse CCSSOr~li0ril'Y -)f P1-Lburqh Search Technology, Inc. Suite 379

-cii',n stems Laboratory 25-b Technology Park/Atlanta 400 N. Capitol, NW

Saf- Hall Norcross, GA 300942 Washington, DC 20001

P ;.buir;h. PA

Page 39: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

DDr. Linda B. SmithDr. W. Steve Selman Department ot Psychology Mr. Brad SympsonOASD(MRA&L) Indiana University a Personnel R&D Center2B269 The Pentagon San Diego, CA 92152-6800Washington, DC 20301¢ "Dr. Alfred F. Smode Dr. John Tangney

Dr. Sylvia A. S. Shafto Senlor Scientist AFOSR/NLOERI Code 0nA Boiling AFB, DC 20332

555 New Jersey Ave., NW Naval Training Systems CenterWash-ngton, DC 20208 Orlando, FL 32813 Dr. Kikumi Tatsuoka

CERL

Dr. T. b Sheridan Dr. Richard E. Snow 252 Engineering ResearchDept. of Mechanical Engineerini Department ot Psychology LaboratoryMIT Stanford University Urbana, IL 61801Cambridge, MA 02139 Stanford, CA 94306 Dr. Martin M. Taylor

Dr. Ben Shneiderman Dr. Elliot Soloway DCIEMDept. of Computer Science Yale University Downsview, Ontario

* University of Maryland Computer Science Department DACollege Park, MD 20-42 P.o. Box 2158ANADA

Dr. Ted Shortliffe New Haven, CT 06520 Dr. Perry W. ThorndykeComputer Science Department Dr. Richard Scrensen FMC CorporationStanford University Navy Personnel R&D Center Central Engineering Labs

Stantord, CA )4305 San Diego, CA 92152-6800 1185 Coleman Avenue, Box 580Santa Clara, CA 95052

7'r Lee r.'. an Dr. Kathryn T. Spoehr Major Jack ThorpeStanford Yniversrty Brown University

_42 iathcart WaY Department of PsychologyDARPA- Deartent f Pychoogy1400 Wilson Blvd.Stanford, CA 94305 Providence, RI 02912 Arlion v2r % Arlington, VA 22209

,r. RinJall Sh-maker James J. Staszewski Dr. Douglas Towne'.'- ;avaL Research Laboratory Research Associate Behavioral Technology Labsode Carnegie-Mellon University B845vS.aElen Labs

4S vekAeuS.W. Department of Psychology 1845 S. Elena Ave.4555 :ver>. k Avenue, .WDeatntoPscogyRedondo Beach, CA 90277

Wshngton DC 20375-5000 Schenlei Park

Dr. Robert . Siegler Pittsburgh, PA 15213 Dr. Amos Tversky

Carnegie-Mellon University Dr. Marian Stearns Stanford University

epartment of Psychology SrI Intranat nl Dept. of Psychology

S,:henley Park 333 Ravenswood Ave.' s~run. PA 15213 Room B-$324 Dr. James Tweeddale

Herbert A. Simon Menlo Park, CA 94025 Technical Director

:e ar, ment of Psychology Dr. Robert Sternberg Navy Personnel R&D Center-arnegie-Mellon University Department of PsychologySan Diego, CA 92152-6800

3chenley Park Yale UniversityPittsburgh, PA 15213 alUnvriyDr. Paul TwohigBox 11A, Yale Station Army Research Institute=CL Robert Simpson New Haven, CT 06520 5001 Eisenhower Avenue

• Defense Advanced Research Dr. Albert Stevens Alexandria, VA 22333

Projects Administration Bolt Beranek & Newman, Inc. Headquarters, U. S. Marine Corps1400 wilson Blvd. 10 Moulton St. Code MPI-20Arlington, IA 22209 Cambridge, MA 02238 Washington, DC 20380

S Dr L+a M SimutLz Dr. Paul J. Sticha[T.:cctunal Thhnology Senior Staff Scienti.st Dr. Kurt Van Lehn

Department of PsychologyP I ,y.3t--mj Area Training Research Division Carnegie-Mellon University;,.. A -[ HumRRO) £i.;,nhower Avenue 1100 S. Washington Schenley Park

AI.-xanlrt.i,, iA 22333 Alexandria, VA 22314 Pittsburgh, PA 15213

[r H Wallace Sinalko Dr. Thomas Sticht Dr. Beth Warren' a-'[ower Oes~th Navy Personnel R&D Center Bolt Beranek & Newman, Inc.

tav,] AIviscry Services San Diego, CA 92152-6800 50 Moulton Street.mithsotian Institution Cambridge, MA 02138

%, P0l North Pitt Street Dr. David StoneAlexandria, VA 22314 KAJ Software, Inc. Dr. David J. Weiss

3420 East Shea Blvd. N660 Elliott HallDr. Derek Sleeman Suite 161 University of MinnesotaStantord] University Phoenix, AZ 85028 5 E. River RoadSchool ot Education Minneapolis, MN 55455Stanford, CA 94305 Cdr Michael Suman, PD 303

Naval Training Systems Cente Roger Weissinger-BaylonDr. Charles F. Smith Code N51, Comptroller Department of AdministrativeNorth Carolina State Univers Orlando, FL 32813 SciencesDepartment of Statistics Naval Postgraduate SchoolRaleigh, NC 27695-8703 Dr. Hariharan Swaminathan Monterey, CA 93940

Laboratory of Psychometric a Dr. Donald WeitzmanDr. Edward E. Smith Evaluation Research MrE

0 Bolt Beranek & Newman, Inc. School of Education MITRE50 Moulton Street University of Massachusetts 1820 Dolley Maison Blvd.Cambridge, MA 02138 Amherst, MA 01003 Macen VA 202

Page 40: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Dr. Keith T. Wescourt Dr. Joseph L. YoungFMC Corporation Memory & CognitiveCentral Engineering Labs Processes1185 Coleman Ave., Box 580 National Science FoundationSanta Clara, CA 95052 Washington, DC 20550

Dr. Douglas Wetzel Dr. Steven ZornetzerCode 12 Office of Naval ResearchNavy Personnel R&D Center Code 1140San Diego, CA 92152-6800 800 N. Quincy St.

Arlington, VA 22217-5000Dr. Barbara WhiteBolt Beranek & Newman, Inc. Dr. Michael J. Zyda1 Moulton Street Naval Postgraduate SchoolCambridge, MA 02238 Code 52CK

Monterey, CA 93943-5100LCDR Cory deGroot WhiteheadChief of Naval OperationsOP-112GIWashington, DC 20370-2000

Dr. Michael WilliamsIntelliCorp1975 El Camino Real WestMountain View, CA 94040-2216

Dr. Hilda WingArmy Research Institute5001 Eisenhower Ave.Alexandria, VA 22333

A. E. Winterbauer* Research Associate

Electronics Divi3ionDenver Research Institute

, University Park- Denver, CO 80208-0454

Dr. Robert A. WisherU.S. Army Institute for the

Behavioral and Social Sciences5001 Eisenhower AvenueAlexandria, VA 22333

Dr. Martin F. WiskoffNavy Personnel R & D CenterSan Diego, CA 92152-6800

Dr. Merlin C. WittrockGraduate School of EducationUCLALos Angeles, CA 90024

Mr. John H. WolfeNavy Personnel R&D CenterSan Diego, CA 92152-6800

Dr. Wallace wulfeck, IIINavy Personnel R&D CenterSan Diego, CA 92152-6800

Dr. Joe YasatukeAFHRL/LRTLowry AFB, CO 80230

Dr. Masoud YazdaniDept. of Computer ScienceUniversity of ExeterExeter EX4 4QLDevon, ENGLAND

Major Frank Yohannan, USMCHeadquarters, Marine Corps(Code MPI-20)Washington, DC 20380

Mr. Carl YorkSystem Development Foundation181 Lytton AvenueSuite 210

0. Palo Alto, CA 94301

L 111&9

Page 41: IN INTELLIGENT TUTORING AND AP DEPT I OF STAN-CS-87-i175 ... · 7 -ri8 53 using and evaluating differential modeling in intelligent tutoring and ap (u) stanford univ ca dept unclas;sified

Sr r'rr ran an an JflI~nn ~7Wfl nWWWnrWfl -. -. -. fl

tJ.

w~.

)

J

U'pm

5- 'V.5-

5-

5,.

5.

*

5-

55

*55

*