Developing A Hidden Domain for Human and Electronic Students Presented at IJCAI ALIHT Workshop July 2011

Jeremy Ludwig, Alex Davis Stottler Henke Associates

San Mateo, CA

{ludwig, davis} @stottlerhenke.com

Mark Abrams, Jon Curtis Cycorp

Austin, TX

{mabrams, jonc} @cyc.com

DISTRIBUTION STATEMENT A: Approved for Public Release, Distribution Unlimited

Bootstrap Learning Objective: Develop an electronic student (e-student) to learn from focused human-style instruction. Example: Drawing the letter “p”

Bootstrap Learning Framework

!Figure 1: The BLADE Framework

2.2 Human Learning and Teaching

"#$%!#&'!()*+,!-.),&%/0+/&1!+,!%2'&)3%,&!+'')*%'!#&'!#,'&/)-&./!0.''%''%'!+44!/%4%5+,&!-+0+6#4#&#%'1!+,3!#&'!7.+4!#'!&.!4%+/,!)'#,7!&(%!'+*%!#,'&/)-&#.,!*%&(.3'!)'%3!6%&8%%,!()*+,'9!:'!0+/&!.;!&(%!<5+4)+&#.,!=%+*1!8%!+/%!,.&!+44.8%3!+--%''!&.!%2'&)3%,&!#*04%*%,&+&#.,!3%&+#4'!&.!%,')/%!.)/!6%,-(*+/$'!+/%!),6#+'%39!>%!$,.8!&(+&!&(%! 4%+/,%/'! +/%! 3%'#7,%3! &.! 6%! 3.*+#,2#,3%0%,3%,&1! +,3! &(+&! &(%?! +/%! '0%-#+4#@%3! &.! 0+/&#-)4+/! A+&)/+4!B,'&/)-&#.,!C%&(.3'!DABC'E!/+&(%/!&(+,!0+/&#-)4+/!0/.64%*!3.*+#,'9!=(%! +/%+! .;! -.*0)&%/! &)&./#,7! -+,! 6%! '%%,! +'! +,! #,5%/'%! 0/.64%*! &.! 8(+&! 8%! +/%! #,5%'&#7+&#,79! B,!

0+/&#-)4+/1! &(%! +/%+! .;! &%+-(+64%! +7%,&'! 6%+/'! '.*%! ')/;+-%! '#*#4+/#&?! &.!F"9! B,! &(#'! ;#%431! ()*+,! '&)3%,&'!&%+-(! 4%+/,#,7! +7%,&'! #,!./3%/! &.! #*0/.5%! &(%#/! .8,!),3%/'&+,3#,7!.;! -.,-%0&'! DG"%+/,#,7!6?!=%+-(#,7HE9!I,%!%J+*04%!#'!&(%!GF%&&?K'!F/+#,H!'?'&%*!LM+5#'!%&!+491!NOOPQ9!R.8%5%/1!#,!&(%'%!'?'&%*'!&(%!#*0./&+,-%!#'!04+-%3!.,!(.8!8%44!&(%!()*+,!#,'&/)-&./!4%+/,'1!,.&!.,!&(%!-+0+6#4#&#%'!.;!&(%!4%+/,#,7!+7%,&9!

2.3 Human Case Studies

I)/!()*+,!'&)3#%'!)'%!8%44!$,.8,!&%-(,#S)%'!;/.*!6%(+5#./+4!/%'%+/-(1!+'!-.5%/%3!#,!'&+,3+/3!&%J&'!')-(!+'!LT.'%,&(+4! +,3! T.',.81! UVVUQ! +,3! LW#,1! NOOXQ9! B,! 0+/&#-)4+/1! '#,-%! 4#&&4%!8+'! $,.8,! +6.)&!8(+&! ;+-&./'!8.)43!6%!-/#&#-+41!.)/!%*0#/#-+4!+00/.+-(!#'!&(+&!.;!%J04./+&./?!-+'%!'&)3#%'9!:,!%J04./+&./?!-+'%!'&)3?!#'!&(%!6%'&!+00/.+-(!8(%,! 4#&&4%! #'!),3%/'&..3!+6.)&! &(%!')6Y%-&!),3%/!'&)3?!LW#,1!NOOXQ9!=(%! #,&%,&! #'! &.!6)#43!+!3%%0%/! ),3%/'&+,3#,7! .;! &(%! 0(%,.*%,+! #,! S)%'&#.,! +,3! &.! ;./*)4+&%! &(%! 6%7#,,#,7'! .;! +! -.//%'0.,3#,7!&(%./?!&(+&!-+,!6%!&%'&%31!/%5#'%31!+,3!%J0+,3%3!8#&(!;)/&(%/!%*0#/#-+4!'&)3#%'9!

3. PHASE I STUDY: BLOCKSWORLD

=(%!Z)//#-)4)*!=%+*!0%/;./*%3! +,! #,#&#+4! -+'%! '&)3?! &.! %J04./%!8(+&! +00/.+-(%'! +! ()*+,! &%+-(%/! DR=E!8.)43!&+$%!#,!&%+-(#,7!+!0+/&#-)4+/!-)//#-)4)*!&.!+,!%2'&)3%,&1!8(+&!+'')*0&#.,'!+!&%+-(%/!8.)43!*+$%!+6.)&!+!'&)3%,&1!+,3!(.8!&(%'%!+'')*0&#.,'!8.)43!8./$!#,!&(%!-.,&%J&!.;!+,!%2'&)3%,&!LF%/4+,3!+,3![%//?1!NOOVQ9!=(%!R='!8%/%!+'$%3!&.!-.,'#3%/!&%+-(#,7!&.!&(%!4%5%4!.;!+!6/#7(&!&8.!?%+/!.439!=(%!3.*+#,!;./!&(#'!'&)3?!8+'!&(%!F4.-$'8./43!'#*)4+&%3!%,5#/.,*%,&!-/%+&%3!6?!Z?-./01!B,-9!=(#'!%,5#/.,*%,&!-.,'#'&%3!.;!+!G-4+8H1!./!-/+,%24#$%!3%5#-%!8(#-(!&(%!%2'&)3%,&!-.)43!-.,&/.4!&.!*+,#0)4+&%!G'S)+/%!64.-$'H!+,3!G4.,7!64.-$'H9!B,!&(#'!-+'%!'&)3?1!%+-(!.;!;#5%!()*+,!&%+-(%/'!;#/'&!+&&%*0&%3!&.!&%+-(!+,!%2'&)3%,&!&.!-.,'&/)-&!+!'&+-$!.;!

&(/%%!64.-$'!+,3!&(%,!&.!6)#43!+!'#*04%!G3../8+?H!.)&!.;!64.-$'!#,!&(#'!%,5#/.,*%,&9!=(%!&+/7%&!8+'!+!'#*04%!'&/)-&)/%!.;!&8.!'&+-$'!.;!64.-$'!&.00%3!8#&(!+!4#,&%49!>%!)'%3!+!>#@+/3!.;!I@!D>I@E!LM+(46+-$!%&!+491!UVVPQ!'&?4%!*%&(.3.4.7?1!8(%/%!&(%!&%+-(%/'K!,+&)/+424+,7)+7%!#,'&/)-&#.,'!8%/%!&/+,'4+&%3!#,&.!0/%-#'%!&%/*'!DB"E!;./!&(%!%2'&)3%,&!6?!+!()*+,!#,&%/0/%&%/9!!

From: Robert D. Grant, David DeAngelis, Dan Luu, Dewayne E. Perry and Kathy Ryall. TOWARDS EVALUATING HUMAN-INSTRUCTABLE SOFTWARE AGENTS, IHCI 2011, Rome, July 2011

Role of Curriculum 1.  Instruct the e-student 2.  Test the e-student Advantages of a defined set of curricula: •  Algorithm development •  E-student testing •  Developing a repository

HD Requirements 1.  Open to learning from experience 2.  Need to learn from experience to succeed 3.  Explicit reasoning 4.  Elements easily visible to and encoded by an

automated system 5.  Problems come up reasonably often in real

life

Role of Human Comparison Curriculum is necessary and sufficient Provide baseline for expected e-student

performance

Human Comparison Requirements 6.  Include problems that will challenge the

human students. 7.  But not too challenging 8.  Same problems will be presented to human

students and the e-student.

Ground Station Domain

Student fills the role of ground station operator, controlling satellite communication

Demonstration

Laddered Curriculum Each rung in the ladder: •  Is single a concept (e.g. procedure, pattern) •  Contains multiple lessons by telling,

example, or feedback

Rungs build on top of one another to form a ladder Units are used to group related rungs

Phase II Laddered Curriculum

Abnormal Component

Fault 1 Fault 2 Fault 3 Fault 4 Fault 5 Fault 6

Unit 1 Abnormal

Unit 2 Identify

Fault 1 Fault 2 Fault 3 Fault 4 Fault 5 Fault 6 Unit 3 Repair

Fault 1 Fault 2 Fault 3 Fault 4 Fault 5 Fault 6 Unit 4 Fault -> Repair

Diagnose and Repair

Unit 5 Procedure

Example Human Lesson This lesson teaches what abnormal component means.

12

I will provide examples.

13

14 Baseband_Processor_1 is an abnormal component.

15 Notice that some of these indicators are red.

16

Notice that the Baseband_Processor_1 indicators are expected to be black.

17 Notice that Baseband_Processor_1 is on the active path.

18

We’re done with this example.

Example e-student Trace Lesson: HD-Abnormal-CBE-LearningLesson [ Utter( name=Utter-3574, timestamp=Timestamp(system=lesson,offset=379), utterance=LessonTeaches(name=LessonTeaches-3573,arg1=WhenTrue(name=WhenTrue-3572,arg1=Abnormal)), addressee=Student(name=theStudent)) ] [ Utter( name=Utter-3591, timestamp=Timestamp(system=lesson,offset=574),utterance=Method(name=Method-3590,arg1=byExample), addressee=Student(name=theStudent)) ] [ Imperative( name=Imperative-3635, timestamp=Timestamp(system=lesson,offset=808), request=startSimulation(name=startSimulation-3634,modifiers=[0],faults=["Antenna_Motion_Error"]), addressee=World(name=theWorld)) ] [ WorldDone( name=WorldDone-3644, timestamp=Timestamp(system=lesson,offset=1952),imperative=Imperative-3635,source=World(name=theWorld), addressee=Teacher(name=theTeacher)) ] [ Perception( name=Perception-3646, timestamp=Timestamp(system=lesson,offset=2025), gainedPercepts=[ ComponentConnection(

… for about 70 more pages

Phase III Laddered Curriculum

Abnormal Component

Fault 1 Fault 2 Fault 3 Fault 4 Fault 5 Fault 6

Unit 1 Abnormal

Unit 2 Identify

Intrack Azimuth Fault 3 Fault 4 Fault 5 Fault 6 Unit 3 Repair

Fault 1 Fault 2 Fault 3 Fault 4 Fault 5 Fault 6 Unit 4 Fault -> Repair

Diagnose and Repair

Unit 5 Procedure

Task Change Unit 6 Triggers Indicator Change

Perform Diagnose and Repair

Unit 7 Monitor

Update Perform Diagnose and Repair

Monitor Bias

Performance Results

Results are not intended for direct comparison: •  Does not include all units •  Does not include injection penalties

Future Work Significant future work exists in the challenges

contained in this curriculum that were tested on humans but not addressed by the e-student:

•  Complicated repair procedures •  Analogous repair procedures •  Context-based monitoring •  Concept revision

Acknowledgements We want to thank our colleagues at BAE Systems, Bester Tracking Systems, Cycorp, Stottler Henke Associates, and University of Texas at Austin for their significant input throughout the curriculum development process: Manfred Bester, David DeAngelis, Robert Grant, Dan Luu, John Mohammed, Dewayne Perry, Benjamin Rode, Howard Reubenstein, Kathy Ryall, and Candy Sidner, This material is based upon work supported by the United States Air Force Research Laboratory (AFRL) under Contract No. FA8650-07-C-7722. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

Questions?