Learning by demonstration technology has long held thepromise to empower nonprogrammers to customize andextend software. Recent technical advances have enabled

its use for automating increasingly complex tasks (Allen et al.2007; Blythe et al. 2008; Burstein et al. 2008; Leshed et al. 2008;Cypher et al. 2010). However, fielded applications of the tech-nology have been limited to macro recording capabilities,which can only reproduce the exact behavior demonstrated bythe user.

This article describes the successful deployment of learningby demonstration technology that goes well beyond macrorecording by enabling end users to create parameterized proce-dures that automate general classes of repetitive or time-con-suming tasks. This task-learning technology originated as aresearch system within DARPA’s Personalized Assistant ThatLearns (PAL) program, where it was focused on automating taskswithin a desktop environment (Gervasio, Lee, and Eker 2008;Eker, Lee, and Gervasio 2009; Gervasio and Murdock 2009).

From those research roots, PAL task learning has evolved intoa fielded capability within the Command Post of the Future(CPOF) — a military command and control (C2) system usedextensively by the U.S. Army. The CPOF software is part of theArmy’s Battle Command System, and as such is standard equip-ment for virtually every Army unit. Since its inception in 2004,thousands of CPOF systems have been deployed.

Articles

SUMMER 2012 15Copyright © 2012, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Learning by Demonstration for a Collaborative

Planning Environment

Karen Myers, Jake Kolojejchick, Carl Angiolillo, Tim Cummings, Tom Garvey, Matt Gaston, Melinda Gervasio, Will Haines,

Chris Jones, Kellie Keifer, Janette Knittel, David Morley, William Ommert, Scott Potter

n Learning by demonstration technology haslong held the promise to empower nonprogram-mers to customize and extend software. Wedescribe the deployment of a learning bydemonstration capability to support user cre-ation of automated procedures in a collabora-tive planning environment that is used widelyby the U.S. Army. This technology, which hasbeen in operational use since the summer of2010, has helped to reduce user work loads byautomating repetitive and time-consumingtasks. The technology has also provided theunexpected benefit of enabling standardizationof products and processes.

CPOF is a geospatial visualization environmentthat enables multiple users to collaborate in devel-oping situational awareness and planning militaryoperations. Much of CPOF’s power comes from itsgenerality, providing tremendous flexibility forhandling a wide range of missions. The flip side ofthis flexibility, however, is that CPOF provides fewbuilt-in processes to support specific work flows. Asa result, CPOF can require significant user interac-tion to complete tasks (that is, it is “click inten-sive”).

Task learning provides tremendous value forCPOF by enabling individual users and collectivecommand staffs to create customized, automatedinformation-management schemes tailored toindividual preferences and the staff’s standardoperating procedures, without needing softwareengineers for extensive recoding. Task learning canreduce work load and stress, can enable managingmore tasks with better effectiveness, and can facil-itate consideration of more options, resulting inbetter decisions.

In Learning by Demonstration to Support Mili-tary Planning and Decision Making (Garvey et al.2009), we described the core task-learning tech-nology, its integration into an initial PAL-CPOFprototype, and an extensive exercise-based evalua-tion of the prototype conducted by the U.S. Armyin December, 2008. At this evaluation, users over-whelmingly endorsed the capabilities provided bytask learning. Users stressed that task learning“saves time and that time equals lives.” The resultsled to a recommendation by the Army to fullyincorporate PAL task learning into CPOF, with theobjective of deploying it for operational use.

That objective has been realized: PAL task learn-ing has been integrated into the mainline CPOFsystem and its incremental fielding throughout theU.S. Army was begun in the summer of 2010. ForCPOF users, task learning speeds time-critical pro-cessing, eases task loads, reduces errors in repeti-tive tasks, and facilitates standardization of opera-tions.

This article describes our experiences in transi-tioning the initial PAL-CPOF prototype to the field,summarizing valuable lessons learned along theway. As one would expect, the deployment processinvolved refining and hardening the prototypecapabilities along with significant systems integra-tion work. In addition, we continuously engagedwith users to ensure the technology’s usability andutility. This engagement included working closelywith a U.S. Army unit that was preparing fordeployment, to help personnel incorporate tasklearning into their operational processes.

One unexpected outcome of the interactionswith the unit was an expanded value propositionfor task learning, moving beyond the originalmotivation of automating time-consuming

processes to further include standardization ofprocesses and products. We collaborated exten-sively with the unit to develop a comprehensivelibrary of learned procedures that capture criticalwork flows for their daily operations. Interestingly,the unit made fundamental operational changes totake greater advantage of the automation enabledby task learning.

We begin with an overview of CPOF followed bya summary of the PAL task learning. We thendescribe the process of getting to deployment, cov-ering technical challenges encountered, unitengagement activities, and an Army-led assess-ment of the technology. Next, we discuss the field-ing of the technology, including trade-offs made toensure deployability, the impact of the deployedtechnology, and lessons learned. We close with asummary of ongoing work to deploy additionalfunctionality and to broaden the user base for tasklearning in CPOF.

Command Post of the FutureCommand Post of the Future (see figure 1) is astate-of-the-art command and control visualiza-tion and collaboration system. CPOF originated ina DARPA program focused on advanced user inter-face design for C2 environments. It grew out of aneed to enable distributed command posts toprocess greater amounts of information and to col-laborate effectively on operations. CPOF is built onthe CoMotion platform, which was derived fromvisualization research on SAGE (Roth et al. 1994)and Visage (Roth et al. 1996).

Three design concepts lie at the heart of CoMo-tion: information centricity, direct manipulation,and deep collaboration.

Information Centricity. CPOF is organized roundthe principle of direct interaction with informa-tion. In any C2 environment, the ability to incor-porate new information dynamically is critical tothe success of an operation. CPOF uses geospatial,temporal, tabular, and quantitative visualizationsspecifically tailored to accommodate informationin the C2 domain. Though many prior visual ana-lytics tools operate on static data dumps, CPOF’s“live” visualizations continually update inresponse to changes sourced from users’ interac-tions or from underlying data feeds. CPOF is high-ly composable, permitting users to author newinformation directly in visualizations or to createcomposite work products by assembling multiplevisualizations in a single container “product.”

Direct Manipulation. CPOF makes heavy use ofdrag-and-drop and other direct manipulation ges-tures to afford users content management, editing,and view control operations. By employing a smallset of interactions with great consistency, simplic-ity and predictability emerge.

Articles

16 AI MAGAZINE

Deep Collaboration. CPOF offers a deep collabo-ration capability, beyond pixel sharing and chat.Any visualization or composite product in CPOFsupports simultaneous interaction by every userwith access to it, supporting the collaborative cre-ation of plans and analysis products. Leveraging an“over the shoulder” metaphor, sharing in CPOFhappens as a natural side effect of user activities,providing shared visibility among distributed teammembers (just as sharing occurred naturallyamong colocated users in command posts prior toCPOF).

CPOF is used daily at hundreds of distributedcommand posts and forward operational bases.The software spans organizational echelons fromcorps to battalion, with users in functional areasthat include intelligence, operations planning, civ-il affairs, engineering, and ground and aviationunits. CPOF is used extensively to support C2 oper-ations for tasks covering information collection

and vetting, situation understanding, daily brief-ings, mission planning, and retrospective analysis.A detailed description of CPOF’s operational utili-ty is provided in Croser (2006).

The CPOF interface enables users to composeand share many “simultaneous but separate” prod-ucts tailored to task and area of responsibility. Onesuch product is a storyboard, an example of whichis depicted in figure 2. A storyboard is typically cre-ated in response to a significant event, such as adowned aircraft. While the content and layoutvary for different classes of events, storyboards typ-ically provide a summary of key details, relevantoperational graphics, impact analyses, and recom-mended actions. Other products created withinCPOF include information briefings, commonoperating pictures, resource-level estimates, andfragmentary orders.

Users can collaborate synchronously in CPOF byinteracting with a set of shared products. The live-

Articles

SUMMER 2012 17

Shared visualizationsfacilitate deep collaboration

across all users

Direct manipulation interfaceprovides consistent and

predictable behavior acrossthe system

A highly composableworkspace supports individual

tailoring of visualizationsbased on mission-speci�c

work�ows

Private visualizations supportindividual work activities while

information centricity maintainscurrency throughout the system

Figure 1. A CPOF Display for a Notional Mission, Showing Geospatial and Tabular Displays of Information along with Palettes for Quick Drag-and-Drop Insertion of Information.

ness of visualizations ensures data updates flowrapidly to users, and the “over the shoulder”metaphor allows one user to monitor another’swork, thus minimizing interruptions with requestsfor information. Collaboration group size spansthe spectrum from individual (analysis, eventtracking) to small group (mission planning) toorganization wide (update briefings, missionrehearsals).

Each major deployment typically has severaldata partitions (each a CPOF repository) that allowthe U.S. Army to manage risk and redundancy forthe overall system. Each major deployment hasdedicated field support representatives and seniortrainers residing with the unit on their bases toadminister the system and support its users.

Task Learning in PAL-CPOF The simplicity of interaction with CPOF affordsusers great flexibility in creating rich visualizationsover a wide variety of data. But this powerful capa-bility often comes at the cost of time-consuming,click-intensive processes. Task learning addressesthis problem by letting users automate repetitive

processes, allowing them to focus on more cogni-tively demanding tasks. For example, the outlinefor the storyboard depicted in figure 2 can takemany minutes to create manually, thus slowingdown the process of responding to time-criticalevents. Task learning can be used to create a pro-cedure that generates a storyboard template auto-matically when invoked, enabling the soldier tofocus immediately on content for the storyboardrather than having to devote precious time to rotevisualization construction.

In the paper by Garvey et al. (2009), wedescribed the initial integration of task learninginto CPOF. Here, we provide a brief overview of thelearning technology, the procedure execution andediting functionality, and the rules and work flowsthat together constitute the task-learning capabili-ty added to CPOF.

LAPDOG: Learning by DemonstrationThe LAPDOG system (Gervasio, Lee, and Eker2008; Eker, Lee, and Gervasio 2009; Gervasio andMurdock 2009), depicted in figure 3, provides thelearning by demonstration capabilities from thePAL program that support task learning withinCPOF. Given a trace consisting of a sequence of

Articles

18 AI MAGAZINE

A map provides detailedgeo-spacial informationand context of relevance

to the event

Different functional areascollaboratively document

their impact analyses

Critical information forthe event is summarized

Recommended actionsde�ne selected

response to the event

Figure 2. A Sample CPOF Storyboard.

actions performed by a user to achieve a particulartask, LAPDOG generalizes the trace into a proce-dure for achieving similar tasks in the future.

An action model provides the semantic descrip-tion of the demonstrable actions in an application,together with mechanisms for the instrumentationand automation of those actions. An early chal-lenge in the development of the CPOF action mod-el was finding the right level of abstraction. Cap-turing actions at the level of primitive datachanges (for example, create an entity, move a con-tained entity to the front/back of its container)allowed for a very compact action model andgreatly simplified instrumentation but led tolengthy procedures with minimal generalizationand poor comprehensibility. To support an actionmodel at the level humans typically describe theiractions (for example, dispense a frame, center amap on an entity) required investing in a transla-tion layer to convert lower-level events into corre-sponding higher-level actions. The resulting actionmodel thus comes at a higher cost of instrumenta-

tion but enables more successful generalizationand increased understandability for visualizationand editing. The current action model covers most,although not all, of the core user activities inCPOF.

LAPDOG employs a data flow model in whichactions are characterized by their inputs and out-puts, with outputs serving as inputs to succeedingactions. LAPDOG generalizes demonstrations intoprocedures in two ways. First, it performs parame-ter generalization, unifying action outputs andinputs and replacing constants with variables orexpressions over variables to capture the data flowin the demonstration. Second, it performs structuregeneralization, inducing loops over collections ofobjects by generalizing repeated sequences ofactions.

LAPDOG’s loop induction algorithm supportsadvanced features such as the ability to learn loopsthat iterate over ordered or unordered collectionsof objects (lists and sets), involve expressions over

Articles

SUMMER 2012 19

LearnedProcedure

structure generalization

ABCDBCDBCDE

A(BCD) *E

induce loops over: • sets or lists • multiple lists in parallel • functional expns over lists • generating lists

data!ow completion

search over info-producing actions and KB relations

A2 A1

Ak

Aj

A2A1

HypothesisSpace

A(– a) B(-m)C(+a +m – [b c d])D(+b –j)E(+d –k)G(+[j k])

A(–$X)B(–$Y)C(+$X +$Y –$Z)D(+ !rst ($Z) –$U)E(+ last ($Z) –$V) G(+ list ($U $V))

parameter generalization

!nd all uni!cations/variablizations across terms

H isesiHypotheSpace

EventTrace

LearnedProcedure

heuristic selection

prefer: • shorter procedures • direct supports • existing supports • closest support

Figure 3. Learning Procedures by Demonstration in LAPDOG.

those collections, process multiple lists simultane-ously, and accumulate outputs. Although a power-ful generalization technique, loop induction hasbeen surprisingly difficult to see fully realized inCPOF for two primary reasons: (1) iteration is a dif-ficult concept for end users, and (2) identifying col-lections in CPOF currently involves idiosyncraticgestures in the user interface.

LAPDOG features the ability to infer actionsequences for completing data flow in situationswhere the linkage between outputs and inputs isimplicit. For example, in preparing a text messagebased on an incident report, the user will typicallyuse information from particular fields in thereport.  Because instrumentation will not capturethis linkage, without data flow completion thelearned procedure would require the message textto be input by the user, thus losing the implicitrelations between the report fields and portions ofthe message.  However, by inferring these relations,LAPDOG can instead induce a parameterized mes-sage template for use in preparing text messages forthe procedure.  This inference of implicit data flowimproves LAPDOG’s ability to generalize across theparameters of procedure actions, reducing theinputs required for a learned procedure and so sim-plifying its use.

LAPDOG was not specifically designed to learnfrom a single example but that has become its pri-mary mode of use in CPOF since users have gener-ally been unwilling to provide multiple demon-strations. LAPDOG generalizes from a singleexample with the aid of heuristics for filtering theset of alternative hypotheses. Specifically, it prefersmore recent or more direct supports, action out-puts over procedure inputs, existing procedureinputs over new ones, and loops over straight-linealternatives.

Procedure Execution and EditingIn the initial PAL-CPOF prototype, the SPARKagent framework (Morley and Myers 2004) wasused to execute learned procedures. SPARK is a fea-ture-rich system designed to support the sophisti-cated control and reasoning mechanisms requiredby practical agent systems (for example, Morley,Myers, and Yorke-Smith 2006; Yorke-Smith et al.2009). The execution of learned procedures withinthe deployed system is provided by a lightweightversion of SPARK focused specifically on procedureexecution.

Because many CPOF users have no program-ming experience, it is important to provide com-prehensible visualizations that communicate whata learned procedure does and to assist users inmaking valid edits. This is particularly importantin collaborative environments such as CPOF, toallow users to benefit from being able to modifyand reuse procedures created by others. On the PAL

project, we developed technology for procedurevisualization and assisted editing (Spaulding et al.2009). Our approach, refined through feedbackfrom numerous user engagements, augments theaction model with a flexible metadata capability toimprove visualization and editing. It also employsprocedure analysis techniques to detect editingproblems and suggest fixes.

Composing Higher-Level Work Flows Learned procedures provide significant value byautomating time-consuming functions in CPOF,such as the creation of storyboard templatesdescribed earlier. Our earlier interactions with theU.S. Army revealed that the value of task learningwas greatly enhanced by introducing rules toenable automatic invocation of learned proceduresbased on some triggering event. Rules were usedextensively in the original PAL-CPOF prototype tofacilitate rapid response to a range of operationalevents. Three types of rules were supported: timerules, data rules, and area rules. Time rules invokeprocedures at absolute or relative times or with aspecific frequency, data rules invoke proceduresbased on particular changes to properties of objectsin the system, and area rules invoke proceduresbased on particular changes to predefined areas ona map. For example, a data rule could be createdthat invokes a procedure for creating a storyboardtemplate tailored to a particular event class whenan event of that type is recorded in the system.

Taking this concept a step further, as shown infigure 4, it is possible to compose collections ofrelated rules and procedures into more complexwork flows that automate larger chunks of func-tionality. Work flows open the door to automatingcritical responses to significant events in accordwith standard operating procedures (SOPs). Forexample, PAL work flows have been created thatautomatically construct and arrange work spaces,notify selected staff of events, track assets, and pre-pare simple reports for the user’s approval and dis-semination. As discussed below, work flows provedinstrumental to our deployment strategy but intro-duced some additional challenges.

Given that work flows encompass multiple pro-cedures and rules that can interact in potentiallysubtle ways, creating complex work flows requiresa level of sophistication not met by most CPOFusers. Successful management of work flowsrequired us to consider classes of users with differ-ent capabilities and responsibilities, A significantpart of our design effort leading up to deploymentwas devoted to creating tools and best practices tofacilitate understanding and management of anecosystem of interacting procedures and rules byusers of differing skill levels.

Articles

20 AI MAGAZINE

Getting to Deployment Our efforts to deploy the PAL task-learning tech-nology within CPOF involved two main thrusts.One focused on the hardening and refinement ofthe task-learning capabilities, along with their inte-gration into the mainline CPOF system. The sec-ond focused on user engagement to ensure theoperational effectiveness of the technology. Thissection describes key challenges that arose on thetechnical side, along with our user engagementefforts and their impact on the development anddeployment processes.

Technical Challenges Transitioning the task-learning technology intoCPOF presented numerous software-engineeringchallenges, independent of the learning capabilityitself. The collaborative nature of CPOF requiresintegration to be sensitive to problems inherent to

distributed, multiuser systems. CPOF is a fieldedplatform but also continues to evolve over time.Finally, the data that CPOF manages is dynamicand voluminous.

Deployment of the technology also raised anumber of challenges more directly linked to tasklearning. One was developing an expressive yetsustainable action model. To address this chal-lenge, we evolved our action model framework tobe extensible so that it could both grow to accom-modate changes in CPOF and enable proceduresfrom prior versions to be upgraded easily to newversions.

High-volume, concurrent procedure executionpresented a second challenge. The PAL task-learn-ing technology was developed originally as part ofan intelligent desktop assistant, for which execu-tion typically involved small numbers of proce-dures executed one (or a few) at a time underexplicit user control. During exploration of the use

Articles

SUMMER 2012 21

Time

Area

Data

129

63

Procedure: A parameterized sequence of steps thatautomates a family of related tasksExample: a procedure that creates a storyboard template for aparticular type of Signi!cant Event

Work ow: A collection of related rules and proceduresthat automate more broadly scoped functionality

Example: An Asset Tracker work"ow that uses rules and proceduresto actively maintain up-to-date status information for designatedsets of resources

Rule: A mechanism for automatically invoking alearned procedure based on a triggering event relatedto a change in time, data, or elements within a geographical areaExample: a rule that automatically invokes the storyboard generationprocedure when a Signi!cant Event of that type is reported

129

63

Figure 4. PAL Automation Elements: Procedures, Rules, and Work Flows.

of the task-learning technology within CPOF, theautomatic invocation of procedures through rulestook on an increasingly significant role. In thedeployed version of the system, rule-based invoca-tion can lead to hundreds of procedures executingsimultaneously, which was well beyond the designparameters for the desktop assistant. Addressingconcurrency with high volumes of procedure exe-cution required a complete reimplementation ofthe PAL-CPOF communications infrastructure toeliminate timing problems triggered by these high-er than anticipated invocation rates.

Another significant challenge was the difficultyof porting PAL products (procedures or work flows)between units, which have data elements specificto their needs. Task learning generates parametersbased on the data elements referenced in a demon-stration. Procedures easily fit into user workprocesses because dependencies on the unit-spe-cific data partition are encoded as default parame-ters. However, ensuring that a PAL product worksfor other units requires “rewiring” those defaults tothe appropriate local data sources. For example,one key data element within a CPOF repository isthe SigActs table, which records the significantactions of relevance to a particular unit. A unitlinked to a different CPOF repository would haveits own local SigActs table. For a procedure taughtwithin the context of one repository to work on asecond repository, default references to the SigActstable would need to be reset accordingly. For com-plex products, this localization process proved tobe particularly demanding, requiring deep knowl-edge of both the author’s intent and the unit’s data

configuration. For the initial deployment, werelied on human experts to perform this task.

User Engagement Prior to FieldingOur transition plan emphasized continuous userinteractions to develop and refine the task-learn-ing concept in CPOF. To this end, we conducted aseries of user-centered design exercises with thedeploying unit. These exercises were conductedusing the Double Helix methodology that wasapplied successfully to the development anddeployment of the initial CPOF system.

The name Double Helix derives from simultane-ous technology refinement based on insights fromuser interactions and refinement of the Concept ofOperations (CONOPS) for its use within the opera-tional domain (see figure 5). In the Double Helixmethodology, technologists work with users inrealistic settings to uncover technology issues,understand the operational “sweet spot” for thecapability, and identify opportunities for its use tosolve relevant operational problems.

Our Double Helix exercises yielded severalinsights that informed our deployment effort.

Initial demonstrations of simple PAL capabilitiesfacilitated user acceptance of the technology. In gener-al, introducing new technology into a large-scale,distributed operational environment requires strictfeature specifications and configuration control. Inaddition, deployment typically requires safety andencapsulation for users with limited training whileplacing unfortunate restrictions on more experi-enced users. Such requirements run counter to thespirit of end-user task learning and could be

Articles

22 AI MAGAZINE

New designs emerge from a cyclical process of design and evaluation. Frequent experimentation with interim software in realistic scenarios simultaneously evaluates

the CONOPS and the technology. At every step, designers better understand userneeds and domain experts gain new insights into what is possible.

CONOPS

Technology

Practitioners Prototype and TestTechnologists

Figure 5. CPOF’S Double Helix Process: Building Common Ground between Technology Developers and Domain Experts.

expected to reduce user acceptance of such tech-nology. To mitigate this risk, our engagement teamintroduced PAL by initially creating small proce-dures that automated routine, repetitive tasksrather than complex work flows. By first under-standing these simple examples, users were morelikely to embrace the technology and to conceiveof more complex support that PAL could provide.This approach proved successful in enabling low-risk, progressive adoption of the technology.

Creation of a library of automation products built byexpert users greatly increased adoption of the technolo-gy. During user engagements, we identifiedCONOPS common to multiple units that could besupported by generic, but sophisticated, PAL workflows. Expert users were able to automate several ofthese “best practice” work flows using task learn-ing. This led to the introduction of a PAL Library asa way to disseminate these more advanced workflows.

The PAL Library significantly strengthened thecapability deployment process. First, it conveyedto users the “art of the possible” (that is, what canbe done with the PAL technology). Prior to devel-opment of the library, users often had limitedunderstanding of how PAL products could helpthem. Afterward, there was a marked increase incapability expectations among the user base. Sec-ond, it provided complete interactive productswith more compelling functionality than individ-ual procedures. Third, it enabled users to reap thebenefits of PAL without having to understand howto create procedures and rules. Finally, the librarycreated an effective conduit to introduce enhancedcapabilities into the field.

Incremental integration of PAL capabilities intooperational SOPs was essential for successful technolo-gy transition. Initial attempts to automate collabo-rative processes monolithically failed because thechanges imposed on the users were too substantial.Based on these early experiences, we switched to amore gradual approach that would enable PALwork flows to be adopted incrementally into exist-ing SOPs. Our adoption path consisted of the fol-lowing steps: (1) an isolated work flow for a singleuser, (2) an individual work flow shared by multi-ple users, (3) a set of individual work flows withina PAL-initialized organizational process, and (4) aset of collaborative work flows supporting multipleusers. Following this path facilitated incrementaladoption rather than an all-or-nothing change,which led to increased user acceptance for thetechnology. In this way, adoption could be “con-tagious.” This approach enabled the use of tasklearning to expand beyond automating currentSOPs to providing higher-order decision supportthat would not otherwise have been practical.

NTC AssessmentThe U.S. Army performed an assessment of the PALsoftware midway through the project to determinewhether to proceed with deployment. The assess-ment targeted both the operational utility of thePAL capabilities and system performance metrics(for example, bandwidth utilization, server/clientperformance, software quality). To enable realisticoperating conditions, the Army chose to performthe assessment in conjunction with the unit’s stan-dard three-week rotation at the National TrainingCenter (NTC) at Ft. Irwin in May of 2010.

The purpose of the NTC rotation was to preparethe unit for operational deployment through avariety of training exercises, culminating in a“final exam” executing a realistic mission overseenby an experienced team of observer/controllers.The unit was provided with an engineering releaseof a version of CPOF that included the learningtechnology and the PAL Library that it had helpedto develop in the months leading up to the event.The unit had the freedom to use PAL capabilitiesto the extent that it found them helpful for com-pleting their assigned mission; however, evalua-tion of the PAL capabilities was not an explicitobjective for the soldiers.

Assessment of the PAL technology was donethrough a series of questionnaires that wereadministered to members of the unit and theobservers/controllers. These questionnaires solicit-ed feedback on both the usability of the PAL tech-nology and the extent to which it enabled the unitto conduct its operations more effectively. Basedon the responses to these questionnaires, the Armyconcluded that the PAL technology significantlyimproved CPOF operations throughout the rota-tion, in certain situations reducing hours of workto minutes. A report summarizing the assessmentstated that PAL capabilities “improve a unit’s capa-bility to efficiently execute their roles, functionsand missions through better collaboration andinformation sharing.” It was also determined thatthe PAL capabilities did not negatively affect thesystem performance metrics of interest. Giventhese results, the Army decided to proceed with thefull integration of PAL task learning into CPOF andto allow the unit to deploy with the PAL capabili-ties.

Fielding The original plan for deployment involved aroughly two-year effort split over two phases. Thefirst phase was to begin integration into the main-line CPOF system while concurrently developingnew functionality (informed by user engagement)that would further enhance the value of task learn-ing within CPOF. The second phase was to focus

Articles

SUMMER 2012 23

on hardening and integration, with the objectiveof delivering the final capability for operationaldeployment.

Early in the effort, the U.S. Army requested thatthe deployment cycle be accelerated in order to getthe task-learning technology into the hands ofusers faster. This expedited fielding led to a require-ment for a hardened capability that was fully inte-grated into CPOF roughly nine months after thestart of the effort. Not all of the planned technicalfunctionality could be sufficiently hardened andintegrated to meet this aggressive new schedule.The end result was that capabilities would be lim-ited in the first release, with additional functional-ity planned for incorporation into subsequentreleases of CPOF.

Functionality Trade-offsTwo main considerations determined what func-tionality to make available in the first deployment.

One consideration was the potential for users toteach and execute procedures with deleterious sideeffects, such as compromising critical data or inter-fering with system operation.

A second consideration was the need for train-ing to take full advantage of the capabilities afford-ed by the PAL technologies. Our Double Helixinteractions introduced us to the real world ofdeployment, redeployment, and assignment rota-tion, wherein the pool of users available for train-ing changed frequently, never really reaching thecritical mass of skill needed to fully understandCPOF and PAL. Indeed, many of the users withwhom we interacted obtained the bulk of theirCPOF knowledge through informal, on-the-jobtraining in the form of interactions with moreskilled colleagues. This sort of informal trainingrarely affords the trainee the opportunity to createa mental model sufficient to understand the con-sequences of end-user programming.

Based on these considerations, we decided notto make the learning by demonstration capabilitydirectly available to all users of CPOF in the initialdeployment. Rather, a typical user could insteadaccess prebuilt procedures and work flows con-structed by a small team of well-trained librarybuilders. A typical user has permission to run anyavailable library product (work flow or procedure)but not to extend or modify library products, oreven create stand-alone procedures that addresstheir individual needs. Library builders have accessto the core learning by demonstration capability inorder to extend and modify the library in theater.In our initial deployment, the library builders arefield support and training personnel, who are morereadily trained and have greater facility with pro-gramming constructs, making it easier for them todemonstrate and manipulate procedures. It isimportant to note, however, that these library

builders are typically not professional softwareengineers and gain a great deal of flexibility andpower from the task-learning capability.

Imposing this restriction was a difficult decisionto make, as the team is fully committed to makingthe learning technology available to all CPOFusers. Ultimately, however, we decided that itwould be advisable to roll out the technologyincrementally to provide some of the benefits ofthe learning capability while minimizing associat-ed risks. As discussed in the Ongoing Work sectionbelow, we are continuing to extend the range ofthe capabilities that are available to all users of thesystem and are on track to deploy those in futurereleases of CPOF.

Operational DeploymentFor approximately six months beginning in latesummer of 2010, a team deployed first to Iraq andthen to Afghanistan to upgrade CPOF to a versioncontaining the PAL learning technology. Theupgrade team consisted of senior trainers to con-duct classes and teach best practices, softwaredevelopers to oversee the upgrade and collect per-formance and debugging information, and a unit-engagement team to help tailor and extend thePAL Library using task learning. Over severalmonths, the upgrade team visited each of theCPOF installations and spent 2–5 weeks conduct-ing training classes, upgrading the repository, andproviding one-on-one support.

Despite the military’s reputation as a regiment-ed organization, its processes can be surprisinglydecentralized; each unit has a unique operatingenvironment and is given a fair degree of leewayto accomplish its missions. Further, each com-mander sets his own operating procedures andreporting requirements. The engagement team firstcollected information from the unit about its localprocesses, then used this data to customize thegeneric, previously developed library work flows tothe needs of the individual unit. The engagementteam iterated with the commanders and CPOFoperators to teach them how to customize and usethese work flows, and helped with further refine-ments. Concurrently, the team trained the fieldsupport representatives to provide ongoing sup-port for the new capabilities.

There was substantial variability in adoption lev-els for the task-learning technology. One unit thathad just arrived in theater was adapting theprocesses left behind by the outgoing unit and wel-comed the assistance to refine and automate theirprocesses. In cases like this, the engagement teamobserved substantial adoption of PAL work flowsand a high level of enthusiasm for the new capa-bilities (with one unit giving the team a standingovation and high-fives after a demonstration of thecapability). In contrast, a unit that was close to

Articles

24 AI MAGAZINE

rotating out of theater had little interest in takingon the risk of new processes and tools.

One indicator of the degree of adoption of PALwas the number of modification requests received.In a few cases, the engagement team had little con-tact with the unit after customizing its PAL workflows and conducting training; in other cases, theteam received a steady stream of requests forchanges and new features. The ability to receive arequested enhancement in days to weeks ratherthan the typical 18-month software release cyclefor CPOF generated tremendous excitement, open-ing the door for units to adapt CPOF on the fly toin-theater dynamics.1

In several instances, the engagement teamworked with users who had minimal CPOF experi-ence. This inexperience was a disadvantage duringtraining because it meant spending a substantialamount of time on basic system usage. However,this inexperience also provided a substantialadvantage — these users drew little distinctionbetween native CPOF capabilities and PALenhancements. This allowed them to leapfrog theirmore experienced peers: rather than first learn amanual process and then understand the PAL pro-cedures to automate that process, these users grav-itated straight to using the automation.

One of the biggest lessons learned was theinverse correlation between the user-perceivedcomplexity of a PAL work flow and the degree towhich it was adopted. For example, the “story-board creator” work flow allowed an individualsoldier to create a visual summary of a significantevent in only a few clicks. This work flow fit wellwithin existing processes and produced story-boards that looked similar to those that the sol-diers created manually. As such, soldiers couldincorporate this work flow into existing processeswith minimal change. This incremental adoptionallowed a few soldiers to try the new tools, whileothers took a wait-and-see approach. Adoptionspread as word-of-mouth recommendation over-came initial reluctance.

On the other hand, a powerful but complexSigAct (that is, significant action) managementwork flow saw substantially less adoption. Thismay have been partially due to the modeledprocesses being overly specific to the unit that weworked with prior to deployment. Worse, its bene-fits were dependent on having all of the partici-pants in the SigACT reporting process switch to thenew work flow at once. Beyond the original unit,there was little enthusiasm for moving everyone tothis new process simultaneously. Based on thesefindings, the engagement team made modifica-tions to the SigACT work flow so that soldiers canuse portions of it without requiring a specific pre-scribed reporting structure.

As units embraced the PAL technology, some

began to formulate new automated work flowsbased on their individual operational needs. Manyof these work flows centered on automating exist-ing processes. However, a handful, such as an assettracker, suggested entirely new unit SOPs (forexample, making a transition from manually edit-ing text tables to tracking and recording asset sta-tus automatically), thus showing evidence of usersmodifying their behavior to more fully take advan-tage of the technology.

Ongoing Work As noted above, concerns about safety, usability,and training led to our decision to restrict access totask learning in the initial deployment. To addressthese concerns, we have delivered several capabili-ties for incorporation into the next release of CPOFand are continuing to develop others for a thirdand final planned integration phase. We expectthat incorporating these mechanisms will enableresponsible extension of the task-learning capabil-ity to all users of CPOF.

One strategy has been the introduction of safe-guards to limit inappropriate procedure execu-tions. These safeguards range from permissionsand type checking for identifying procedures thatare expected to fail during execution, to identify-ing loops with inordinately large numbers of itera-tions at execution time (which could signal a mis-guided application of the procedure). A secondstrategy has been additional support for navigatingthe library, making it easier for users to locate workflows of interest and instantiate them to work intheir local environments.

Finally, we have built an interactive editor with-in CPOF that enables users to visualize and modi-fy learned procedures. The editor, derived concep-tually from the original PAL prototype, is expectedto increase understanding, and hence acceptance,of the task-learning technology among basic userswhile providing advanced users with the means toadapt learned procedures quickly rather thandemonstrating them again from scratch to incor-porate changes. The ability to modify and reuseprocedures created by others is particularly valu-able in CPOF, where sharing is inherent to theunderlying collaborative user experience.

ConclusionsThe PAL learning by demonstration technologyhas seen wide adoption within CPOF, a U.S. Armysystem of record employed daily by thousands ofsoldiers as an integral part of their mission opera-tions. The open-ended composability of CPOFmakes it an ideal target for the end-user automa-tion enabled by the learning by demonstrationtechnology. Although integration into CPOF pre-

Articles

SUMMER 2012 25

sented numerous technical and operational chal-lenges, the result is a customizable system thatreduces time spent on tedious tasks while also facil-itating standardization of best practices. By freeingsoldiers to focus on higher-level tasks, PAL signifi-cantly improves mission-execution capabilities.

Our experiences in fielding the task-learningtechnology to the U.S. Army offers guidance forfuture such efforts. User acceptance is greatly facil-itated by a deep understanding of the processesthat the technology will support, flexibility toallow incremental adoption, and ongoing userengagement that includes technology demonstra-tions to highlight “the art of the possible.”

AcknowledgmentsThis material is based upon work supported by theUnited States Air Force and DARPA under Con-tracts No. FA8750-05-C-0033 and FA8750-09-D-0183. Any opinions, findings and conclusions, orrecommendations expressed in this material arethose of the authors and do not necessarily reflectthe views of the United States Air Force andDARPA.

The authors thank Dr. Robert Kohout (DARPA),LTC Richard Hornstein (Army TBC), and AndyHarned (Army TBC) for their leadership and sup-port, and BG(R) Pat O’Neal, MG(R) Thomas Gar-rett, COL Reginald Allen, COL Justin Gubler, LTCMichael Spears, LTC Brian Gruchacz, MAJ CharlesAyers, MAJ Anthony Whitfield, and Marshall Reed,for their assistance in aligning the task learningtechnology with user needs.

The authors also thank the following individualsfor their contributions to system development,library building, and user engagement: ChickAgnew, Susan Alsue, Naomi Anderson, Stuart Belt-son, Sonya Berkovich, Sharon Booth, KåreDigernes, Patrick Dominic-Savio, Vanessa Fine, EricGardner, Dave Goldstein, Daragh Hartnett, TimHolland, David Jacobs, Melinda Janosko, ChrisJoslin, Suzanne Kafantaris, Eric Kaun, AnthonyKim, Warren Knouff, Tom Kilsdonk, Tom Lee, PatMartin, Tom Martin, John Parise, Seth Porter, Per-ry Rajnovic, Andrew Riggs, Shahin Saadati, JoanSaz, Dennis Sigel, Rob Smoker, David Stevens, JimStraub, Rebecca Torbochkin, and Rob Umberger.

Approved for public release, distribution unlim-ited.

Note1. The nine-month deployment of the PAL capability wasenabled by the army’s desire to transition the technolo-gy as soon as possible, which led to an opportunisticincorporation of the technology into a previouslyplanned release.

ReferencesAllen, J.; Chambers, N.; Ferguson, G.; Galescu, L.; Jung,H.; Swift, M.; and Taysom, W. 2007. PLOW: A Collabora-

tive Task Learning Agent. In Proceedings of the 22nd AAAIConference on Artificial Intelligence. Menlo Park, CA: AAAIPress.

Blythe, J.; Kapoor, D.; Knoblock, C. A.; Lerman, K.; andMinton, S. 2008. Information Integration for the Masses.Journal of Universal Computer Science 14(11): 1811–1837.

Burstein, M.; Laddaga, R.; McDonald, D.; Benyo, B.; Cox,M.; Roberston, P.; Hussain, T.; Brinn, M.; and McDermott,D. 2008. POIROT — Integrated Learning of Web ServiceProcedures. In Proceedings of the 23rd AAAI Conference onArtificial Intelligence, 1274–1279. Menlo Park, CA: AAAIPress.

Croser, C. 2006. Commanding the Future: Commandand Control in a Networked Environment. Defense andSecurity Analysis 22(2): 197–202.

Cypher, A.; Dontcheva, M.; Lau, T.; and Nichols, J., eds.2010. No Code Required: Giving Users Tools to Transform theWeb. Amsterdam: Morgan Kaufmann.

Eker, S.; Lee, T.; and Gervasio, M. 2009. Iteration Learn-ing by Demonstration. In Agents That Learn from HumanTeachers: Papers from the 2009 AAAI Spring Symposium.Technical Report SS-09-01. Menlo Park, CA: AAAI Press.

Garvey, T.; Gervasio, M.; Lee, T.; Myers, K.; Angiolillo, C.;Gaston, M.; Knittel, J.; and Kolojejchick, J. 2009. Learn-ing by Demonstration to Support Military Planning andDecision Making. In Proceedings of the of the 21st Confer-ence on Innovative Applications of Artificial Intelligence.Menlo Park, CA: AAAI Press.

Gervasio, M.; Lee, T.; and Eker, S. 2008. Learning EmailProcedures for the Desktop. In Enhanced Messaging: Papersfrom the 2008 AAAI Workshop. Technical Report WS-08-04. Menlo Park, CA: AAAI Press.

Gervasio, M., and Murdock, J. 2009. What Were YouThinking? Filling in Missing Dataflow through Inferencein Learning from Demonstration. In Proceedings of the2009 International Conference on Intelligent User Interfaces.New York: Association for Computing Machinery.

Leshed, G.; Haber, E.; Matthews, T.; and Lau, T. 2008.CoScripter: Automating and Sharing How-to Knowledgein the Enterprise. In Proceedings of the 27th InternationalConference on Human Factors in Computing Systems (CHI).New York: Association for Computing Machinery.

Morley, D., and Myers, K. 2004. The SPARK Agent Frame-work. In Proceedings of the 3rd International Joint Confer-ence on Autonomous Agents and Multiagent Systems. LosAlamitos, CA: IEEE Computer Society.

Morley, D.; Myers, K.; and Yorke-Smith, N. 2006. Con-tinuous Refinement of Resource Estimates. In Proceedingsof the 5th International Joint Conference on AutonomousAgents and Multiagent Systems. New York: Association forComputing Machinery.

Roth, S. F.; Kolojejchick, J.; Mattis, J.; and Goldstein, J.1994. Interactive Graphic Design Using Automatic Pres-entation Knowledge. In Proceedings of the Conference onHuman Factors in Computing Systems (CHI 1994), 112–117.New York: Association for Computing Machinery.

Roth, S. F.; Lucas, P.; Senn, J. A.; Gomberg, C. C.; Burks,M. B.; Stroffolino, P. J.; Kolojejchick, J.; and Dunmire, C.1996. Visage: A User Interface Environment for Explor-ing Information. In Proceedings of the IEEE Symposium onInformation Visualization, 3–12. Piscataway, NJ: Institue ofElectrical and Electronics Engineers.

Sanders, R.; Milligan, R.; and Roberts, K. 2009. DARPA

Articles

26 AI MAGAZINE

Command Post of the Future Enhanced with the Person-alized Assistant that Learns 2008 Demonstration. Tech-nical Report, Battle Command Battle Laboratory – Ft.Leavenworth. Washington, DC: US Army.

Spaulding, A.; Blythe, J.; Haines, W.; and Gervasio, M.2009. From Geek to Sleek: Integrating Task LearningTools to Support End Users in Real-World Applications.In Proceedings of the 2009 International Conference on Intel-ligent User Interfaces. New York: Association for Comput-ing Machinery.

Yorke-Smith, N.; Saadati, S.; Myers, K.; and Morley, D.2009. Like an Intuitive and Courteous Butler: A ProactivePersonal Agent for Task Management. In Proceedings of the8th International Joint Conference on Autonomous Agentsand Multiagent Systems. Richland, SC: International Foun-dation for Autonomous Agents and Multiagent Systems.

Karen Myers is a principal scientist in the AI Center atSRI International, where she directs a research groupfocused on planning and agent technologies. Her currentinterests lie with the development of mixed-initiative AIsystems that enable humans and machines to solve prob-lems cooperatively. She has a Ph.D. in computer sciencefrom Stanford University.

Jake Kolojejchick is an entrepreneur and innovatorfocused on creating effective, natural decision-makingenvironments that allow diverse, distributed teams tocope with highly uncertain, information-intensivedomains. He is currently chief scientist at GeneralDynamics C4 Systems, pursuing research in mixed-ini-tiative systems, cyber situational awareness, and autono-my in sensor systems.

Carl Angiolillo is a user experience designer specializingin making complicated things seem simple. He has anM.S. in human-computer interaction from Carnegie Mel-lon University where he programmed intelligent tutoringsystems and has previous experience in software devel-opment, teaching, and theatrical set design.

Tim Cummings is a staff software engineer with Gener-al Dynamics C4 Systems. He has a B.S. in computer sci-ence from Brigham Young University and has 12 years ofindustry experience on a broad range of commercial andmilitary software projects.

Tom Garvey is a principal scientist and associate directorof SRI International’s AI Center where he oversees groupsdeveloping technologies in planning, human-centeredcontrol, agents, robotics, knowledge acquisition, learn-ing, and command and control. He has a Ph.D. in elec-trical engineering and computer science from StanfordUniversity.

Matt Gaston is the director of the Carnegie Mellon Uni-versity (CMU) Cyber Innovation Center and an adjunctassociate professor at the CMU Institute for SoftwareResearch. Prior to joining Carnegie Mellon, Gaston wasthe director of research for General Dynamics C4 Systemsand before that spent nearly 10 years at the NationalSecurity Agency.

Melinda Gervasio is a senior computer scientist in theAI Center at SRI International, where she works on inte-grated intelligent systems that utilize machine learning,automated reasoning, and human interaction, with a

particular focus on learning procedural tasks. She has aPh.D. in computer science from the University of Illinoisat Urbana-Champaign.

Will Haines is an interaction designer in the AI Center atSRI International, where he designs usable interfaces forend-user programming systems. His interests lie indeploying advanced technology into real-life operationalsettings. He has degrees in computer science and human-computer interaction from Carnegie Mellon University.

Chris Jones is a senior software engineer at SRI Interna-tional. His interests include concurrency, networks, andmultiagent architectures. He has a B.S. in computationalbiochemistry from Montana State University.

Kellie Keifer is the operations director for the AI Centerat SRI International and the program manager for thePAL Military Transition effort. Keifer has extensive expe-rience in the transition of research technology into oper-ational systems, including projects in air traffic manage-ment prior to joining SRI. She has a master’s degree incomputer science from the University of Illinois, Urbana-Champaign.

Janette Knittel is the lead product concept designer forGeneral Dynamics C4 Systems | Viz. Knittel is an expertin designing and developing systems for soldiers, com-manders, and analysts in a battle command environ-ment. She has a bachelor of fine arts degree fromCarnegie Mellon University.

David Morley is long-time consulting research scientistfor SRI International and a former SRI employee. Hisinterests are in practical applications of agent systems,especially those that involve human interaction and thatdeal with failure and change. He has a Ph.D. in comput-er science from the University of Melbourne, Australia.

William Ommert is a principal engineer with GeneralDynamics C4 Systems | Viz where he leads research andproduction software development efforts focused in theareas of visualization and system interoperability.  He hasa B.S. in mathematics and computer science fromCarnegie Mellon University and has previous experiencein robotic systems design. 

Scott Potter earned a Ph.D. in cognitive systems engi-neering from the Ohio State University in 1994 where hisresearch focused on representing intelligent system infor-mation for fault management.  He is currently a seniorsystems engineer at General Dynamics C4 Systems | Viz,where he designs visualization-centered decision-supportsystems for complex, dynamic, high-risk environments. 

Articles

SUMMER 2012 27