The Mobile Agents The Mobile Agents 2005 Field Test at 2005 Field Test at

MDRS:MDRS:Planning for ExplorationPlanning for Exploration

William J. ClanceyWilliam J. ClanceyNASA Ames Research

CenterChief Scientist, Human-Centered

ComputingIntelligent Systems Division

Brahms: Bill Clancey, Maarten Sierhuis, Ron van Hoof, Mike Scott, Serge Yentus

MEX: Rick Alena, John Ossenfort, Charles Lee, Ed Walker, Hugo Notario

RIALIST: John Dowding, Jim HieronymusGeology: Abby Semple & Brent GarryERA: Jeff Graham, Rob Hirsh, Eduardo Herrera, Bill

VreugdeRST: Shannon Rupert + 12 participating scientistsNREN: Ray Gilstrap, Kevin BassSciOrg: Dan Berrios, Rich Keller

MA 2005 Participants

QuickTime™ and aH.263 decompressor

are needed to see this picture.

Origins of MA Design:Historical & Analog Missions

Apollo’s CapCom, a virtual assistant on the moon

Biologists navigating in Haughton Crater (1999)

Crew struggles with GPS during MDRS5 simulation

QuickTime™ and aSorenson Video decompressorare needed to see this picture.

WILLIAM J. CLANCEY, PI & MAARTEN SIERHUIS, PROJECT MANAGER

Human-Systems Integration

• Device Control: sensors/instruments, tools, robots, systems (life, power, computing)

• Data Flow: Queue, sample, store, associate, share, display

• Work Management: Monitor, advise (loc, time, activity), interpret, alert

Voice-Commanded Toolkit

Technical Features

• Multi-Agent System

– Functional & personal

– Distributed control & data-processing

• Wireless Computing

– Platform independent

– Tested over 5 km

• Modular Configurations

– Plug & play (Java API)

– E.g., 5 robots, 3 cameras, 5 biosensors, 2 computer sensors, dGPS, headsets

Astronaut Backpack System

Mobile Agents Utah Field Test 2005

• 50 Participants; 17 days; 2 NASA centers & several universities

• Multi-robot, auto-recon, & freeform exploration scenarios

• More proactive ERA design

• More proactive RST

• More robust, faster, reusable agent system (> 60)

• Improved alerts, queries (> 90), & auto-associations

• EVA crew less reliant on HabCom crew member

Geologists Exploring with Mobile Agents

New MA’05 Operations

• Interacting with the robots to determine their status: Whom are you following? Do you still have comms?

• Establish dependence on robot relay: Boudreaux, join me.

• Contextual robot move commands: Come here. Go there.

• Directed photos: Take a picture of work site 5.

• Astronaut naming of “work stations" (default 1 m areas) within work sites: Call this location 9 with a radius of 5 m.

• Astronauts can refer to names teammates create.

• Requesting locations and navigation information: Where is work site 5? Where am I? List all the locations near work site 6. Where is the next activity?

New ERA Hardware

Instrument Arm Robotically Deployed Relay (RDR)

RDR Pin Assembly Thibodeaux (2nd ERA)

ERA Spiral Recon Plan

MDRS

Mobile Agents Pooh’s CornerComms Network (2005)

Rick Alena, NASA Ames,March 2005

Plan B: ATV with ERA Agent & Operator Display

Mobile Agents MDRS38Configuration

AstroAgent

HabCom

Agent

ERAAgent

ERA

Hab Crew

Astro

RST

Collaborative Interplanetary Science

EVA astronaut 2

ERA Brahms VM

ERA agent

Comm. agent

ERA

MDRS Crew

SpaceSuit_2 Brahms VM

SpaceSuit_1 agent

Astro_1 agent

EVA astronaut 1

SpaceSuit_1 Brahms VM

SpaceSuit_1 agent

Astro_1 agent

SUNY BuffaloRST

Mars SocietyRST

AZ

CA

RST Facilitator

UK/NY

Compendium

MeetingReplay Hab Brahms VM

CapCom agent

ScienceOrganizer

agent

CompendiumAagent

ScienceOrganizerAagent

ScienceOrganizerCompendium

Maarten Sierhuis, RIACS/NASA Ames

EVA Data in Science Organizer (WkStn9)

From EVA to Desktop

From: capcom@agentisolutions.comSubject: MDRS New ImageCollection:

image_collection6Date: April 15, 2005 5:36:55 PM PDTTo: mdrs_rst@agentisolutions.comCc: mdrs_crew@agentisolutions.com,

mdrs_ops@agentisolutions.com

New ImageCollection: image_collection6

EVA Plan: Day_5_Astro_EVA_2_To_Beyond_Poohs_Corner_PlanActivity: WorkInRavine

Creator: AstroOneObject ID: AstroOneModel_IMAGE_COLLECTION_3

TimeStamp: 04/16/2005 01:08:07Latitude: 3824.201417 NORTHLongitude: 11046.8688492 WEST

Link for RST: https://marst.arc.nasa.gov/mdrs2005/183971. . .

Science Organizer Data Map

Generated During EVA

Dan Berrios, NASA Ames

MDRS IT Network

OpsConWin2003 Server

HabObs

LCDDisplay

MonitorDisplay

WLANAP900+

DirectWay56 Kbps

HabcomLaptop

MEX-NETWLAN Root

Printer

ERA Workstations

MDRS Top FloorNetwork

ShorelinePhone

Processor

MEX WorkStations

LCDDisplay

LCDDisplay

Hub

TP 390

MDRS LowerNetwork

Hub

MDRS WLANDHCP

SW

NREN Satellite Link Network

Phone Lines

HubMA Workstations

Hub

ScienceOrganizer

ERAWorkStations

MEX-Hab(g)WLAN Root

Ext HabWired Network

Rick Alena, NASA Ames,March 2005

ISP/IT NETWORK(TAZ to MDRS internal hubs)

MEX Net(hubs to Tropos & EVA Computers)

KaOS Registry(agent locator)

Mobile Agents ComputersMobile Agents Computers(AstroPacks, ERAs, HabCom)(AstroPacks, ERAs, HabCom)

Agents(Personal, Loc, Nav, Plan, Comm)

Peripheral Devices(dGPS, biosens, cameras, robots, db, email)

Mobile Agents Systems Architecture

Experiment Observations(Day 5 EVA 2 Beyond Pooh’s Corner)

• Enables independent, simultaneous work, which geologists prefer

• But agents talk over astro conversations– Astro cannot hear that partner

is listening to agent– Astro interleaves response with

conversation– Results in “What did you say?”

• Ready for next level:Voice loop management, alternative feedback, protocol personalization

Overall: System Responsive and

processed data appropriately

Example Good Results

Where is Boudreaux?

Boudreaux, stop.

How much time is left?

What is the current activity?

(Astro2) Where is Astronaut 1?

Where am I?

Warning: Time exceeded.

Sorry: Can’t do that.

Warning: Battery is low.

Some Not So Good Results

Misunderstood “okay” as volume change command.

Boudreaux is 1.1794857 E 7 meters North-East-East from you.

Thibodeaux, join me (OK)

Astro2 remarks to partner that join command worked; Astro 1 is listening to his agent and didn’t understand; Astro2 repeats what she said, but agent hears this as the command itself!

DesertRATS (Sept 05)

• Ames, JSC, Glenn @ Meteor Crater area

• Scout Rover = ERA• 2 Pressurized Suits &

Helmet-mounted display• Power, pressure, coolant

systems data• Standard commands:

Move, Go To, Photo, Print label, Where is…?

• EVA plan monitored

Scout Rover: Mobile Agents allows

voice commanding

For more information…

• http://www.marssociety.org/MDRS/fs04– “Mission Info”– Daily reports– Photographs

• http://bill.clancey.name– 1st Space Exploration Conf. – Simulating Activities– Field Science Ethnography– Automating CapCom– Roles for Agent Assistants in

Field Science

• Brahms: www.agentisolutions.com

Special thanks to Abby Semple and Brent Garry