mutual empowerment in human-agent-robot teams 16 december 2010 hart workshop jurriaan van diggelen
TRANSCRIPT
Mutual Empowerment in Human-Agent-Robot Teams
16 December 2010
HART Workshop
Jurriaan van Diggelen
Problem statement
• Achieve more with less people
• Automation can help to:– Make better use of available semi-structured
information sources– Support decision makers in dealing with the
complexity of problems (war amongst the people)
The big number cruncher
• Monolithic approach, BNC replaces existing infrastructure
• AI-complete
Sensor data
Twitter data
UAV images
Problem solution
Towards a human-machine team solution
• Solution must be provided by a human machine team
• Mutual empowerment seeks to improve team performance by:– Compensating weaknesses of humans and
machines– Optimizing strengths of humans and
machines
Types of Mutual Empowerment
human machineHMI
Intelligent Interfaces
human machine
CM
I
CC
I
HMI
User empowerment
DistributedArtificialIntelligence
CollectiveIntelligence
ME handbook
Goal
Methodology
Functional design
PrototypingValidation
•Use cases•Claims•Cognitive requirements•Ontologies•Performance measures•Tests/benchmarks
•System requirements•Functional modules•RDF interface specifications•Prototypes
•Mixed reality validation•Data collection
Tool support
Domain Exploration
•Domain•Human Factors•Technology
Situated Cognitive Engineering
• Methodology supports– Incremental design– Reuse of earlier work (Prototypes, tests,
requirements, use cases) – Collaborative development
Example
Phase 1: domain exploration
• Domain– USAR– UGV, UAV– Operators in field
• Human Factors– Maintaining situation awareness– Cognitive overload– Adaptive teams
• Technology– Collaborative tagging, crowd sourcing– Mixed initiative systems– Adaptive/ adaptable automation
Phase 2: Functional design (1)• Use cases
• Cognitive requirements
• Claims
UC 23• UAV classifies camera image as victim with certainty-level Unsure• Operator of Robot1 is notified of the potential victim and views the
camera images • Operator of Robot1 classifies the image as victim with certainty level Certain• Operator of Robot2 is notified about the victim• …
CR 5.1 Uncertainty managementOperators and agents can publish and change the certainty value of informationUse cases: UC 23
CR 5.1 • + improves situation awareness of operators and agents• - increases cognitive taskload
Phase 2: Functional design (2)
• Ontologies
• Performance measures– E.g. situation awareness measure
• Tests/benchmarks– Test for evaluating performance
something
action event item
victimrobot
Phase 3: Prototyping
• Develop system requirements that implement the cognitive requirements.
• Bundle system requirements in functional modules.
• Reuse existing base platform
Trex
Trex• Filter: which
data do you want to see? selection of semantic tags in Sparql
• Projection: How do you want to see the data?graphical object with attachment-points for semantic tags
Functional modules supported by Trex
• User configurable information filters• User configurable information visualization• Realtime semi-structured data exploration• Collective relevance assessment• Uncertainty management• Human-in-the-loop AI
P Q R S THuman Machine Crowd Machine
Human-in-the-loop AI
DEMO
Future work
• Develop functional modules for:– Joint conflict resolution– Adaptive Interruptiveness– Network awareness– Policy awareness– Capability awareness– Activity awareness
Conclusion
• Mutual Empowerment library provides a flexible way to– Increase application possibilities of AI– Employ potential of collective intelligence– Reuse and structure our knowledge of
human-machine collaboration tools
Technology InvestigationDomain Analysis Human Factors
Metrics
Cognitive Requirements
ClaimsUse cases
OntologiesTests
Exploration
Functional design
Prototyping
Core functions
Functional modules
System Requirements
PrototypeRDF interfaces
TestingSimulation Test participants Empirical results