l c sl c s supporting technology for group interaction howard shrobe mit ai lab oxygen workshop,...
TRANSCRIPT
L C S
Supporting Technology for Group Interaction
Howard Shrobe
MIT AI Lab
Oxygen Workshop, January, 2002
L C SA Three Pronged Attack on Collaborative Knowledge Management
• Ubiquitous human-centered, perceptually enabled environments.
• An adaptive infrastructure that understands the context and content of problem solving discourse (at least a little)– It accurately indexes all information.
– It distributes information to those who can use it.
– It can find resources (information and people) to help a collaboration.
• Tools informed by an understanding of the domain and the organization.– An “almost expert” (apprentice) system
* Which knows when to ask questions:
* What it doesn’t understand is often what’s most important
* What isn’t obvious is what needs to be documented and distributed
• Documentation almost for free by embedding a helpful computational system in our normal work flow
L C SAn Example of Group Interaction and Shared Plans
ClarifyInterface
Propose
Code
Document
Brainstorm
Critique
The side-tracker interface would be much clearer if it worked this way!
I think I can make it work that way, but does it affect anything else!
That should work, as long as it doesn’t break the special hack for managers.
I’ll look into that, you all can start the coding and doc updates
Plan
L C S
Plan
KB
GoalPlan
KnowledgeBase
Haystacks
Knowledgeacquisition
Facilitator
SharedPlans
SharedInfo
Shared Semantic Web
Inference
ResourceDiscovery
Facilitator Agent
Knowledge-Based Infrastructure Helps People Collaborate:
• By acting as an assistant to meeting facilitators it can help to capture important information such as:– Issue, positions, arguments for and against positions– Commitments, action items– Video and audio transcripts of meetings
• By understanding the role of individuals within the organization it can help to decide who should see what information.
• By understanding the technical capabilities of individuals it can help to match “who can do what” to “what is required”.
L C S
• Information nodes:• Goals of the project• Proposed methods for the goals• Arguments in favor and against methods• Documents supporting these arguments
• Node format:• Much of the content is opaque to the system
(e.g. Multi-media fragments)• Some slots are understood by the system• And, short natural language annotations can be
attached, parsed and understood by START• Formal representations are also possible
• Relational information:• Links showing the relationships between the
information nodes• Link types are meaningful to the system
• Organizational descriptions: • Resource descriptions including capabilities,
roles and interests (for personnel)• Process goals and plans
Meeting Manager
Plan
KB
GoalPlan
KnowledgeBase
Haystacks
Knowledgeacquisition
Facilitator
SharedPlans
SharedInfo
Shared Semantic Web
Inference
ResourceDiscovery
email web
The Oxygen Collaboration Infrastructure:Knowledge Based Collaboration Webs
L C S
Oxygen Can Support Human Collaboration by Making Simple Inferences
• Oxygen deduces new information using:– The background knowledge base– The types of the links– The slots of node structures that are understood– The semantic content of the node annotations– Descriptions of the resources and people in the organization
• Oxygen routes information and discussion topics to those whose:– Organizational role requires it– Interests suggest it– Capabilities and skills might be useful
• Oxygen posts new goals and initiates new discussion processes to address these goals.
• Oxygen supports human interactions in a manner relevant to the organization’s problem solving context
L C SKey Oxygen Components Make This Much Easier by Bringing Computers Into Our World
Microphone Array
Tracking Cameras
Video Displays
Pointing Camera
Pointing Camera
L C SA Meeting Web
Video/Audio Transcript (quicktime movie)
Position 1Start time;End time:
Position 2Start time;End time:
Discourse Structure
Issue 1Start time;End time:
Issue 2Start time;End time:
Supporting Argument
Start time;End time:
Refuting Argument
Start time;End time:
Agenda ItemStart time;End time:
Agenda ItemStart time;End time:
Agenda ItemStart time;End time:
Meeting Structure
CommitmentStart time;End time:Who:DeadlineDiscourse
CommitmentStart time;End time:Who:DeadlineDiscourse
Action Items
CommitmentStart time;End time:Who:DeadlineDiscourse
People
L C S5 Keys Challenges for Adaptive Interfaces
• Providing a practical level of knowledge representation that enables groups interactions and grounding in the real world of space and time
• Providing services in a multi-user environment while making optimal use of the currently available resources
• Recovering from equipment failures, information attacks, etc.
• Coordinating and fusing information from many sensors and modalities
• Capitalizing and recognizing context
• Maintaining Security and Privacy and trading these off against other goals
L C S
Challenge 1: Grounding in Real-World Semantics
• We want to build applications that service many individuals and groups of individuals
• These people will move among many physical spaces
• The devices and resources they use change as time progresses
• The context shifts during interactions
• The relevant information base evolves as well.
L C S
Research Agenda: Knowledge Representations
• People– Interests, skills, responsibilities, organizational role
• Organizations– Members, structure
• Spaces– Location– Subspaces– Devices and resources
• Resources• Information nodes
– Topic area, place in ontology, format• Services
– Methods, parameter bindings, resource requirements• Agents
– Capabilities, society, acting on behalf of whom• Events
– People identification, motion in new space, gestures
L C S
Challenge 2: Adaptive Resource Management
• In most systems, applications are written in terms of specific resources – (e.G. The left projector in michael’s office, or worse yet physical
address).
• This is in conflict with– Portability across physical contexts
– Changes in equipment availability across time
– Multiple applications demanding similar resources
– Need to take advantage of new resources
– Need to integrate mobile devices as they migrate into a space
– Need to link two or more spaces
• What is required is a more abstract approach to resources in which no application needs to be tied to a specific device.
L C S
AbstractService
ServiceControl Parameters
User’sUtility
Function
The binding of parameters has a value to the user
Resource1,1
Resource1,2
Resource1,j
Each method requires different resources
The system selects the method which maximizes net benefit
User requests A
service with certain parameters
ResourceCost
Function
The resourcesUsed by the methodHave a cost
Net benefit
Each method binds the settings of The control parameters in a different way
Method1
Method2
Methodn
Each service can beProvided by several
Methods
The system adapts by having many plans for each service
L C S
Security and Privacy Issues Are Addressed by Factoring in the Cost of Violating a Security or Privacy Policy
ResourceCost
Function
Resource1,1
Resource1,2
Resource1,j
AbstractService
User’sUtility
Function
Net Benefit
Method1
Method2
Methodn
Security PoliciesSecurity Policies
Challenge 6: Security and PrivacyIs Addressed by This Infrastructure
L C S
Challenge 3: Robustness and Recovery From Failures
• The intelligent room renders services by translating them into plans involving physical resources– Physical resources have known failure modes
• Each plan step accomplishes sub-goals needed by succeeding steps– Each sub-goal has some way of monitoring whether it has been
accomplished
– These monitoring steps are also inserted into the plan
• If a sub-goal fails to be accomplished, model-based diagnosis isolates and characterizes the failure
• A recovery is chosen based on the diagnosis– It might be as simple as “try it again”, we had a network glitch
– It might be “try it again, but with a different selection of resources”
– It might be as complex as “clean up and try a different plan”
L C S
I need to ask a question of a systems
wizard
Plan 1: • Locate A systems wizard in the E21
• Monitor: check that person is still there• Turn on the selected projector
• Monitor: check that projector turned on• Project the message• Done
• Monitor: check that the person noticed the message
I don’t see light on the screen
I see sally by the screen
Projector-1 must be broken.We’ll try again, but using
Projector-3.
PlanBreakdown
Recovering From Failures
L C S
Frobulate the sidetracker
White BoardPlace Manager
Sally is near The Whiteboard
If A Person approaches A device ?x and Grammar ?y is relevant to ?xThen Activate Grammar ?y
Activate the Drawing Grammar
The Drawing GrammarIs Relevant to the Whiteboard
Challenge 4: Exploitation of Context
• The attention of the system should be focused by what people in the facility do and say.
• Speech recognition should be biased in favor of things going on in the facility– Speech system is made up of many “ grammar fragments”
– Grammar fragments are activated (and deactivated) when perceptual events (visual or speech) suggest they should be
• Visual Interpretation should also be biased by context
L C SChallenge 5: Perceptual Coordination
• Separate the implementation of perceptual tasks from the uses to which perception is put
• All perceptual modules advertise the class of “behavioral events” they are capable of recognizing and signaling– These events are organized into a taxonomy– Some events are “ close to the physics” (e.g. an object was
observed at location <x,y>)– Some events are more abstract (e.g. a person is near the white-
board)– The same event can be signaled by quite different perceptual
modules (e.g. both face and voice recognition can localize a person).
• Other modules register their interest in certain classes of events– Requests at a higher level in the taxonomy subsume lower level
events
• Modules which receive low-level events may register for and collate many different classes of events – They combine these and signal higher-level events
L C SResearch Agenda: A Dynamic Event Bus For Perceptual Coordination
VisualTracke
r
Signal body motion
VoiceIdentification
Interested in body motionSignal the location of individuals
FaceRecognition
Interested in face locationI signal the location of individuals
White BoardContext Manager
Interested in the location of individualsSignal people approaching the whiteboard
FaceSpotter
Interested in body motionI signal face location
L C S
Plan
KB
GoalPlan
KnowledgeBase
Haystacks
Knowledgeacquisition
Facilitator
SharedPlans
SharedInfo
Shared Semantic Web
Inference
ResourceDiscovery
Facilitator Agent
Oxygen’s Knowledge-Based Infrastructure Helps People Collaborate:
• By acting as an assistant to meeting facilitators it can help to capture important information such as:– Issue, positions, arguments for and against positions– Commitments, action items– Video and audio transcripts of meetings
• By understanding the role of individuals within the organization it can help to decide who should see what information.
• By understanding the technical capabilities of individuals it can help to match “who can do what” to “what is required”.
• By understanding the context within which people act, it can react in appropriate ways.