project ‘aire’ and semantic networks seminar ‘multimodal rooms’ jochen von spiczak
TRANSCRIPT
Project ‘aire’and Semantic Networks
Seminar ‘Multimodal Rooms’
Jochen von Spiczak
slide 2 / 40
content:
overview
- project ‘aire’· overview / interesting examples
- semantic networks· introduction
· knowledge representation in IEs
· description of SNs,especially in intelligent rooms
· realization of SNs
· conclusion / future work
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 3 / 40
content:
project ‘aire’
- ‘aire’ ~· Agent-based Intelligent Reactive
Environments
- MIT Artificial Intelligence Lab
- examining perceptual computing systems and applications
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 4 / 40
content:
overview
- tools for collaboration support:· supporting interactions among people,
spaces and mobile devices
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support
slide 5 / 40
content:
overview
- software infrastructure· communication
· resource management
· device control
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure
slide 6 / 40
content:
overview
- human-computer interfaces· new interfaces for people to interact
with their environments
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure HCIHCI
slide 7 / 40
content:
overview
- ‘aire’ spaces· large range of form factors and sizes
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure HCIHCI
conference rooms,personal workspaces,transitional spaces
conference rooms,personal workspaces,transitional spaces
slide 8 / 40
content:
examplesoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure HCIHCI
conference room,personal workspace,transitional spaces
conference room,personal workspace,transitional spaces
slide 9 / 40
content:
examplesoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure HCIHCI
- ‘Metaglue’· Java-based agent infrastructure
· providing communication and levels of abstraction
· base of the rest of the project
- ‘Metaglue’· Java-based agent infrastructure
· providing communication and levels of abstraction
· base of the rest of the project
slide 10 / 40
content:
examplesoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure HCIHCI
slide 11 / 40
content:
examples
- visit this very day:
http://www.ai.mit.edu/projects/aire
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
project ‘aire’project ‘aire’
coll. supportcoll. support infrastructureinfrastructure HCIHCI
slide 12 / 40
content:
semantic networks
- one outstanding aspect of the ‘aire’ project
- knowledge representation as a fundamental design question· semantic networks as such a
representation
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 13 / 40
content:
introduction
- ever-increasing amounts of data about users· who they communicate with,
· their location,
· visited web pages, ...
- aim: · quick, correct assumptions about what
the user would like to do next
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 14 / 40
content:
knowledge representation in IE
- five different roles that KRs play· surrogates for the real world
· ontological commitments
· theories of reasoning
· media for computation
· media for human expression
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
· » KR imposes a filter on what the
environment can see and act on
» example of camera-/microphone-signal
· » easily/less easily identifiable entities
» important for reasoning about items and making deductions based on their layout
· » KR defines how inferences are made by the
reasoning system
» constraints kinds of inferences which are allowed or recommended by the system
· » KR must be able to operate in an efficient
manner via computational processes
» without taking up inordinate quantities of time, memory, or disk space
· » KR represents the human view of the world
» must be simple to use
» turn into human-readable representations
slide 15 / 40
content:
knowledge representation in IE
- much literature about developing contextual systems for IE
- problems of other approaches· interpretation and inferencing is done
by the application
· application must collate and assemble disparate pieces of data
· no cohesive representation of the gained knowledge
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 16 / 40
content:
SN in general
- need for a representation that…· can store information on objects
· can easily track and follow the relationships between them
- introducing semantic networks as one such representation
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 17 / 40
content:
SN in general
- concept of semantic networks· information represented in a directed
graph:» nodes objects
» links relationships
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 18 / 40
content:
SN in general
- variety of node and link types· nodes just like…
» people,
» meetings,
» documents,
» issues,
» discourse items,
» commitments,…
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 19 / 40
content:
SN in general
- variety of node and link types· links just like…
» relationship between people,
» responsibilities for objects (groups/people),
» hierarchies between contributors,
» attendance to meetings,
» interest for objects, …
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 20 / 40
content:
SN in general
- further link types:· endpoints for links not limited to
storage nodes
· can also be created to other links
- allows more complex interactions in the data model
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 21 / 40
content:
SN in general
- easy to discover…· information about a particular object
· relationships between different objects
- little information stored in nodes· network relies more on the links to
define the relationships
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 22 / 40
content:
SN in general
- some inherent advantages· adding data is highly straightforward
· changing data is highly localized
· making inferences is fast and easy
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 23 / 40
content:
SN in general
- further advantages· great flexibility in terms of the data
which is captured and interpreted
· extremely efficient
· easy to bring into human-readable form
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 24 / 40
content:
SN in intelligent rooms
- several areas for using a SN-based representation· user knowledge
· meeting management
· location management
- SN as an appropriate and valuable piece of infrastructure
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 25 / 40
content:
SN in intelligent rooms
- user knowledge· knowledge of the users and
the individual spaces they work with
· simplistic level» simple set of objects, spaces and
the user’s current location
» queries like “Where is Steve located?”
· system as an “intelligent assistant”» far more information and relations must be
included
» more complex queries like“Who is Joe’s superior?”
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 26 / 40
content:
SN in intelligent rooms
- meeting management· capturing meetings as the occur
· typical information» agenda topics,
» action items,
» supporting and dissenting arguments,
» documents such as presentations orweb references, …
· can be linked to a video or audio capture of the meeting
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 27 / 40
content:
SN in intelligent rooms
- location management· location information important for IEs
· especially true when integratinghand-held devices
» require knowledge of the current location
» own coordinate framework of each system
· SN as a location infrastructure» integration and translation between
different coordinate systems
» specify own location descriptions,such as “at home”
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 28 / 40
content:
realization of SNs
- technology of SNs deployed within the project ‘aire’
- current implementation· works on top of a SQL database
· easily extendable to use…» object-oriented databases
» resource description framework (RDF)back-ends
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 29 / 40
content:
realization of SNs
- access to the semantic network· using the Metaglue agent
infrastructure of project ‘aire’
· several agents used to access the SN» MeetingModel agent
» GUI agent
» other viewing/editing agents
· network query language
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 30 / 40
content:
realization of SNs
- semantic network itself· provides only basic interface methods
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
semantic networksemantic network
slide 31 / 40
content:
realization of SNs
- MeetingModel agent· serves merely as an interface to SN
· abstracts out some of the lookup tasks
· broadcast of any changes
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
semantic networksemantic network
MeetingModelMeetingModel
slide 32 / 40
content:
realization of SNs
- network query language· allows user or agents to specify
queries in a formal language
· definition of rules in a syntax similar to Prolog
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
semantic networksemantic network
MeetingModelMeetingModel query languagequery language
slide 33 / 40
content:
realization of SNs
- GUI agent· tree-structure view of stored
information
· creation/editing of information nodes
· currently the primary conduit for information to get into the SN
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
semantic networksemantic network
GUI agentGUI agent
MeetingModelMeetingModel query languagequery language
slide 34 / 40
content:
realization of SNsoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
slide 35 / 40
content:
realization of SNs
- other viewing/editing agents· providing different views of the data:
» web interfaces,
» graph networks,
» specialized meeting views which simply present the agenda or commitments, …
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
semantic networksemantic network
GUI agentGUI agent
MeetingModelMeetingModel query languagequery language
……
slide 36 / 40
content:
example of a SNoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
Stiefelhagen
McDonough
Rogina
Jochen
rest
face tracking
expert-in
speech recognition
expert-in
Mercedes SLK
expert-in
multimodal rooms
responsible-for
responsible-for
responsible-for
assisted
seminar talk
currently-doingbores-to-death
belongs-to
room 225
works-in
slide 37 / 40
content:
example of a SNoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
Stiefelhagen
Jochen
rest
face tracking
expert-in
Mercedes SLK
responsible-for
McDonough
speech recognition
expert-in
multimodal rooms
responsible-for
responsible-for
assisted
seminar talk
currently-doing
belongs-to
Rogina
room 225
works-in
McDonough
speech recognition
expert-in
multimodal rooms
responsible-for
Rogina
room 225
works-in
- “John McDonough.”- “In room 225.”
expert-in
bores-to-death
slide 38 / 40
content:
example of a SN
Stiefelhagen
Jochen
rest
face tracking
expert-in
Mercedes SLK
responsible-for
McDonough
speech recognition
expert-in
multimodal rooms
responsible-for
responsible-for
assisted
seminar talk
currently-doing
belongs-to
Rogina
room 225
works-in
expert-in
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
bores-to-death
slide 39 / 40
content:
conclusion / future work
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
- seems to be the most appropriate knowledge representation for IEs· flexibility of data
· adding/changing data highly localized
· inference generation extremely efficient
- future work· integration within other projects
· automatic sensor-based information recognizing
· incorporate user preference information
slide 40 / 40
content:
summaryoverview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
- project ‘aire’
- semantic networks· one approach to KR
· inherent advantages in general…
· … and in the context of IEs
· realization» implementation
» access hierarchy
· example
· most appropriate approach to KR
summary
- project ‘aire’
- semantic networks· one approach to KR
· inherent advantages in general…
· … and in the context of IEs
· realization» implementation
» access hierarchy
· example
· most appropriate approach to KRslide 40 / 40
content:
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge representation in IE
- SN in general
- SN in intelligent rooms
- realization of SN
- example of a SN
- conclusion /future work
summary
discussion
…?