haystack, mit end-user application development for the semantic web karun bakshi [email protected]...
TRANSCRIPT
Haystack, MIT
End-User Application Development for the Semantic
Web
Karun Bakshi
Computer Science and Artificial Intelligence Laboratory (CSAIL), MIT
Haystack, MIT
Overview
• Tools for performing information based tasks are lacking.
• We need to fix these problems if we are to fully leverage all the information at our disposal.
• I propose an approach to fix these problems and I think the Semantic Web can help.
• In the process we demonstrate– An application of the Semantic Web– Applications for the Semantic Web
Haystack, MIT
Agenda
• Introduction• Related Work• Approach• Demonstration• Implementation• Discussion• Evaluation• Conclusion
Haystack, MIT
Introduction
Haystack, MIT
Information Based Tasks
• Information Based Tasks– Require multiple sets of information and functionality
– Can reuse and visualize the same information and functionality from other tasks
• Use Cases– Military Commander
– Doctor
– Software Project Manager
Haystack, MIT
A Day in the Life of a Software Project Manager
• Receive E-mail Bug Report from Customer• Check Task Assignments• Log Bug in Bug Tracking Software (automatic e-
mail notification)• Schedule meeting via e-mail and update calendar• Update Project Schedule and Task Assignments• Update Project Budget
Haystack, MIT
How Do We Accomplish Such Tasks Today?
• Users Must– Dig deep and wide to find relevant information and
functionality– Mentally collate information– Re-enter Information Data Synchronization? – Lather, Rinse, Repeat
• A solution that allows aggregating relevant resources would– Increase efficiency– Be mission critical
Haystack, MIT
Status Quo: A Potential Solution?
• Build an uber-Application or build a task-focused application. But…– How do I reuse the calendar or e-mail for
another purpose?– What if I need to track additional information?– What if I do a certain new task more often?– How does another user tackle the same
problem?– How do we handle feature creep?
Haystack, MIT
Source of Problem• Tasks are Fluid. Information and functionality
depends on:– User
• Expertise• Preferences• Task Definition
– Task• Evolve over time• New unanticipated tasks appear• New information is available that can be incorporated
• Applications are static!
Haystack, MIT
Problem Definition
• How do we provide users with an interface supportive of their tasks while avoiding the problems engendered by application shortcomings?
• How Important is this Problem?– WWW more information
– WWW more complex tasks
– WWW lay users
– What about ad hoc multi-domain applications?
Haystack, MIT
Related Work
Haystack, MIT
UI Real EstateManagement
Task Focused Applications
Workspace Builder Tools
Configure Presentation
Configure Content
Configure Operations
Configure Task Workspace
Build Task Workspace
Related Work
WinCuts
MyYahoo!
SNAP`
Data ManagementInterface
Task ManagementInterface
Con
figu
rati
onC
reat
ion
Haystack, MIT
MyYahoo! Allows Selecting
Content Layout Relevant Operations
Limited to News Domain
User can configure settings, rather than create them
Haystack, MIT
SNAP Together Visualization
Linked Content Arbitrary Content Multiple Visualizations No UI Programming
Knowledge of schema and SQL, or use fixed queries Fixed visualizations Primarily for data exploration
Haystack, MIT
WinCuts Allows collaboration Peripheral awareness Supports multiple domains
Cannot be persisted Have to go to source window to change content Pixel based content, not semantic Shareable, but not transferable
Haystack, MIT
Important Ideas
• Window Management• Persistent Habitat• Persistent Queries
– Dynamic Content
– Reusable Content
• User Creatable/Editable• Co-Location of Relevant Operations• Synchronized/Linked content
Haystack, MIT
Important Ideas (contd.)
• Little knowledge of underlying schema• Powerful query ability• Tasks require peripheral awareness of other
happenings• No Programming• Multiple visualizations• Low overhead reconfigurability• Applicable to multiple domains
Haystack, MIT
What’s Missing?
• General Purpose Interface– All above properties– Domain Interoperable (support multiple
domains ad hoc and simultaneously)– Transferable– Semantic Interface
Haystack, MIT
Approach
Haystack, MIT
Our Approach• Combine desirable characteristics in a single system• Information Management Interfaces Should Match
Tasks. • Users Know Tasks Best• Therefore Give User Control Over Building Them
– What are the application building primitives that users must control to match the fluid nature of tasks?
• What we are NOT trying to achieve:– The best implementation for particular techniques,
feature sets, UI widgets, etc.
Haystack, MIT
Our Approach (contd.)
• Unify the Data Model• Let Users Build and Populate a Task
Workspace – Formalize + Structure Existing Ideas/Features
• Content• Presentation• Manipulation
– Give users fragments of functionality to select & aggregate for unique tasks
Haystack, MIT
Semantic Web – Motivation
• Information on the World Wide Web is primarily intended for human consumption with complex semantics distributed across text, images, video, sound
• There is no way for machines to use and leverage such information easily and effectively
• Thus, very little on the WWW can be automated, and use of all the information on the WWW is constrained to the limited bandwidth of human processing
Haystack, MIT
Semantic Web – What is it?
• An initiative to embed semantics into content on the WWW to make it amenable to machine processing.
• Semantics are embedded via metadata annotations• Software (agent) understands the annotations
because they follow a particular schema/ontology• Different schemas can be used to describe the
same entity• But with metadata annotations, we begin to have a
substrate for automation
Haystack, MIT
Semantic Web – The Data Model
• Semi-structured (unenforced and/or partial schema validation) data models are very flexible – Captures multiple domains– Models the web
• Linking content from multiple domains in a growing (multi-schema) web
• No single schema to enforce
Haystack, MIT
Resource Description Framework (RDF)
• A specification/technology that captures and defines such a semi-structured data model for the Semantic Web
• Can be used to capture content from multiple domains
• Provides a common interchange format (XML based) that allows applications to communicate
Haystack, MIT
RDF Example
<URI> “Karun”
<URI>
“Max”
“Blue”
<URI>
“John”<URI>
“Blue”
Karun has a blue cat named Max, and Karun’s friend John has a blue car.
name
color
name
name
hasCat
hasCar
color
friend
URI – Universal Resource Identifier
Literal – String data
Haystack, MIT
RDF Example
Karun has a blue cat named Max, and Karun’s friend John has a blue car.
<URI> “Karun”
<URI>
“Max”
“FF”
<URI>
“John”<URI>
<URI>
name
color
name
name
owns
owns
color
friend
<URI>
type
“Cat”
name
<URI>
type
“Car”name
“00”
“00”
red
green
blue
Haystack, MIT
How Does the Semantic Web Fit In?
• A Problem?– More information– Faster generation of information– Highly granular information
• An Opportunity?– RDF models multiple domains– RDF allows granular access and annotation
Haystack, MIT
What’s Left?
• Tools that let users create a task workspace by manipulating fragments of– Content Queries (Channels) Channel Manager– Presentation Views & Layout View Designer and
Workspace Builder– Manipulation Operations Workspace Builder
• Haystack provides – such fragments at the developer level– other supportive capabilities
• Build tools to expose them to the user
Haystack, MIT
Haystack
• An general purpose information management platform
Services/Agent Framework
User Interface Framework
RDF Store
Haystack, MIT
Browsing Paradigm
Haystack Views
Recursive Rendering
Haystack, MIT
Haystack User Interaction
• Direct Manipulation– Drag & Drop– Context Menus
• Operations– UI Continuation– Currying
• Demonstration
Haystack, MIT
Demonstration
Haystack, MIT
Tool Set
• Workspace Designer
• View Designer
• Channel Manager
Haystack, MIT
Haystack, MIT
Implementation
Haystack, MIT
ImplementationWorkspaces
BuilderChannel Manager
Views Designer
Haystack
ContentX
(selecting)
X
(defining)
PresentationX
(layout)
X
(user views)
X
(developer views)
Manipulation
X
(Select Curried Operations,Dynamic
Binding)
X
(developer operations, drag and
drop, context menus)
Haystack, MIT
Implementation (contd.)
• Each Application Building Primitive has– An ontology– A set of views
• Each tool generates RDF entities according to an ontology
• User navigates to each entity and modifies it using its view(s)
• Each entity can have several views, e.g. Design View vs. Usage View
Haystack, MIT
Implementation (contd.)
• Store views, operation, content description and workspace as metadata along with application data in single format
Haystack, MIT
Workspace DesignerDesign Decisions
? Space Allocation to Winlets? Modality in Workspace Interaction? Uniform Treatment of Views? Dynamic Binding and Currying
Haystack, MIT
View Designer Pattern
• Developer Created Designer
• Exposes domain specific capabilities to create/customize views
• Named view that can be reused in any context in Haystack, just like any other view
Haystack, MIT
Property Lens View Designer
Haystack, MIT
Property Lens View Designer (contd.)
• Captures – RDF semantics only (property name and value)
• Resource View• Literal Read-only/Editable
– Layout
• Has design and usage views• Design Intentionally Abstract
– Multi-valued properties are supported– Provides baseline view extensibility without developer
support or additional view designers
Haystack, MIT
(Information) Channels
• What is a channel?– Query
– Dynamic set of items (no ordering) URIs only
– Organizing mechanism for dynamic corpora
• What are the implementation components?– Channel Agent
– Query Primitives• Query
• View of Query Primitive
Haystack, MIT
Channel Query Primitives
<Set of URIs> <Query Primitive Arguments>queryPrimitive
<URI1><URI2><URI3>
…
RDF Query Primitives Operators
Domain Specific Query
PrimitivesProperty Constraints
Haystack, MIT
Channel Query Primitives
RDF Query PrimitiveOperator
Domain Specific Query Primitive
Haystack, MIT
Discussion
Haystack, MIT
Discussion
• Transferable and Personalizable• Capture process in an interface• Low overhead task switching• Extensible by developers• Seamless integration of user/developer extensions• Consistent interaction modality• Domain specific view designers• Benefits of Channels• How much work is this?
Haystack, MIT
Evaluation
Haystack, MIT
Evaluation
• By Example– Our tools/Specialized tools– Paper Writing Workspace bridged multiple
domains– Project Management Workspace
• Preliminary End-User evaluation with Massachusetts General Hospital researchers
Haystack, MIT
Evaluation (contd.)
• Future User Study– Is such “just in time” information management
desirable?– Can users succeed with such an task interface
paradigm (not whether the particular interface implementation is usable)?
Haystack, MIT
Conclusion
Haystack, MIT
Conclusion
• We advocated user specified task workspaces– We identified three primary aspects of tasks
a simple ontology for building applications– We provide tools that allow composing
fragments of functionality into task workspaces• Content Channels• Presentation Views• Manipulation Operations
Haystack, MIT
Conclusion (contd.)
• We demonstrated the usefulness and validity of the ideas (although a field study is currently lacking)
• We have proposed a solution to the general problem of information management that will be even more critical for the Semantic Web– Tools to take advantage of Semantic Web content
immediately– Tools to build applications for the Semantic Web
• We have proposed an application of the Semantic Web
Haystack, MIT
Future Work
• Better UI for Tools
• Other enabling components for Semantic Web– Ontology Translation– View Servers
Haystack, MIT
Acknowledgements
• Project Oxygen and Biomedical Informatics Research Network (www.nbirn.net) for funding
• David Karger for Advising
• Various audiences and friends for helpful feedback on this work and presentation
Haystack, MIT
Q&A