the real-time web in the age of agents
TRANSCRIPT
The Real-Time Web in the Age of Agents
2011 Semantic Technology ConferenceJune 8th, 2011
Joshua Shinavier
• real-time Semantic Web is emerging
• state of the art
• real-time interaction?
• Semantic Web agents
• use cases
• the RDFAgents specification
• queries and subscriptions
• data model
• provenance and information discovery
Overview
2
• let’s mash up the “real-time” social Webwith the Semantic Web
• data sources are plentiful
• e.g. Twitter, Facebook, Glue, ActivityStreams
• data integration is straightforward
• mapping into Semantic Web vocabularies
• linking into Semantic Web datasets
• what about interaction?
3
Real-time interactionwith the Semantic Web
• ...not “real-time”
- static RDF data, occasionally updated
• ...not ubiquitous
- desktop model (browsers, keyword search)
- client-server model
• ...not very interactive
- read-only content, monolithic triple stores
- little “bottom-up” user content
Semantic Web “1.0” was
4
Getting closer to “real-time”
5
hours
minutes
seconds
milliseconds
days
static triple stores
web pages
blogs
microblogs
instant messaging
streaming media
latency technology misc. examples
sparqlPuSH
C-SPARQL
TwitLogic
• mobile Semantic Web
- e.g. RDF On the Go, Android Semantic Web Core library, DBpedia/mSpace/OntoWiki Mobile
• semantic sensor networks
• semantic publishing and presence
- e.g. SMOB, Sharing Spaces
• smart spaces, dialogue-based Semantic Web UI
More ubiquitous, more interactive
6
• real-time Semantic Web is growing organically
• common themes:
- rapidly changing data
- two-way, peer-to-peer communication
- participation of “ubiquitous” devices large and small
• we need shared frameworks for real-time interaction
The 10,000-foot view
7
Enter Semantic Web agents
8
“The real power of the Semantic Web will be realized when people create many programs that collect Web content from diverse sources, process the information and exchange the results with other programs.The effectiveness of such software agents will increase exponentially as more machine-readable Web content and services becomes available.”
— Tim Berners-Lee et al., 2001
• peer-to-peer communication
• proactive, event-driven information sharing
• conventions for provenance tracking
• support for a variety of interaction protocols
- e.g. for query answering, subscriptions, contracts
By “agent”, we mean:
9
( )not that kind of agent.
• Agent A → Agent B: “Who is Josh?”
• Agent B → Agent A: “Josh is a presenter at SemTech.”
Use cases: basic query answering
10
• Agent A → Agent B: “Who is Josh?”
- Agent B → Agent C, Agent D: “Who is Josh?”
- Agent C → Agent B: “Josh is a presenter at SemTech.”
- Agent D → Agent B: “Josh has the Twitter handle @joshsh and has written these tweets: ...”
• Agent B → Agent A: “Agent C says that Josh is a presenter at SemTech. Agent D says that Josh has the Twitter handle @joshsh and has written these tweets: ...”
Use cases: query delegation with provenance
11
• Agent A → Agent B: “Keep me up to date about SemTech.”
• Agent B → Agent A: “Will do!”
• Agent B → Agent A: “Josh has just written this tweet about SemTech: ...”
• Agent B → Agent A: “Craig has just posted this review of SemTech: ...”
Use cases: real-time data streams
12
• Agent A → Agent B: “Keep me up to date about SemTech.”
• Agent B → Agent A: “Will do!”
- Agent B → Agent C: “Keep me up to date about SemTech.”
- Agent C → Agent B: “Will do!”
- Agent C → Agent B: “Josh has just written this tweet about SemTech: ...”
• Agent B → Agent A: “Agent C says that Josh has just written this tweet about SemTech: ...”
Use cases:syndication with provenance
13
• human limit of real-time ≈ 10 - 100ms
- these are simultaneity thresholds, central to human sensory integration
• technical limits of real-time are similar
- 134ms for a round-trip around the Earth (ideally)
- 28ms from New York to San Francisco and back
- actual Internet latency is 2 to 3 times higher, but
• TCP’s three-way handshake is a problem
- we need long-lived connections and bidirectional data flow, as in XMPP (Jabber)
Why agents? Extreme real-time
14
• FIPA (Foundation for Intelligent Physical Agents)
• most widely-adopted body of agent standards
• supports peer-to-peer, asynchronous, and event-driven communication
• transport agnostic
- compatible with HTTP, XMPP, etc.
• easily coupled with Web APIs, Semantic Web data formats
Why agents? The standards
15
• RDFAgents specification
• extends FIPA
• enables query answering and real-time data streams with provenance
• includes:
• RDF content languages
• two interaction protocols: Query and Pub-Sub
• support for trust, privacy, information accountability
• conventions for peer-to-peer information discovery
RDFAgents = + data streams
16
Queries
17
Subscriptions
18
Example: a message
19
(inform-ref :sender (agent-identifier :name http://example.org/twitlogic! :addresses (sequence xmpp:[email protected])) :receiver (set (agent-identifier :name http://example.org/consumer! :addresses (sequence xmpp:[email protected]))) :protocol fipa-subscribe :conversation-id c976b710a5 :language rdf-trig :content "@prefix dc: <http://purl.org/dc/terms/> .@prefix ex: <http://example.org/> .@prefix sioc: <http://rdfs.org/sioc/ns#> .@prefix sioct: <http://rdfs.org/sioc/types#> .@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
{ ex:post1785782813 a sioct:MicroblogPost ; dc:title \"#SemTech begins again!\" ; dc:created \"2011-06-06T19:43:35.000Z\"^^xsd:dateTime ; sioc:topic <http://twitlogic.fortytwo.net/hashtag/semtech> .}")
RDF dataset
sender and receiver
“conversation” metadata
message type
• agents exchange RDF Datasets, in the sense of SPARQL
- any number of Named Graphs
- may be annotated with provenance metadata
- a single default graph
- contains asserted (accepted) statements
• agents identify themselves with Semantic Web URIs
- agent contact information in FOAF
• Semantic Web Publishing vocabulary captures the provenance trail
RDFAgents data model
20
21
<urn:uuid:be0c72c6-2b8f-4134-b309-690039f8c419> { ex:post1785782813 a sioct:MicroblogPost ; dc:title "#SemTech begins again!" ; dc:created "2011-06-06T19:43:35.000Z"^^xsd:dateTime ; sioc:topic <http://twitlogic.fortytwo.net/hashtag/semtech> .}
{ <urn:uuid:be0c72c6-2b8f-4134-b309-690039f8c419> a rdfg:Graph ; swp:assertedBy <urn:uuid:be0c72c6-2b8f-4134-b309-690039f8c419> ; swp:authority ex:twitlogic .
ex:twitlogic a foaf:Agent ; foaf:mbox <xmpp:[email protected]> .}
agent metadata
a named graph
the default graph (accepted statements)
graph metadata
generic content
Example: a dataset from a message
• keeping track of who said what
- publishers deserve credit for their content
- consumers need to decide what to believe
• RDFAgents specifies a provenance-preserving transformation for each dataset received
- records “A said that B said X” relationships between agents and graphs
• simple policies unravel the provenance trail to accept or reject RDF statements
• provenance metadata also supports information discovery
The provenance trail
22
• make real-time streams out of social data sources
- e.g. Twitter agents, ActivityStreams agents, etc.
• combine streams from different sources
• integrate with Web services, Linked Data and SPARQL endpoints
• attach SPARQL-based filters for smart content feeds
• pipe the data into a triple store, publish it as Linked Data, power your real-time applications
What can you do with RDFAgents?
23
• implemented in Java with JADE and Sesame
- LEAP kernel for mobile devices
- all of the usual RDF formats incl. TriG, N-Quads
- transport protocols: XMPP, HTTP, IIOP
• open-source
• in progress: high-throughput RDF data streams with AllegroGraph
• in progress: Droidspeak library for Android OS
RDFAgents implementation
24
( )
• Semantic Web is moving into real-time and ubiquitous environments, but
• common interaction models are needed
• Semantic Web agents are particularly appropriate
• RDFAgents = Semantic Web + FIPA for transport-agnostic, peer-to-peer queries and data streams with provenance
25
Conclusion
• RDFAgents:
• http://fortytwo.net/2011/rdfagents/spec
• https://github.com/joshsh/rdfagents
• Tetherless World Constellation: http://tw.rpi.edu
• Franz Inc: http://www.franz.com
• Institute of Automation: http://english.ia.cas.cn
• TinkerPop: http://tinkerpop.com
• Contact:
• [email protected], @joshsh
Thanks!
26