(more) transparency transformation
DESCRIPTION
Presentation given Spring/Summer/Fall 09 various venuesTRANSCRIPT
transparency transformation
solution architecture tools and techniques
george thomas
#gtra-sym09
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
Web Oriented Architecture (WOA)• REpresentational State Transfer (REST)
– The architectural style of the World Wide Web– aka Resource Oriented Architecture (ROA)
• hyperlinks dereference (information) resource representations– HTTP URI's and content negotiation
• user agent prefers .htm, .xml, .rdf, .etc
• statefulness– servers maintain resource state, clients maintain application state
• RESTful Web services– HTTP uniform interface
• CRUD analog to HTTP PUT/GET/POST/DELETE– contrast to Remote Procedure Call (RPC) style Web services
• SOAP/WSDL, you design the methods to invoke
• global visibility (the Web) and persistence (permalinks)– caching, crawling, indexing
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
XForms - human data capture• Orbeon server side XForms engine, Ajax browser GUI's
• catalog and builder apps• create new XSD bound forms• populate, persist, search• Tomcat and eXist• off-line capability• transformation pipeline
Atom Publishing Protocol (APP)• automated invocation of the RESTful Web service
– HTTP PUT/POST the spreadsheet or XML instance doc• to atomserver.codehaus.org
• where else is APP used?– Google Data API's, Microsoft Live Framework
Atom Syndication Format• transform XForm or APP captured info into XHTML+RDFa • (permalinked) public recordset in feed entry <content>
RDFa enabled 'deep link' discovery• Rich Snippets from Google
• SearchMonkey from Yahoo
the london-gazette.co.uk
london-gazette.co.uk/listing
web page = web service
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
Web as DB - Web API• Linked Open (Government) Data (LOD)
• SPARQL endpoints
linkeddata.org
linkedmdb.org – LOD example
RDF/XML from W3C Tabulator
from web of docs to web of data
querying the Web
faceted browsing
• click through on this (S-P-O) result set, use as query
federated correlation• graph based dynamic schema evolution across silos
– centralization/normalization not required (or realistic/practical!)
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
many to many set visualization
ad-hoc analyst/end-user 'meshups'
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
analytics in the wild
cloud scale analytics• Google
– GFS and MapReduce• Proprietary• page rank impl
• Apache Hadoop– HDFS and MapReduce
• open source• Amazon EMR, Cloudera
facebook.com
agenda• An overview of Web Oriented Architecture (WOA) design principles that
have made the Web the most successful distributed computing platform ever created will be given.
• Technologies for exposing raw data and publishing semantically enriched structured data for persistence and syndication on the Web as public records will be described.
• Technologies that enable interoperability across these published assets and currently disparate data sources to achieve low cost, large scale data federation will be described.
• Widgets and services that consume and transform this data for interactive and integration purposes will be discussed in the context of different stakeholder views.
• A Web-scale approach to Business Intelligence leveraging Cloud Computing approaches to data archive analysis will be described.
• Finally, the applicability of the proposed solution architecture to the Federal Segment Architecture Methodology and tools like Visualization to Understand Expenditures in IT will be discussed.
application to EA discipline• getting there from here
– stop:• publishing / analyzing / visualizing unstructured data• using structure data only in file or message exchanges
– start:• align Gov and Web architecture (including EA KB's!)• divide and conquer / publish component ontologies• linking them
– continue:• embracing emergent structure and continuous
improvement• using open source and enabling crowd-sourcing
q&a - discussion• thanks for your time and attention!
• contact me– http://xri.net/=george.thomas
– GSA OCIO Enterprise Chief Architect– FCIOC-AIC Services Subcommittee Chair– W3C eGov IG invited expert– OMG GovDTF Steering Committee– Graduate School Faculty SOA Instructor