oopsla 2007 - the web: distributed objects realized!
DESCRIPTION
Mark Baker & Stuart Charlton's tutorial on RESTful web servicesTRANSCRIPT
The Web: Distributed Objects Realized!
Mark Baker, Coactus ConsultingStuart Charlton, BEA Systems
2
About Us
Mark BakerCoactus Consulting
CTO-level consulting and large-scale distributed systems architect
Leading proponent of “REST style web services” for over seven years
Prior work at Nortel, Sun Microsystems, and several startups
Co-developed internet, mobile, and web standards through the IETF, W3C, and OMA
Mark’s Bloghttp://www.markbaker.ca/blog/
mailto:[email protected]
Stuart CharltonBEA Systems
Enterprise architect in BEA’s Consulting practice
Co-develops BEA’s global consulting strategy and service offerings
Specialized in distributed systems architecture, data warehousing, and object technology
Prior work in EA, consulting, and mentoring both the financial services & telecommunications industries
Stu Says Stuffhttp://stucharlton.com/blog/
mailto:[email protected]
3
Agenda
Part 1
How We Got Here
Understanding Distributed Object Interoperability
Exploring the Design of REST Identity and Reference Handling Evolution Multi-Organizational
Interoperability Control Flow through Hypermedia
Part 2
RESTful Web Services in Action
Exploring the Web’s Architecture
Applied Techniques for RESTful Web Services Resources-Oriented Architecture Security Approaches Transactions & Reliability
5
Where we’ve come from…
“The next shift catalyzed by the Web will be the adoption of enterprise systems based on distributed objects and IIOP (Internet Inter-ORB Protocol).
IIOP will manage the communication between the object components that power the system. Users will be pointing and clicking at objects available on IIOP-enabled servers.”
- Marc Andressen,Netscape Cofounder(October, 1996)
6
The Object Web Shall Arise!
“To move to the next step, the Web needs distributed objects. We call this next wave of Internet innovation the "Object Web.“
The HTTP form you submit is still the basic unit of client/server interaction. This clumsy work-around is not suitable for full-blown client/server applications that require highly interactive conversations between components. It also does not scale well.”
- CORBA, Java, and the Object Web BYTE Magazine (October 1997)
7
8
By 1999…. XML Web Services Shall Arise!
…Over the past decade, component technology has been the cause of many arguments, disagreements, and debates.
This component-induced friction can be traced to two primary factors:
1. “My Glue is Better than Your Glue”
2. “I Can’t Believe You Program In That Language”
- Don BoxLessons from the Component Wars: An XML Manifesto(September 1999)
9
By the mid-2000s…. what about “just” the Web?
10
Our Claims:
1. The dream of the “Object Web” is largely here: it’s the Web!
2. Constraints in a systems architecture lead to emergent properties.
3. The Representational State Transfer (REST) architectural style defines constraints which helped guide the web towards a globally scalable, interoperable, and extensible system.
4. The REST architectural style is both an evolution of past approaches to networked computing, and a radical departure.
5. More work remains to be done!
Understanding Distributed Object Interoperability
12
The Distributed Object Request Broker (ORB)
The Goals of an ORB:
Encapsulate all of the mechanisms to:Find the object for the request
Prepare the implementation to receive the request
Communicate the data making up the request
Ensure complete transparency of object location and implementation
13
Inside an Object Request Broker
Source: OMG CORBA 2.3 Architecture & Specification
14
How can a web of distributed objects span organizations?
15
Interoperability Domains
A distributed system, even if the goal is transparency, has several co-existing domains of interoperability
A domain is a set of objects sharing a common characteristic or abiding by common rules.Source: OMG CORBA 2.3 Architecture & Specification
16
Types of Interoperability Domains
Referencing The scope of a distributed object reference
Representation The scope of a message transfer syntax & protocol
Network Connectivity Potential scope of a network message
Network Addressing Potential scope of a network address
Security The extent of a particular security policy
Others…
Source: OMG CORBA 2.3 Architecture & Specification
17
Solutions to Multiple Interoperability Domains
Canonicalization
Simple and scalable, at the cost of reduced runtime performance
Low barrier to entry for new clients, high barrier for new servers
Point-to-Point Adaptation
Good runtime performance,not as scalable, increasing complexity
Low barrier to entry for new servers, high entry for new clients
CanonicalDomain
CanonicalDomain
Domain ADomain A Domain BDomain B
Domain ADomain A
Domain CDomain C Domain BDomain B
18
Solutions for a “Global Object Web”?
Goal seems to be primarily about use and reuse: more consumption Thus, lower the barriers to new clients!
Scope of Object References: Global
Scope of Representation: Global (but extensible)
Scope of Network Connectivity: Global
Scope of Security: Shared Agreement Requires at least some level of Global Canonicalization
19
Solutions for a “Global Object Web”?
Further Concerns:
Unpredictable type, usage, and volume of information Thus, shaped around extensible, streamable, cacheable, identifiable
information
Both Human and Automated actors Thus, simple, reusable, and extensible interfaces, appropriate for either
… How would you architect such a system? This is atypical of most distributed object systems architectures in practice
20
Two Very Different Types of Architectural Domains
Client or Builder-focused
Identifiable Client or Customer(s) Clearly identifiable builders Clearly identifiable users Control governance & control Overall architectural control System is a result of deliberate
value judgement by the client
Examples Most IT projects, software
products, civil structures, etc.
Collaborative Systems
No identifiable client Many builders Many (unpredictable) users No central control in terms of
development, or operation System is a result of the voluntary
choices of the participants
Examples Internet and WWW, electrical
power systems, joint military operations, etc.
Source: Eberherdt Rechtin, The Art of Systems Architecting
21
Building Emergent Capability into an Architecture
Collaborative systems are about creating emergent capability
The architecture of the system is the constraints on interfaces Components & implementations are out of scope and out of control
An example style of such constraints: Client/Server Architecture
Constraints:
Active client, Reactive server, client initiates communication Emergent properties:
Simplifies server, enables multiple organizational domains
Client Server
22
Composing Constraints into an Architectural Style
Constraints both enable and detract from emergent properties, and may be combined together
Example: Client/Server + Layered System + Caching
+ Stateless Interaction
Enables: Simplified Clients and Servers (due to statelessness & C/S) Shared Caching (due to layering) Improved Visibility & Reliability (due to statelessness)
Add in: Code on Demand (COD)
Result: Simplifies clients, Improves Extensibility, but Reduces Visibility
Source: Roy Fielding, Architectural Styles and the Design of Network-Based Software Architectures
23
The Representational State Transfer (REST) Style
Originally referred to as “The HTTP Object Model”
Source: Roy Fielding, Architectural Styles and the Design of Network-Based Software Architectures
24
Some Major Differences between REST and Distributed Objects
1. Most distributed architectures do not assume application requirements Yet, the Web is an application with goals & requirements! The overall goal: sharing information and capabilities at a global scale REST assumes the requirement is to provide an architecture of
participation for a globally networked information space
2. Most distributed architectures do not require globally scoped interoperability domains REST insists on a uniform interface to ensure global scope interoperability The uniform interface is what we explore next!
These differences apply elsewhere:SOAP Web Services, RPC, etc. are also just general toolkits
25
RESTful Distributed Objects in contrast to SOAP Web Services
Perceived advantages:
Communication occurs through XML document exchange Agreement occurs in the network protocol
The XML InfoSet is the basis of interoperability No worrying about object-oriented purity / language-oriented bindings
Disadvantages: Prefers interface descriptions for specific (non-uniform) interfaces All data must flow through the XML InfoSet, other types are secondary Nearly eliminates the notions of:
Object lifecycle (creation, removal, activation)
Object identification & references (i.e. linking) Assumes transfer protocol independence is possible and desirable
In practice, transfer protocol semantics leak into message exchanges
26
A Word on REST vs. The World Wide Web
Consider three levels of abstraction:1. Architectural Styles: Representation State Transfer (REST)
2. Architecture: The World Wide Web
3. Implementation: “User Agents, Web Servers, Proxies, etc.”
REST is a model of constraints and emergent properties
The Web is the most widely used and understood implementation
Exploring the Design of REST
28A Global, Networked, Shared Information Space
29Users Consume & Manipulate Information via Agents
30Information Objects Must Be Identifiable
31An Object’s Interface is separated from Implementation
32State is exchanged in a stateless, self-describing manner
33
Handling Evolution
34Meaning is Determined by the Context of Relationships
35Content & Relationships May Change Over Time...
36...But the Identity of Information Must Remain Stable
37
The Information Space consists of Identifiable Conceptual Mappings
State(at a point in time)
Object
38
Instead of Objects & State, REST calls these Resources & Representations
Representation(at a point in time)
Resource
39
The conceptual mapping should remain consistent over time!
Concept
Representation(at a point in time)
Resource
“Customer 31319”
Concept
“The state of Customer 31319 at a
point in time”
Server
Client
(stable)
40
Example:Different Representations
Concept
XML Document
Resource
“Customer 31319”
Concept
“The state of Customer 31319 at a
point in time”
Server
Client
(stable)
CSV File
41
Multi-Organization Interoperability
42An Organization Manages Information
43Information Spans Organization Boundaries
44Clients Span Organizational Boundaries
45Clients & Infrastructure Evolve Independently
46
So, representations must be uniformly specified
47
What Domains are in a Representation?
The means and intent of communication All clients and servers should at least be able to
communicate, regardless of the particular resource Thus, communication must be very generic and
uniformly applicable to all resources
The information bits themselves, along with their self-descriptive interpretation Information can be represented in many ways, even if
it’s the same concept
The references for the requested or related resources Relationships between a medium identifiers can
denote the meaning of information
48
How Stable Should the Domains of a Representation Be?
Data ElementData Element Level of StabilityLevel of Stability Authority over ChangeAuthority over Change
Means & Intent of Communication
Extremely Stable Global
Representation of InformationVery Stable Widely Shared Agreement
Information Resource IdentifierStable Organization
49
Control Flow
50
Embed Control FlowInside the Representation Itself
51
As Information is Manipulated or Conveyed,So Are The References for “Next Steps”
52
Hyperlinks Define thePossibilities for Communication
53Application State is Held By the Agent
54
Summary of the REST Uniform Interface
All valuable resources are identified by a globally scoped, uniform, resource identifier Simple, consistent ability to (de)reference both information & services
Resources are manipulated through the exchange of representations Stateless, visible communication to encapsulated resources
All exchanges are self-descriptive Enables visibility and extensibility
All access methods mean the same for all resources Communication semantics are universal
Hypermedia is the engine of application state Information and controls are intertwingled
55
Agenda
Part 1
How We Got Here
Understanding Distributed Object Interoperability
Exploring the Design of REST Identity and Reference Handling Evolution Multi-Organizational
Interoperability Control Flow through Hypermedia
Part 2
RESTful Web Services in Action
Exploring the Web’s Architecture
Applied Techniques for RESTful Web Services Resources-Oriented Architecture Security Approaches Transactions & Reliability
RESTful Web Servicesin Action
57
RESTful Light Bulbs
Imagine any lightbulb, anywhere, is an HTTP-supported resource
Multiple representations might be supported Application/lightbulb+xml An HTML form
E.g. Turning on a lightbulb:PUT http://example.com/kitchen/main HTTP/1.1
Content-Type: application/lightbulb+xml
<lightbulb>
<state>on</state>
</lightbulb>
How does this impact the scale of our lightbulb system?
58
A Hypermedia Purchase Order
Without URIs and Links
POST /purchaseOrders HTTP/1.1
<?xml version="1.0"?>
<updatePurchaseOrder>
<purchaseOrder id=“31811” orderDate="1999-10-20">
<shipTo country="US">
<name>Alice Smith</name>
<street>123 Maple Street</street>
<city>Mill Valley</city>
<state>CA</state>
<zip>90952</zip>
</shipTo>
<items>
<item partNum="872-AA"> <quantity>1</quantity>
<USPrice>148.95</USPrice>
<comment>Confirm this is electric
</comment>
</item>
</items>
</purchaseOrder>
</updatePurchaseOrder>
With URIs Hyperlinks
PUT /purchaseOrder/31811 HTTP/1.1
<?xml version="1.0"?>
<purchaseOrder orderDate="1999-10-20">
<shipTo
href=“http://alice.com/address”/>
<items>
<item href=“http://lawnsrus.com/872-AA”>
<quantity>1</quantity>
<comment>Confirm this is electric
</comment>
</item>
</items>
</purchaseOrder>
How do either of these impact evolution over time?
59
Exposing Legacy Information as Resources
Reusing web Infrastructure to support the legacy
E.g. Cacheable, Searchable, Composable Customer Data
RESTful Web Service Gateway
Legacy Customer Information System
Composite App
DB
Web Cache Web Crawler
http://company.com/customer/30133
Other
Resources
61BEA Confidential | 61
Web Architecture
A View of the Current Web Architecture
Foundation
Internet ProtocolFoundation
Hypertext Transfer Protocol (HTTP)
Sec
uri
ty &
Au
then
tica
tio
n
Man
agem
ent
& C
ach
ing
BGP
ApplicationProtocolStandards
ARPDNSIPTCP
Multipurpose Internet Mail Extensions (MIME)
Interoperability
CrawlersCrawlersUser AgentsUser Agents Semantic WebSemantic Web MicroformatsMicroformats
Web Sites Resource-OrientedSystem Interfaces
ContentSyndication
Media Types
HTMLHTML XMLXML RSS / AtomRSS / Atom
URI/IRIUnicode
JavaScriptJavaScript ImagesImages VideoVideo
JSONJSON
Form dataForm data
62
Inside an Object Request Broker
Source: OMG CORBA 2.3 Architecture & Specification
63
The New ORB: A Web Server
64
Hypertext Transfer Protocol
GET /docs/2.2/ HTTP/1.1\r\nHost: httpd.apache.org\r\nAccept: text/html, text/*, */*\r\nUser-Agent: GET/7 libwww-perl/5.40\r\n\r\n
HTTP/1.1 200 OK\r\nDate: Thu, 31 Jul 2007 15:40:09 GMT\r\nServer: Apache/2.2.4\r\nContent-Type: text/html\r\nContent-Language: en\r\nTransfer-Encoding: chunked\r\nEtag: “b381faa81c”\r\nCache-control: max-age=3600\r\nVary: Accept-Language\r\n\r\n 4090\r\n<HTML><HEAD> …
65
Highlights of the HTTP Interface
Methods
GET Retrieve a resource’s state
PUT Store/replace a resource’s state
POST Create/append to a resource’s state
DELETE Remove a resource’s state
HEAD Retrieve resource & control metadata
OPTIONS Retrieve what methods are supported
TRACE Retrieve a loop-back of the request
CONNECT Request that a proxy tunnel the request
Response Code Ranges
100…Informational
200…Success indicators Request has succeeded New resource created
300…Redirection Content negotiation Resources that have moved permanently or
temporarily Cacheable resources that have not changed
400…Bad Requests Method not supported Unauthorized access Resource not found currently or permanently
500…Server Errors Feature not implemented Service temporarily unavailable Internal server errors
66
How Stable Should a Web Representation Be?
Data ElementData Element Level of StabilityLevel of Stability Authority over ChangeAuthority over Change
Means & Intent of Communication
(e.g. identifiers, operations & metadata)
Extremely StableGlobal
(HTTP & URI Syntax)
Representation of InformationVery Stable
Widely Shared Agreement(MIME Types)
Information Resource IdentifierStable
Organization(Allocated URIs)
67
How does a Client Understand its Information?
Stable, standardized, streaming media formats Multipurpose Internet Mail Extensions (MIME) Types Generic containers for more specific information
The context of relationships to other identifiers Information may embed links within a document E.g. an HTML tag <IMG SRC=“http://images/customer/31319...”> implies:
A link is to an associated resource
This resource should respond to HTTP GET
This resource will be an image of some type
Code-on-demand that can interpret the information for the client E.g. Java Applets, ActiveX Controls, Flash Plug-ins, JavaScript, AJAX
68
Intermediaries
Proxies are client-determined intermediaries
Gateways are server-determined intermediaries
Caching with HTTP Conditional GET
GET /docs/2.2/ HTTP/1.1\r\n
Host: httpd.apache.org\r\n
Accept: text/html, text/*, */*\r\n
If-None-Match: “38d8af8210bb”\r\n
\r\n
HTTP/1.1 304 Not Modified\r\n
Content-Type: text/html\r\n
Content-Length: 0\r\n
ETag: “38d8af8210bb”\r\n\r\n
ProxyUser Agent
(Browser)
GatewayServer
69
Some flaws in the Deployed Web Architecture
MIME Types Benefit: reuse most of the metadata
and media descriptions for email But, designed for Email, not the web Focus on packaging lots of
information into a single package
Hypermedia prefers linking to subsets
Workarounds for layered content
e.g. GZIP transfer encoding of a text/html document
Assumes a lossy transport Centralized type registry
Cookies Site-wide opaque data
Reduced visibility, privacy, and security
Can lead to counter-intuitive behaviour
Some sites do not handle the browser’s back button
Persistent data should be a resource
Eg. A Shopping Cart URI
But, requires browser extensibility
Alternatives are emerging with the popularity of AJAX-like techniques to store & retrieve preferences
Applied Techniques for RESTful Web Services
71
A Design Method forResource-Oriented Architecture
1. Figure out the data set
2. Split the data set into resources
3. For each resource:a. Name the resources with URIs
b. Expose a subset of the HTTP interface
c. Design the representation(s) accepted from the client
d. Design the representation(s) served to the client
e. Integrate this resource to others, using hypermedia links and forms
4. Consider use casesApply standard hypermedia types (e.g. HTML, Atom), where applicable
5. Consider error conditionsApply standard hypermedia types (e.g. HTML, Atom), where applicable
72
Common & Emerging Hypermedia Types
The big six:
XHTML, HTML, Flash, PDF, RSS, and Atom
Emerging:
Atom Publishing Protocol, HTML Microformats
On the horizon:
HTML 5
RDF, GRDDL and Semantic Web standards
Media types usually define a general purpose control flow:
73
RESTful Security Approaches
Identity Assertion OpenID
URI-based digital identity
Open ID 2.0 is adding attributes, secure messages
Kerberos / SPNEGO
Popular on intranets, requires a server
Not likely internet scalable
Current implementations require excessive round-trips
74
RESTful Message Confidentiality & Integrity
Mutual SSL Robust, widely supported Does not enable intermediary visibility or
persistent signatures
XML Signature & Encryption But XML Canonicalization is extremely
complicated
S/MIME or PGP Key negotiation is usually out of band
OpenID 2.0 may help Widely deployed within the email system
(perhaps not as widely used) Fits within existing HTTP infrastructure Not widely deployed within HTTP Limitations:
http://blog.jclark.com/2007/10/why-not-smime.html
Image: Mark Wahl, ldap.com
75
Transactions & Lost Updates
Conditional PUT A RESTful approach to optimistic locking
PUT /test_resource HTTP/1.1\r\nHost: bea.com\r\nAccept: */*\r\nExpect: 100-continue\r\nIf-None-Match: “381fa9311bc”\r\n\r\n
HTTP/1.1 412 Precondition Failed\r\nDate: Thu, 31 Jul 2007 15:40:09 GMT\r\nServer: Apache/2.2.4\r\nContent-Type: text/html\r\n
76
RESTful Distributed TransactionsDesign Sketch: Resource-Oriented Units of Work Resource for the Transaction Manager & each Transaction A new resource is created for each registered resource, enabling isolation Transaction manager acts as an agent to register resources, prepare & commit
77
Reliable Messaging
Ordered messaging Assumes a protocol will order stateful, asynchronous application exchange
patterns Whereas, REST assumes stateless, synchronous application exchange
patterns
Each request requires a response
Requests may be pipelined, will be responded to in-order
Exactly-once processing Legitimate need for resource state changes, e.g. In HTML forms, eliminating the “double form submit” button Two possible solutions:
Application-level check (e.g. unique ID in HTML hidden form field)
POST Exactly Once: http://www.mnot.net/drafts/draft-nottingham-http-poe-00.txt
79
Summary
The “Global Object Web” is largely here: it’s the Web!
The Web is similar to a Distributed Object System Objects ~= Resources Resources may be referenced, created, updated and/or deleted
Global-scale interoperability requires constraints on communication Unpredictable evolution & configurations are the norm Serendipitous reuse is more common than planned reuse Must depend on a stable, but extensible, uniform interface to communicate
Many traditional distributed systems challenges can be solved in a RESTful manner
Reliability, optimistic locking, security, transactions
More work remains to be done – “Learn to love the Web”!
Alternative Architectures
When is REST not appropriate?
82
Alternative Architectures
Transparent Remote Access Developer transparency is paramount Information sharing is unimportant Evolution is federally controlled (e.g. producers & consumers are under
the same control ) Distributed Objects or XML Web Services can be appropriate
Globally Consistent Transaction System Tightly Coupled Single-System Image (e.g. a cluster) Consistency trumps High Availability Accomplished through language-level bindings (e.g. Java XA, MSDTC)
or distributed object network protocols (e.g. TIP or IIOP)
83
Alternative Architectures
Stateful, In-Order, Reliable Messaging When you don’t have the time or inclination to share data Most interactions are not safe or idempotent The application does not lend itself well to statelessness The physical scale (number of servers) is relatively small at each layer Clients are trusted & controlled in the same federation as the servers The network is controlled & predictable
Structured Data Analysis, Manipulation and Reporting Remote Data Access (SQL, SPARQL) interfaces use general-purpose
connectors Requires extensive knowledge of the schema Often requires per-client state maintenance
84
Alternative Architectures
Stateful, real-time two-way communication E.g. voice or video communication / conferencing, P2P file transfer Requires a synchronous two-way streaming architecture
However, stream signaling and negotiation arguably can be RESTful
E.g. BitTorrent registries & trackers require the Web to function
85
Alternative Architectures
High-speed Publish/Subscribe Event Notification HTTP is limited to request/response semantics May be significant protocol overhead Arguably a limitation of HTTP 1.1, not REST
Examples of fitting publish/subscribe into HTTP: Simulating an Event Stream as a long HTTP GET Request Installing an HTTP Server on the Agent side
No obvious message correlation, hindering visibility
86
Events on the Web: Comet-Style AJAX Applications
Examples include: Google Mail’s GTalk
integration Meebo instant messaging Renkoo Jot Live
Simulate events through long HTTP GET requests
88
Further Reading
Andreessen, Mark. IIOP and the Distributed Objects Model http://web.archive.org/web/19961026035558/http://www3.netscape.com/comprod/columns/techvision/iiop.html
Box, Don. Lessons from the Component Wars: An XML Manifesto http://msdn2.microsoft.com/en-us/library/aa468562.aspx
Fielding, Roy. Architectural Styles and the Design of Network-Based Software Architectures http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
Fielding, Roy. ApacheCon 2007 Presentation: The Rest of REST http://roy.gbiv.com/talks/200709_fielding_rest.pdf
The Object Management Group, The Common Object Request Broker: Architecture and Specification http://www.omg.org/docs/formal/99-10-07.pdf
The OpenID Foundation http://openid.net
Orfali, Harkey, and Edwards. BYTE Magazine (October 1997), CORBA, Java, and the Object Web http://www.byte.com/art/9710/sec6/art3.htm
Maier & Rechtin. The Art of Systems Architecting. CRC Press, ISBN 0849304407
Nieslen & LaLiberte. Editing the Web: Detecting the Lost Update Problem Using Unreserved Checkout http://www.w3.org/1999/04/Editing/
Nottingham, Mark. Post Once Exactly (POE) Internet Draft http://www.mnot.net/drafts/draft-nottingham-http-poe-00.txt
Richardson & Ruby. RESTful Web Services. O’Reilly Media, ISBN 0596529260
Russell, Alex. Comet: Low Latency Data for the Browser http://alex.dojotoolkit.org/?p=545
World Wide Web Consortium, Architecture of the World Wide Web, Volume One http://www.w3.org/TR/webarch/