grid computing
DESCRIPTION
TRANSCRIPT
Grid Computing
• Grid computing is all about achieving greater performance and throughput by pooling resources on a local, national, or international level.
• Issues:– Determination of resources to be allocated to the VO.– Determination of the resources to be allocated to a given
application.– Communication between heterogeneous resources from
different locations.– How to parallelize an application to make it suitable for execution
on a Grid.– How to secure the shared resources.
More Formal Definition
• A grid is a system that:– Coordinates resource sharing in a de-centralized manner (i.e.,
different VOs).– Uses standard, open, general purpose protocols and interfaces.– Delivers non-trivial qualities of service.
• Guaranteed bandwidth for application.
• Guaranteed CPU cycles.
• Guaranteed latency.
The Sentence That Almost Killed the Golden Goose
• “Hence, our Grid architecture is first and foremost a protocol architecture, with protocols defining the basic mechanisms by which VO users and resources negotiate …”– Led to the Grid revolt of 2003 and displaced the OGSI with a
Web services architecture.
Layered Architecture of a Grid
• Fabric: Resources to be shared in Grid systems.• Connectivity: All of the protocols that allow resources to
communicate. – For example: TCP/IP, HTTP, XML, SOAP.
• Resource: Services and protocols that enable the management of individual resources:– Information Protocols: Allows us to access information about the
resource. – Management Protocols: Manage aspects of the resources to a
certain degree.• E.g., monitoring the way resources are being used.
• Collective: Services and protocols for managing multiple resources.– Take a collection of resources and make them work together to
solve a common task.– Examples include:
• Resource registries.
• Allocation and scheduling services.
• Monitoring Services.
• Data Management services.
• Applications– Fundamental user of Grid Services.
Summary
• Grid architecture:– Fabric– Connectivity– Resource– Collective– Applications
Open Grid Services Architecture
• Developed by the Global Grid Forum to define a common, standard, and open architectures for Grid-based applications. – Provides a standard approach to all services on the Grid.
• VO Management Service.
• Resource discovery and management service:
• Job management service.
• Security services.
• Data management services.
• Built on top of and extends the Web Services architecture, protocols, and interfaces.
Web Services: W3C Working Group
• Definition: A Web service is a software system identified by a URI [RFC 2396], whose public interfaces and bindings are defined and described using XML. Its definition can be discovered by other software systems. These systems may then interact with the Web service in a manner prescribed by its definition, using XML based messages conveyed by Internet protocols.
Figure 1.10. A stateless Web Service invocation
Figure 1.11. A stateful Web Service invocation
• Homework up for grabs! The following topics will be presented in next weeks class:– Web services architecture.– XML and SOAP.– Web services resource framework.– Overview of Globus TK4.
Summary
• Grid architecture:– Fabric– Connectivity– Resource– Collective– Applications
• Recall that most web services are stateless (do not save information between successive invocations.
• OGSA specification requires stateful services.• This resulted in the Web Services Resource Framework
(WSRF) that specifies how to make web services stateful (among other things).
• Joint effort between Web Services and Grid communities (OASIS).
Web Services
• A client/server system • Concept similar to Remote Procedure Call (RPC),
Remote Method Invocation (RMI), only applied over HTTP
Specifications for/of making Web Services Stateful.
Defines common standards, and open architecture for grid-based applications.
Relationship between OGSA, GT4, WSRF, and Web Services
Web Services
• Allow creation of client/server applications.• Platform and language independent based on XML.• Most use HTTP for transporting messages (firewalls
generally do not attack HTTP traffic.• Lend themselves naturally to build loosely coupled
distributed systems.
• Disadvantages:– Transmitting data in XML is not too snappy.– Still evolving.
Invocation
• Discover Web Service– Universal Description, Discovery and Integration (UDDI) protocol also being
standardized by OASIS.
• Receive addresses of requested service from discovery service.• Ask service to describe itself (i.e., how to invoke it).• Service responds with Web Service Description Language
(WSDL).• Invoke service using a SOAP message.• Responds with SOAP message.
Web Services
Web Services Architecture
The Web Services Architecture is specified and standardized by the World Wide Web Consortium, the same organization responsible for XML, HTML, CSS, etc.
Web Services Addressing
• Addressed via URI (Uniform Resource Identifiers). – Basically same as URLs.– Difference is that URLs are for humans, URIs are for machines.
• Services generally invoked by program stubs. – Generate SOAP requests and interpret SOAP responses.– Stubs generated by some tool (generally) based on WSDL
received from server.
1.Client will call the client stub. The client stub will turn this 'local invocation' into a proper SOAP request. This is often called the marshaling or serializing process.
2.The SOAP request is sent over a network using the HTTP protocol. The server receives the SOAP requests and hands it to the server stub. The server stub will convert the SOAP request into something the service implementation can understand (this is usually called unmarshaling or deserializing)
3.Once the SOAP request has been deserialized, the server stub invokes the service implementation, which then carries out the work it has been asked to do.
4.The result of the requested operation is handed to the server stub, which will turn it into a SOAP response.
5.The SOAP response is sent over a network using the HTTP protocol. The client stub receives the SOAP response and turns it into something the client application can understand.
6.Finally the application receives the result of the Web Service invocation and uses it.
1. Web Service: Code that provides the service.
2. SOAP Engine: Handles SOAP requests and responses (e.g., Apache Axis).
3. Application Server: Living space for required applications (e.g., the SOAP engine). Apache Jakarta Tomcat is an application server.
4. HTTP Server.
Web Services Resource Framework
• Web Service + Stateful Resource = WS-Resource• WS-Addressing specifies an endpoint reference.
– Each resource has a unique key included in address.
• Stateful resource has three characteristics– It is composed by state data defined in XML format.– It has a life cycle.– It can be manipulated by one or more Web Services.
Four Required Characteristics of WS-Resources
• Atomicity: Stateful resource updates within a transactional unit are made in an all-or-nothing fashion.
• Consistency: Stateful resources should always be in a consistent state even after failures.
• Isolation: Updates to stateful resources should be isolated within a given transactional work unit.
• Durability: Provides the permanence of stateful resource updates made under the transactional unit of work.
Four Required Characteristics of WS-Resources
• Atomicity: Stateful resource updates within a transactional unit are made in an all-or-nothing fashion.
• Consistency: Stateful resources should always be in a consistent state even after failures.
• Isolation: Updates to stateful resources should be isolated within a given transactional work unit.
• Durability: Provides the permanence of stateful resource updates made under the transactional unit of work.
• This sounds pretty similar to the mutual exclusion problem (recall the dining falcons problem).– Can semaphores provide all of these characteristics?
Resource Properties
• Provide a view on the current state of the resource.• Service Data Values:
– Properties of the service, operation results, runtime information, etc
– MetaData• Information regarding the service data values.
– E.g., last modified, last use of data, etc.
– Information needed to manage the state.• Similar to metadata but refers to resource as a whole.
WSRF Specification
• WS-ResourceProperties– Defines how the state of a WS-resource can be manipulated by Web
Service.– A resource property maps to an individual component of the resource
stat.– Methods to retrieve, change, and delete visible properties of a WS-
Resource.
• WS-ResourceLifetime– Mechanisms to manage the lifecycle of our resource.
• WS-ServiceGroup– Related to groups of Web services or groups of WS-resources.
• Add new service to the group, find a service in a group, remove the service from the group, etc.
WSRF Specification
• WS-Resource Lifecycle– Mechanisms to manage the lifecycle of our resource.– Creation:
• Usually created by a resource factory.
– Destruction:• The means by which a stateful resource is destroyed and system
resources recovered.
– Resource Identifier:• Must have at least one resource identifier.
• WS-ServiceGroup– Related to groups of Web services or groups of WS-resources.
• Add new service to the group, find a service in a group, remove the service from the group, etc.
WSRF Specification
• WS-ServiceGroup– Organize collections of WS-Resources to build registries or
services that can perform collective operations. – Defines means for managing heterogeneous collections of Web
Services.– Uses memberships, rules, and constraints to define groups.– A group is a collection of members that meets some constraints
defined using resource properties.
WSRF Specification
• WS-BaseFaults– Standard way of representing faults in execution of service.
• WS-Notification.• Publish/subscribe mechanism.
• Defines interfaces that clients can use to subscribe to topics of interest, and receive notifications asynchronously.
– Example: Changes to resource properties.
• WS-Notification Members:– WS- BaseNotification
– Defines mechanisms that allow a subscriber to register receive notifications from a producer
– Ws-Topics– Hierarchical organization of notification messages that subscribers can
understand and subscribe to.
– WS-BrokeredNotification:• Defines interface to Notification Broker that manages subscriptions
in the systems.
Why Move to Stateful Resource Model from OGSI
• Too bulky: Not a clean separation to support incremental adoption.
• Object-oriented framework not deemed suitable by large segments of the Grid/Web Services communities.
• The Web Services framework was already heavily used, with standard protocols. No need to create an additional set of protocols.
• Web services are open-source.
Web Service Description
• Recall use an Interface Description language (IDL) called WSDL to formally describe a service, what is does, how it is accessed, etc.
Homework
• Write a web service called “calculator”.– Provide both the client and the server.– Develop an information service that can be contacted to find this
service.• Can use IP address and port number.
• Define how the client will interact with information server to retrieve URL of calculator service.
– Send message to calculator asking it what operations it can perform and parameters to operations.
• You define how this interaction takes place.
– Invoke service.