holding slide prior to starting show
DESCRIPTION
Holding slide prior to starting show. Portals for Workflow Composition and Service Management. David W. Walker School of Computer Science Cardiff University. http://www.cs.cf.ac.uk/User/David.W.Walker /. System Portals. We need portals to be able to handle tasks such as: : - PowerPoint PPT PresentationTRANSCRIPT
Holding slide prior to starting show
Portals for Workflow Composition and Service
Management
David W. Walker
School of Computer Science
Cardiff University
http://www.cs.cf.ac.uk/User/David.W.Walker/
System Portals
• We need portals to be able to handle tasks such as: :– Problem specification, resource discovery and job
composition– Service deployment and lifecycle management– Submission, execution, monitoring, and (maybe)
interaction
• These tasks are quite generic, rather than application-specific.
• Portal can behave rather like a “Grid Shell”
Web Services
• Everything is a (Web/Grid) service.
• This includes:– Computation routines– Access to files and databases– Components of the Grid infrastructure,
such as workflow enactment engines, resource monitors, etc.
A Common Approach to Workflow
• Visual service composition.
• Service interfaces and other metadata expressed in an XML-based service description document.
• Services registered with, and discovered through, a registry.
Visual Service Composition Portlet
• Applications and high-level services are composed from interacting services.
• Composition can be done graphically using a “drag and drop” style of interface to draw a workflow.
• In a workflow graph the nodes represent services and the edges represent data flow or control flow.
• Need special nodes to represent loop and conditional constructs.
• Need query mechanism to discover services
Workflow Description
• A workflow can be represented with an XML-based workflow description language such as WSFL, SWFL, SCUFL, BPEL4WS, etc.
• A workflow description document can be passed to a workflow enactment engine, which provides an execution environment for the workflow.
Two Hard Problems
• Semantic specification of applications• Scheduling of workflow nodes on distributed
resources.– Early binding model: bind to specific service/platform
at composition time (“validation”).– Intermediate binding model: bind at “compile” time
(when converting from XML to executable form).– Late binding model: bind dynamically at runtime.
• Later binding allows the use of more up-to-date information to make scheduling decisions.
Mobility
• Should (some) services be mobile and/or clonable?
• Allows the automatic creation of new services that are heavily used.
• May be important for autonomic grids.
• Generalises current data migration and replication concepts.
Communication and Notification
• Another important issue is how to carry out the data flow and control flow in a workflow.
• Need an asynchronous messaging system to support interoperability and to handle connection problems.
Portlets
• Portlet technology endorsed by GCE working group of GGF, e.g., JetSpeed
• Web services have user-facing ports that provide content to users.
• Portlets can be regarded as interfaces to that content, specified via an extension to WSDL.
• Portals can be built from collections of portlets, e.g., as a set of panels for job submission, job status, visualisation, etc.
• Portlets provide as component model for user interfaces in the same way that web services provide a component model to middleware.
Grid-Oriented Portal Work at Cardiff
• Visual Service Composition Environment (VSCE) for building applications based on service workflow.
• Service Workflow Language (SWFL) for describing composite applications.
• Tools for transforming between workflow graph SWFL executable code.
• Jini Service-oriented Grid Architecture (JISGA). An execution environment taking SWFL as input.
• Java-C Automatic Wrapper (JACAW). Wraps C code as Java using JNI for deployment as Web service in JISGA.
• Grid-Service-in-a-Box (GSiB). Supports service creation, deployment, management, and monitoring.
Motivation and Targets
• Easy-to-use tools are needed to allow scientists to take full advantage of the new Grid without them needing detailed knowledge of the underlying infrastructure.
• GSiB project is developing a visual problem-solving environment/portal for all service users (clients and providers).
Service Deployment
• A web service– Its public interfaces and bindings are described in
an XML document.– It is registered in a service registry to allow service
discovery and query.
• Usually additional code or wrapper is needed to make existing software available as a Web service – a lot of repeated work.
• Legacy problems exists.
Service Deployment
• JACAW (Java-C Automatic Wrapper) automatically generates the JNI wrapping code for a legacy code.
• Axis Java2WSDL utility generates WSDL document from the Java code.
• UDDI registry server.• UDDI4J provides a Java API for
interacting with a UDDI registry.
Service Deployment
JNI Wrapper
Legacy code
WSDL Document
UDDI Registry
JACAW
JAVA2WSDL
UDDI4J
Service Composition
• Graphical symbols for constructing a flow model.
Data input port
Data output port
Control input port
Control output port
Data link
Control link
Normal and assignment activity
Control activity
An Example
A
B C
For
retA = activityA(..);
for( i=0; i<100; i++){
activityB(retA, i);
}
retC = activityC();
retA
YES
NO
retA i
Service Workflow Language (SWFL)
<activity name="for"> <for setParallel="no"> <input message=“ForInput”/> <expression><![CDATA[index=0;index<100;index++]]></expression> </for></activity>
<dataLink name=“A_For” source=“A” target=“for”> <swflMap sourceMessage=“AOutput” targetMessage=“ForInput”>
<part> <sourcePart name=“retA”/><targetPart name=“retA”/> </part> </swflMap></dataLink>
To be continued
Service Workflow Language (SWFL)
<dataLink name=“For_B” source=“for” target=“B”> <swflMap sourceMessage=“ForOutput” targetMessage=“BInput”> <part> <sourcePart name=“retA”/><targetPart name=“retA”/> <sourcePart name=“i”/><targetPart name=“i”/> </part> </swflMap></dataLink
<controlLink name=“A_For” source=“A” target=“for” /><controlLink name=“For_B” source=“for” target=“B” controlPort=“YES”/><controlLink name=“For_C” source=“for” target=“C” controlPort=“NO”/><controlLink name=“B_For” source=“B” target=“for” />
Service Workflow Language (SWFL)
• SWFL is XML based for describing interacting Web services.
• Extends Web Service Flow Language (WSFL) by – supporting the application of all the
conditional and loop control constructs of the Java language to the composition of Web services.
– allowing more general data mappings.
Another Picture
ValidatorValidator
Create SWFLCreate SWFLDrawing ScreenDrawing Screen
swflJobswflJob
WorkflowEngine
WorkflowEngine
Visual Service Composition Environment
Execution Environment
SWFL2Java
• It converts the description of a job in SWFL into executable Java code.
SWFLDescriptionDocument
Java Executable
Code
DataflowGraph
ControlFlowGraph
FlowModel
Visual Service Composition Environment
• Collaborative code development environment using a visual programming tool for seamlessly integrating services into higher-level services or complete applications.
• Applications are created by plugging together services to form a workflow graph. Service metadata is used to check if interfaces are compatible.
• Different developers can place their components in a shared repository, e.g, UDDI or Jini lookup service.
• Output is in the form of SWFL.
JISGA
• JISGA consists of two main parts:– A WorkflowEngine service– A JobProcessor service
• Grid application is submitted to a WorkflowEngine service as SWFL.
• Sequential jobs are handled directly by the WorkflowEngine service.
• Parallel jobs involve multiple JobProcessor services.
What the Workflow Engine Is
• Processes service-based applications described in an XML-based workflow language.
• Determines the order of execution and generates the harness code.
• Executes the code which includes discovering services, invoking services, receiving results and sending results to clients.
What the Workflow Engine Does
• Converts a SWFL description of a composite service-based job into an executable Java code, and executes it.
Java Executable Code
<?xml…><JFlowModel……………………….……….….
SWFL description
IntermediateFlowModel
object
SWFL2Graph Graph2Java
SWFL2Java
To be continued…
Future Research• Further development of GSiB for creating,
deploying, managing and monitoring Grid applications.
• More experiments in integration of Jini services and Web services in JISGA.
• Demo applications to illustrate use of JISGA on UK e-Science Grid.
• Integration of better scheduling in JISGA.
Final Comments
• Service-oriented approach to Grid computing is generally agreed on
• But there are many options for how such Grids might be build (GT3, Jini, JXTA, etc)
• Ability to interoperate between different Grids is important.
• Need to get more experience with actually using services for e-Science.