ben butchart wolfgang emmerich university college london [email protected]
DESCRIPTION
Orchestration of an OGSI-enabled scientific application using the Business Process Execution Language. Grids and Applied Language Theory (GAL2003). Ben Butchart Wolfgang Emmerich University College London [email protected] [email protected]. Overview. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/1.jpg)
Orchestration of an OGSI-enabled scientific application using the Business Process Execution Language
Ben Butchart
Wolfgang Emmerich
University College London
Grids and Applied Language Theory (GAL2003)
![Page 2: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/2.jpg)
Overview
• How we became interested in workflow• Why we chose BPEL• A BPEL appetizer• Shortcomings of existing BPEL implementations• Shortcomings of BPEL language itself• Overcoming some problems• Comparison with GSFL• Conclusion and future work• Discussion
![Page 3: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/3.jpg)
The Beginning…
A messy but beautiful data flow diagram
![Page 4: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/4.jpg)
Control logic of the Polymorph prediction application (not distributed)
configure system
loop 1..29 space groups
loop 1..X high denisty structurescollate results
molpak
dmarel
yes
is last structure ?
no
yes
no
is last spacegroup ?
![Page 5: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/5.jpg)
Control logic of the distributed polymorph prediction application
molpak molpak molpak
dmarel dmarel dmarel dmarel dmarel dmarel
collate results
configure system
![Page 6: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/6.jpg)
So why BPEL ?
• WSFL and XLANG deprecated• Microsoft, IBM, BEA triumvirate powerful
market force• Submitted to OASIS for standardization• First implementations already available for
evaluation (IBM BPWS4J, Collaxa)• Do not want to diverge from standard Web
Services technology set
![Page 7: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/7.jpg)
Instantiating SOAP message
<soapenv:Body> <runPoly xmlns="http://poly"> <ns1:atomArray xmlns:ns1="http://www…/cml2/core"> <ns1:atom elementType="H" x="3.17" y="2.43“ z="0.02"/> <ns1:atom elementType="H" x="-1.74" y="2.61" z="-0.38"/> </ns1:atomArray> <spacegroups> <spacegroupLabel>AA</spacegroupLabel> <spacegroupLabel>AB</spacegroupLabel> </spacegroups> <ns2:crystal z="1" xmlns:ns2="http://www…/cml2/core"> <ns2:scalar title="alpha" units="deg">0</ns2:scalar> <ns2:symmetry id="" spaceGroup=""/> </ns2:crystal> </runPoly> </soapenv:Body>
![Page 8: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/8.jpg)
BPEL Example
<sequence name="sequence"> <receive name="receive" partner=“user" portType="p:PolyPort" operation="runPoly" container="request“ createInstance="yes"/> <assign> <copy> <from container="request" part="atomArray"/> <to container="molpakrequest" part="atomArray"/> </copy> </assign>
<assign> <copy> <from container="request" part="crystal"/> <to container="molpakrequest" part="crystal"/> </copy> </assign>
![Page 9: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/9.jpg)
BPEL Example
<assign> <copy> <from container="request" part="spacegroups“
query="./spacegroupLabel[position()=1]" /> <to container="molpakrequest" part="crystal"
query="./ns2:symmetry/@id" /> </copy></assign>
<invoke name="invoke1" partner="provider" portType="mpak:MolpakPort"
operation="runMolpak" inputContainer="molpakrequest"
outputContainer="molpakresponse"/>…
</sequence>
![Page 10: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/10.jpg)
BPEL Features
• Loop and control logic• XPath query assignment• Recursive Composition• Synchronous and asynchronous
communication• Concurrent execution• Correlated messages• Event handling• Fault handling
![Page 11: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/11.jpg)
Shortcomings of BPEL implementations
• Binding to a URL expected at deployment time
• Early implementations could not cope with OGSI extensions to WSDL
• Or properly handle imports• Dynamic creation of new process instances to
improve scalability• Steering – change process during execution
![Page 12: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/12.jpg)
Shortcomings of BPEL language
• Need placeholder objects to support some assignments
• Data aggregation - No way to temporarily collate results
• Cannot specify mechanism for binding (WSDL, UDDI, OGSAFactory)
• Limited lifecycle interface (just receive, pick terminate)
• Verbose• Does not support peer communication
![Page 13: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/13.jpg)
The scalability problem
After Krishnam, Wagstrom, LaszewskiGSFL: A Workflow Framework for Grid Services
Workflow Engine Workflow Engine
Compute Service
Compute Service
Compute Service
Compute Service
Compute Service
Compute Service
![Page 14: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/14.jpg)
<invoke name="invoke1" partner="provider" portType="mpak:MolpakPort" operation="runMolpak" inputContainer="molpakrequest" outputContainer="molpakresponse"/>
getBindingURL(PortType)
BPEL Parser
AbstractBinding
WSDLBinding
UDDIBinding
OGSA Registry
OGSA FactoryBinding
OGSA Registry
UDDI Registry
Molpak serviceCreate instance
<service name="MolpakService"><port><soap:address location= "http://tapas.cs.ucl.ac.uk: 8080/ogsa/services/"/></port></service>
Late binding
![Page 15: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/15.jpg)
Scalability
Workflow Engine
Compute Service
Compute Service
Compute Service
Workflow Engine
Compute Service
Compute Service &Workflow Engine
Compute Service
compute1
compute2
compute3
![Page 16: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/16.jpg)
How we solve the problems
• late binding: extra URL binding mechanisms• scalability: recursive composition• scalability: group binaries into single WS port type • scalability: dynamic deployment and delegation• steering: use BPEL event handling functionality• aggregation: use a dedicated web service• verbose: use GUI/XSLT
![Page 17: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/17.jpg)
Other Web Service Workflow Languages
• WSFL and XLANG – now superseded by BPEL
• GSFL – extension to WSFL to support Grid requirements
• SWFL – another extension to WSFL • Scufl (taverna) / Freefluo
![Page 18: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/18.jpg)
Comparison with GSFL
• recursive composition
• correlated messages
• separates data flow
• peer data flow modelling
• binding mechanism configurable
• WS standard
• cross vendor support
• fault handling and compensation
• concurrent control flow
• message part assignment
• Xpath assignment
• implementations available
• recursive composition
• correlated messages
• separates data flow
• peer data flow modelling
• binding mechanism configurable
• WS standard
• cross vendor support
• fault handling and compensation
• concurrent control flow
• message part assignment
• Xpath assignment
• implementations available
GSFL BPEL
![Page 19: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/19.jpg)
Conclusion
• Can use BPEL for modelling grid applications – no real need for extensions
• But we need an implementation that is Grid enabled
• Might not need full implementation • We should stick with WS technology set if at
all possible
![Page 20: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/20.jpg)
Integration with Scheduler (Sun Grid Engine)
2.getBindingURL(PortType)Trader
Binding
SGE
3. Run a proxy job (jobID, traderURL)
molpak serviceisntance
OGSARegistry
Trader
SGE Wrapper
4. Run a proxy job (jobID, traderURL)
Dummy Process(blocks)
5. runDummy.sh(jobID,traderURL)
BPEL Engine
1.getBindingURL(PortType)
9.invocation completed
6. SGEchooseMe(jobID,myURL)
10.kill
7. search for instance
8.invoke service
compute node
![Page 21: Ben Butchart Wolfgang Emmerich University College London b.butchart@cs.ucl.ac.uk](https://reader034.vdocuments.net/reader034/viewer/2022051820/568149a2550346895db6e3ce/html5/thumbnails/21.jpg)
Late binding: deployment descriptor
<service name="poly/PolyProviderFactoryService/poly1"> <requestFlow> <handler type="BPELRequestHandler"/> </requestFlow> <responseFlow> <handler type="BPELResponseHandler"/> </responseFlow> <parameter name="name" value="Polymorph process workflow"/> <parameter name="className" value="poly.PolyProvider“ <parameter name=“bindingMechanism" value=“Registry"/></service>