s-cube lp: chemical modeling: workflow enactment based on the chemical metaphor
DESCRIPTION
TRANSCRIPT
![Page 1: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/1.jpg)
www.s-cube-network.eu
S-Cube Learning Package
Chemical Modeling: Workflow Enactment based on the Chemical Metaphor
INRIA, CNR, SZTAKI
Zsolt Németh, SZTAKI
![Page 2: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/2.jpg)
Learning Package Categorization
S-Cube
Service Infrastructure
Multi-level and self-adaptation
Supporting adaptation of
service-based applications
![Page 3: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/3.jpg)
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
![Page 4: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/4.jpg)
This talk is about workflow but…
Workflow is just an example
– It is a common programming model for grids
– Features many coordination problems
Focus: coordination
– Large-scale distributed, dynamic, error prone environment and
applications
– Some degree of autonomy, adaptability, self-* must be provided
![Page 5: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/5.jpg)
Workflow
“The computerized facilitation or automation of a
business process in whole or in part” – The Workflow Management
Coalition
a collection of activities that are processed in some order
and where both data-flow and control-flow relationships
may be present
![Page 6: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/6.jpg)
The workflow Management Reference Model (Workflow Management Coalition)
Process Analysis, Modeling
& Definition Tools
Process Definition
Workflow Enactment Service
Applications &
Tools User (human)
Process Design
& Definition
Build time
Run time
Process Instantiation
& Control
Interaction with
Users & Application Tools
![Page 7: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/7.jpg)
Scope of enactment
Abstract workflow
standalone application components
the order in which they are executed
names, references, etc. are logical
Represented by a graph
typically: DAG
Problem
Abstract workflow
Concrete workflow
Physical environment
![Page 8: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/8.jpg)
Scope of enactment
Concrete workflow
standalone application components
the order in which they are executed
selected resources, services
additional activities (e.g. file transfer, staging, etc.)
all names, references, etc. are physical
Problem
Abstract workflow
Concrete workflow
Physical environment
![Page 9: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/9.jpg)
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
![Page 10: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/10.jpg)
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Workflow engine
![Page 11: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/11.jpg)
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
![Page 12: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/12.jpg)
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
![Page 13: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/13.jpg)
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
Concrete workflow
![Page 14: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/14.jpg)
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
Concrete workflow
![Page 15: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/15.jpg)
Problem analysis (current approaches)
The abstract workflow is static
(Typically) no advanced control structures
A priori mapping/ partly a priori mapping
Mapping based on stored information
A priori simulation/ a priori test/ a priori optimisation
Centralised engine
Human interaction
Limited autonomy, limited ability for adaptation
Overall lack of any high level model
![Page 16: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/16.jpg)
Requirement analysis
Workflow enactment in large-scale heterogeneous environments
– should provide a higher level of autonomy
– should be able to adapt to changing conditions
– should be distributed
– should be able to make decisions on partial and actual information
– should support arbitrarily complex control structures
Often a complex problem can be solved in a more simple way using nature inspired models
![Page 17: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/17.jpg)
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
![Page 18: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/18.jpg)
This talk is about a chemical metaphor but…
It is just an example
– Chemical reactions are reasonably similar to workflow enactment
– It has a well established formalism
Generally: nature inspiration for solving complex problems
– Simple, primitive parts behave “intelligently” as a whole
– Ants, termites, cells, molecules, etc.
![Page 19: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/19.jpg)
A chemical metaphor
Reactions are
– autonomous
– distributed
– concurrent
– depending on local conditions
– depending on actual conditions
– not following any a priori pattern
– evolving in time
![Page 20: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/20.jpg)
A chemical metaphor
Reactions are
autonomous
distributed
concurrent
depending on local conditions
depending on actual conditions
not following any a priori pattern
evolving in time
Workflow enactment should
provide a higher level of autonomy
be able to adapt to changing conditions
be distributed
be able to make decisions on partial and actual information
support arbitrarily complex control structures
![Page 21: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/21.jpg)
A vision of chemical enactment
Resources
![Page 22: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/22.jpg)
A vision of chemical enactment
Resources
Activities
![Page 23: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/23.jpg)
A vision of chemical enactment
Resources
Activities
Control
![Page 24: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/24.jpg)
A vision of chemical enactment
React
![Page 25: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/25.jpg)
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized
in the -calculus
Further challenges
![Page 26: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/26.jpg)
From a vision to a model
Materialize information: resource quantums
Define an abstract chemical coordination model
– independent from actual technical realizations
– provide a high-level abstract framework for refinement
– formalism: -calculus
Advance, refine: find chemical examples for
– Complex control
– Fault tolerance
– Resource management
– Optimization
– etc
![Page 27: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/27.jpg)
Resource quantums
The usual solutions
P1 P2
P3
P4
P5
P6
• stored information may be time sensitive
![Page 28: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/28.jpg)
P6
Resource quantums
The usual solutions
P1
P2
P3
P4
P5
![Page 29: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/29.jpg)
Resource quantums (« materialization of information »)
resources are represented as tickets
P1 P2
P3
P4
P5
P6
• tickets represent a guaranteed service
![Page 30: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/30.jpg)
Resource quantums (« materialization of information »)
P2
P5
P1
P3
P4
P6
![Page 31: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/31.jpg)
The -calculus
a declarative, functional formalism
inherently concurrent model of computation
basic data structure: multiset (chemical solution)
– passive molecules: booleans, integers, tuples, naming molecules
– active molecules: -abstraction
reaction: active molecules capture other molecules
– x.M, N → M[x:=N]
execution: perform reactions until a stable (inert) chemical
solution is resulted
![Page 32: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/32.jpg)
The -calculus: -abstraction
Active molecules: -abstraction: PC.M
– P is a pattern that selects elements for the reaction
– C is a condition; the reaction takes place if C is true
– M is the action
Example: i:x, j:y i≤j, x>y. (j+1):x, j:y
Semantics: capture i:x and j:y and replace them by (j+1):x,
j:y if i≤j, x>y
-abstraction is one-shot
Universal matching symbol:
![Page 33: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/33.jpg)
The -calculus
-terms are
– Commutative: M1,M2≡M2,M1
– Associative: (M1,M2),M3≡M1,(M2,M3)
– Realize Brownian motion
Reactions
– Locality: if M1→M2, then M,M1→M,M2
– Solution: if M1→M2, then <M1>→<M2>
Conditional reactions
– xC.M
Atomic capture – x1,x2,…xn.M
![Page 34: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/34.jpg)
Resources
a resource quantum is modeled as a sub-solution
– recall: chemical solutions can be nested
elements in the solution are inert attribute:value pairs
there are mandatory attributes otherwise, the format is felxible
<id:R1, type:comp, proc:16, OS:Linux, …>
<id:N1, type:net, bandwidth:23, …>
<id:R3, type:comp, proc:1, installed:equsolver, network:N1 …>
![Page 35: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/35.jpg)
Elementary activities
the chemical model does not execute an activity, just enacts it – execution is external
exit from the chemical world:
– execute activity on resource using parameter
– a symbolic notation that can be realised in many ways
return to the chemical world
– the execution produces some result or error put back in form of a solution (control molecule)
– <ActivityID:result>
– <ActivityID:result, error:errorcode, …>
– <ActivityID:result, executed:R1, …>
execute A on R→ <A:result…>,<id:R,…>
![Page 36: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/36.jpg)
Resource dependency
activity A needs a resource
<id:r, type:comp, proc:1, >. execute A on r
<id:r, type:comp, proc:1, >.
execute A on r
< id:R1, type:comp,
proc:16, OS:Linux, …>
< id:R2, type:comp, proc:1,…>
< id:R3, type:comp,
proc:1, OS:SunOS, …>
< id:R4, type:comp,
proc:1, memory:23, …> < id:R5, type:comp,
proc:1, disk:12, …>
![Page 37: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/37.jpg)
Resource dependency
<id:r, type:comp, proc:1, >. execute A on r
Captured a matching resource
< id:R1, type:comp,
proc:16, OS:Linux, …>
< id:R2, type:comp, proc:1,…>
< id:R4, type:comp,
proc:1, memory:23, …> < id:R5, type:comp,
proc:1, disk:12, …>
execute A on R3
![Page 38: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/38.jpg)
Resource dependency
Both the resource and the activity are replaced by a control
molecule
< id:R1, type:comp,
proc:16, OS:Linux, …>
< id:R2, type:comp, proc:1,…>
< id:R4, type:comp,
proc:1, memory:23, …> < id:R5, type:comp,
proc:1, disk:12, …>
<A:12, error:0, executed:R3>
< id:R3, type:comp,
proc:1, OS:SunOS, …>
![Page 39: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/39.jpg)
Data and control dependency
activity A waits a result from activity B
<B:x, >. execute A using x
<B:x, >. execute A using x <B:18>
<C:0, error:5>
<D:42>
<H:apple>
![Page 40: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/40.jpg)
Data and control dependency
<B:x, >. execute A using x
Captured a corresponding control molecule
execute A using 18
<C:0, error:5>
<D:42>
<H:apple>
![Page 41: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/41.jpg)
Data and control dependency
Replaced by the result
<A:2, error:0,...>
<C:0, error:5>
<D:42>
<H:apple>
![Page 42: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/42.jpg)
Complex dependencies
An activity needs both input from another activity and
resources
– <id:r, 1>, <B:x, 2>. execute A on r using x
Dependencies can be arbitrarily combined
– find a resource r1 for activity A and a resource r2 for B so that r1 and r2
have the same operating system
– <id:r1, OS:x, 1>, <id:r2, OS:x, 2> .
execute A on r1, execute B on r2
Conditions can be added as well
– <id:r, disk:x, memory:y, > x>30, y>12.
execute A on r
![Page 43: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/43.jpg)
Workflow patterns
Sequence
Conditional
Split
Synchronizing merge
P-split
– p-out-of-n activities follow A
P-merge
– p-out-of-n results trigger activity A
Loop
![Page 44: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/44.jpg)
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
![Page 45: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/45.jpg)
Challenges
What we have so far is a framework
– Principles of a chemical coordination model
– It is just the beginning!
Let’s explore and advance
– Solve further aspects of workflow enactment: fault tolerance,
optimization, resource control, complex workflow structures,
constraints, co-allocation, compensation, etc.
– Find further nature analogon within the chemical model:
temperature, weight, magnetic properties, size, gravity, catalysts,
membranes, etc.
![Page 46: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/46.jpg)
Challenges (examples)
Resource control by chemical agents
– withdraw, modify, block, accept, reject, group, etc.
Assert a neutralization agent: <id:r1, >.
Replace a molecule:
<id:r1, proc:1, OS:x, >.<id:r1, proc:1, OS:Linux, >
Combine two molecules:
(<id:r1, proc:1, >, <id:r1, proc:1, >).
<id:r1, proc:2, >
![Page 47: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/47.jpg)
Challenges (examples)
Fault-tolerance by chemical agents
– error molecules, retry, rollback, redundancy, compensation, etc.
Assert an error handling molecule that reactivates in case of
error
(<Ai:x, >, <Aj:y, >, <id:r, R, >).
(execute A on r using x y,
(<A:errork, >,<id:r’, R, >).execute A on r’ using x y))
![Page 48: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/48.jpg)
Challenges (examples)
Complex resource allocation and co-llocation scenarios by
molecules
– e.g. reserve r1 and r2 so that there is a network link between them
– (<id:r1, proc:1, network:n, >,
<id:r2, proc:4, network:n, >,
<id:n, type:net, >).
(execute A on r1, (<A:x, >.execute B on r2 using x))
![Page 49: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/49.jpg)
Conclusion
The chemical metaphor: autonomous, adapting, distributed,
actual
The nature inspired coordination model
– activities, resources and control are modeled using the same
formalism
– -calculus is not just description but defines execution semantics as
well
– workflow structure, resources and control can be
added/withdrawn/modified in a fully dynamic and adaptive way
![Page 50: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/50.jpg)
Connections to other teaching units
Foundations
– The Chemical Computing model and HOCL Programming
Applications
– Dynamic Adaptation with the Chemical Model
© S-Cube – 50/<Max>
![Page 51: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/51.jpg)
References
Zsolt Németh, Christian Pérez, Thierry Priol: Distributed workflow coordination: molecules and reactions.
IPDPS 2006
Zsolt Németh, Christian Pérez, Thierry Priol: Workflow Enactment Based on a Chemical Metaphor. SEFM
2005: 127-136
Manuel Caeiro, Zsolt Németh, Thierry Priol: A Chemical Model for Dynamic Workflow Coordination. PDP 2011:
215-222
![Page 52: S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c300614a795919718b4584/html5/thumbnails/52.jpg)
Acknowledgements
The research leading to these results has
received funding from the European
Community’s Seventh Framework
Programme [FP7/2007-2013] under grant
agreement 215483 (S-Cube).