service orientation considerations poc r1 david groep, scaling and validation programme
TRANSCRIPT
Service Orientation Considerations PoC R1
David Groep, Scaling and Validation Programme
Service Orientation
• Service Oriented Architecture (SOA)
"an application architecture within which all functions are defined as independent services with well defined invocable interfaces which can be called in defined sequences to form business processes".
• Web services is a collection of technologies, including XML, SOAP, WSDL, and UDDI,
• Thus Web services does not equal service-oriented architecture.
Source: UvA Grid Master IGC2005, Adam Belloum
Service Decomposition
Source: UvA Grid Master IGC2005, Adam Belloum
Service Orientation
• All functions are defined as services.
• All services are independent. Operate as "black boxes";
o external components neither know nor care how boxes are executed
o merely that they return the expected result.
• The interfaces are invocable; At an architectural level, it is irrelevant whether
o they are local or remoteo what interconnect scheme or protocol is used to effect
the invocation, o what infrastructure components are required to make
the connection.
Source: UvA Grid Master IGC2005, Adam Belloum
Service Orientation
• Typically within a business environment Service means business functions, business
transactions, and system services.
• The difference in the types of services. Business functions are from the application's
perspective, non-system functions that are effectively atomic.
Services might be low-level or complex high-level (fine-grained or course grained) functions
Source: UvA Grid Master IGC2005, Adam Belloum
Services are not Servers
• Web Servers (Apache, Tomcat, ISS) Talks HTTP(S) Hosts web sites and pages May be dynamic (.php, .asp, .jsp) Seen by people
• Web Services (Axis, GT4, .NET, …) Bind to a protocol (HTTP(S), SMTP, …) May be hosted in a container Have a syntax and semantics definition Are seen and used by programs
Defining and building Services in WS-*
[See Introduction to Web Services Tutorial]
1. Start with the WSDL2. Then, generate your interfaces
• E.g. with GT4 tools or JWSDP• See appropriate tutorial(s): “Technologies for
Building Grids” or “Sotomajor”
3. Fill the stubs with an implementation4. Deploy your services in a container
EGEE NA3 Training Home has plenty of material:http://www.egee.nesc.ac.uk/trgmat/
Part 2: Web ServicesCaveats and Deployment Hints
Hints, caveats and the constraints in Service Deployment in the VL-e PoC Environment
Some general caveats
• Re-use services• Use generic services where possible
‘job submission service’, ‘data location srv’• Don’t over-do it
WS are not suitable for low-latency HPC or bulk file transfer
• We’re not running a course on WS-RF …
• Not all services (nor architectures) are mature jet Be prepared to make compromises … … as long as it’s in-line with our long-term goals
• Test it works with your favourite workflow system!
Web Services hints
• Web Services are just a syntax
Define proper sementics and document that to make your service re-useable
Match up the semantics with related services
Make sure your services are compliant o with relevant standards (like WS-I Basic Profile)o with the chosen hosting system (GT4)
SOA Editor
Cape Clear SOA Editor, Cape Clear 2003WSDL © EGEE Consortium and partners, see www.gLite.org
Style
There are multiple ways to bind a portType• RPC/Encoded
In the WSDL: <soap:binding style="rpc"
transport="http://schemas.xmlsoap.org/soap/http"/>In Java, methods are like: public void method(String in1, BigInteger myNumber)
• Document/Literal (or wrapped)In the WSDL: <soap:binding style="document" ...>In Java, methods are like: public void method(PurchaseOrder po)
• Just because it’s no trouble: name the XML schema element referenced by an operation
(through a message element) the same as the operation itself This is the only requirement of document/wrapped which isn't
enforced by the WS-I requirements. Note that document/wrapped is a valid document/literal definition.
See also specific presentation on the web page on WDSL styles
Services Hosting
• Usually, a service lives in a container
• Container takes care of Translating incoming SOAP message to appropriate
objects (Java objects, C structures, …)
Enforcing access control Invoking methods Packaging the results Destroying the invoked objects
Containers
• Many containers, but not all the same
• Vary in what they support, e.g., Platform: JVM, .NET, Unix Language (Java, C, Python) WSDL bindings (rpc or document/literal) Security mechanisms (none or GSI)
• VL-e supplied container GT4 (Axis 2.0-RC2++) Modified to do WS-RF, -Addressing, -Notification With GSI Security framework Almost like standard Axis 2
Implementation
• Remember that web services are stateless
Model state via Resources Don’t keep running after the service invocation is
complete Use database/memory based storage, not process-
based state retentiono Much more friendly on the CPUo Resilient to machine/VM restarts
The ultimate horror scenario: R-GMA producer & consumer threads, see Concurrency P&E paper on the web
Security model
• We use SOAP over TLS This is the default in GT4 orders of magnitude faster than MLS but cannot do secure message forwarding
PoC Environment
Hardware and Software
RP, the P4CTB and the PoC
Application development
NL-Grid production clusterCentral mass-storage facilities
+SURFnetInitial compute
platform
Stable, reliable, testedCert. releases Grid MW & VL-
software
VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping Environment
DAS-2, local resources
VL-e Certification Environment
NL-Grid FabricResearch Cluster
Test & Cert.Grid MW & VL-software
Compatibility
Flexible, test environment
Environments
Usage
Characteristics
Virtual Lab.rapid prototyping
(interactive simulation)
Flexible, ‘unstable’
Download RepositoryPoC InstallerCluster Tools
Developer CVSNightly builds
Unit testsstable, tested
releases
Integration testsFunctionality testsAdventurous application people
PoC Release n Release Candidaten+1
Developers environment
Tagged Release Candidates
RapidPrototypingEnvironment
PoC: Hardware and Software
• The PoC Distribution is set of software• The PoC Environment is the ensemble of systems
running the PoC Distribution• The PoC Central Facilities (“CF”) are those systems
managed by the P4 partners and designated as such
PoC Software Distribution
OS Red Hat Enterprise Edition R3
PoC installer
Middleware Globus Toolkit version 2.x as supported by VDT
Current grid services from LCG, EGEE
Globus Toolkit 4.0 (WSRF components)
Storage Resource Broker (version 3.3.1)
• Operating system supported for all operations: RHEL3 & compatibles will work on a variety of similar systems
• Middleware: Services & Hosting Environments
PoC Software Distribution
Integrative Three systems Kepler, Taverna, Triana
OGSA-DAI as installed with GT4
IBIS (version 1.2)
Tools & Libraries ParaView (version 2.2.1) VTK (version 4.4) MESA (version 6.4 with GLUT)
ITK (version 1.4)
FSL (version 3.2)
MRICRO (version 1.39)
java SDK 1.4.2.x deployment/install
octave (version 2.1.69) MPItb (v2.1.6x) MatLabMPI (v1.2)
lamMPI (version x.x.x)
LUCENE (version 1.4.3)
Ant (version 1.6.2 )
SWI-Prolog (version 5.4.7-190)
R
Grid Weka (version 3.4.2)
Nimrod client software (v3.0.0)
GAT
Sesame (client)
PoC Central Facility topology
Service Deployment constraints CF
• Farm of GT4 containers provided on the CF Based on the concept of “VO Boxes” For now: login to these boxes via gsissh Not for compute-intensive work
• You cannot run a container continuously on worker nodes No inbound IP connectivity Resource management and utilization
• And you should not want to, because All services in a container content for CPU and (disk)
bandwidth JVM does not insulate services from each other
Other constraints on the CF
• Worker nodes are allocated in a transient way
Jobs run in a one-off scratch directory No shared file system for $HOME
(and if it happens to be there don’t use it!) Jobs will be queued
o Short jobs get to run fastero Small jobs get to run fastero Priority depends on your VO affiliation
You can connect out, but cannot listen Your job is limited in wall time
Central Hosting
• A common hosting environment is offered on a set of VO Boxes Provided: GT4.0 WS (as specified for R1) Visible from the outside world Highly monitored systems Can talk to the ‘inside’
• Requirements for deployment on the CF Appropriate AuthN/AuthZ must be used for all
services(this is available by default in GT4)
Allow for request tracability (e.g. via log files)
High performance/throughput Services
‘I need lots of * for my service’• Parallelize it
Use fine-grained services decomposition Use MPI (e.g. via IBIS) Use dedicated programs for processing Submit via a resource broker
to the compute service
• Submit jobs to the compute service to run dedicated GT4 hosting environments with your service Keep in mind to submit these to the cluster where
your front-end is running on a VO Box
Complex Services: job submission
• Use a job submission service GT4 WS-GRAM not (yet) supported on CF:
only single-cluster, security model not complete without use of the WSS
gLite CE WS (“CREAM”): also single cluster, security=compatible, deployment in 2006Q3
Brokered submissiono Select appropriate resources via GLUE schema
constraints & JDLo Submit directly via RB API code on client sideo Use RB job submission WS by Bart
Brokered Job Submission Service
• Submit brokered jobs in JDLthrough WS-interface to the Matrix Resource Broker
• WSDL specificationhttp://mu7.matrix.sara.nl/SaraGridService/SaraGridServiceLCG.wsdl
• For detailed info: ask Bart Heupers• Future version of the basic Grid middleware will
have a native WMS WS interface
Questions and Discussion!
http://poc.vl-e.nl/events/poc-ws-induction-20060221/