© cgi group inc. confidential€¦ · (e.g. esb). in order to support complex data exchange...
TRANSCRIPT
© CGI Group Inc. CONFIDENTIAL
Testing of SOA based systems
Staged acceptance
Chris C. Schotanus
2
Introduction
CHRIS SCHOTANUS
Principal IT Consultant
REQUIREMENTS & QUALITY MANAGEMENT
41 YEARS OF IT EXPERIENCE
OF WHICH 34 WITH CGI (AND ITS ANCESTORS)
27 YEARS IN TEST AUTOMATION, TEST PROCESS
IMPROVEMENT & PROCESS ARCHITECTURE
3
Today’s Agenda
01 Introducing SOA
02 Test specific items
03
04
SOA governance aspects
The real life experience
SOA based systems defined
(Ideal) SOA based systems are composite systems made of parts that are
• Virtual: the consumer is not aware of the implementation of the service
• Standardised: there is only one implementation of a function or responsibility
• Modular (replaceable) and compositional
• Abstract: generic, not dedicated to one specific consumer
• Detached: consumer and supplier are fully independent of each others implementation
• Agnostic: there is no direct link or relation between services
4
These parts are called "services"
A service is a bundle of resources that belong to the same context and that are able to solve
operational problems and to support processes. Services are characterised by autonomy, loose
coupling, reusability and a service contract, which abstracts the service implementation
(technology and logic) via a well-defined service interface.
Types of service contracts:
• Basic contract: service operations incl. input/output parameters
• Behavioural contract: parameter combinations, pre/post conditions, service call sequences
• Synchronisation contract: simultaneously service calls
• QoS contract: non-functional requirements
5
SOA principals: service triangle
A service call involves the service provider, the service consumer and the service registry. All
information needed to perform a service call can be gathered during run-time („service binding“)
which enables SOA to allocate resources flexibly and dynamically.
6
Service registry
Service provider Service consumer
Service usage
SOA principals: service composition
Services can be categorised according to their functional stage of expansion:
• Basic services
• Composed services, using compositions of other services to perform a complex action
• Process services, representing long running business processes
Service composition can be achieved in two ways:
• Orchestration (service calls are coordinated by a central instance)
• Choreography (self-governed peer-to-peer communication between services)
Ideally, SOA implementations provide sophisticated workflow engines to allow
configuration of user-defined process definitions.
7
SOA principals: service mediation
When a service call is performed, the code of the service implementation is not integrated into the
service consumer‘s environment but remains and is run in the service provider‘s environment. The
interoperability issues which result from such an approach are addressed by a mediation system
(e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is
typically based on messages.
Examples of message exchange patterns:
• One way (Fire and Forget)
• Request/Response
• Request/Call-back
• Publish/Subscribe
8
9
What this presentation is about
Questions often asked regarding SOA are:
• Who will accept a services?
• How about ownership of services?
• What should we test if one or more services change?
• Interface testing versus end-to-end integration testing?
• Who owns the maintenance budget?
These are the questions I will try to answer
10
Today’s Agenda
01 Introducing SOA
02 Test specific items
03
04
SOA governance aspects
The real life experience
11
Testing services
Accept the product
Define user needs,
Business processes,
Requirements
The supplier/consumer model as reference
Design Build Test Deliver
Consumer
responsibility
Supplier
responsibility
Either Agile or Sequential Process
The SOA acceptance process
• User applications are assembled from or use (web)services
• Service requirements are derived from business requirements
• Services are selected from service registry
• Systems are assembled using services
12
It is a Staged Acceptance Process
Enterprise Messaging Service
The SOA specific test process
13
End user app
SOA Service SOA Service SOA Service
Governance
Operations
Business Owner
Assembly
Enterprise Messaging Service
This might as well be a hybrid or a full agile process like Nexus
14
End user app
SOA Service SOA Service SOA Service
Governance
Operations
Business Owner
Service Integration (Regression) Testing
What should we test if a service is changed?
End-to-End testing versus
Interface testing
15
Basic servicesS8 S9 S10
Cons 1 Cons 2 Cons 3
Orchestrated business
process
Orchestrated service
Data handling services
Business applications
(consumers)
S1 S2 S3
S4 S5 S6
S11 S12
S7
S13
DB2 IMSExternal
Service
What must be tested?
16
Basic servicesS8 S9 S10
Cons 1 Cons 2 Cons 3
Orchestrated business
process
Orchestrated service
Data handling services
Business applications
(consumers)
S1 S2 S3
S4 S5 S6
S11 S12
S7
S13
DB2 IMSExternal
Service
Just the ones directly involved
17
E2E borders
Stubs, Virtual services or
Services running in test
environment
Pro’s and cons
Pro: Less effort, relatively easy to manage
Con: Perhaps less insight in risks
18
Pro: Security, certainty
Con: Difficult to manage, expensive
19
Ideally, in “real SOA”
there should be
no need for
end-to-end testing!
End-2-end testing: where do we stop?
• Chain testing is expensive and complex
• Stakeholders define the product risks
• Only high risk systems justify the effort for chain testing
• In other cases stick to interface based testing!
20
Product Risk Assessment as starting point
Test environments and test data management
21
How to deal with connections to non-existent services
or services that are under change
Mock services (Stubs and Drivers )
• Specific pieces of software
• Simulate behaviour of services not (yet) available
• Allow for isolated testing of services
• Divide the chain into manageable parts
• Are adaptable to the wish of the tester
• Provide good control of data
• Good and fault situations are predictable
22
Driver
Testobject
MockMocks are interchangeable with virtualization tools
• Every service can virtually be
available
• Flexible and adjustable test data
• Maintained by testers
• High degree of reuse
But:
• Relatively expensive (additional
tooling required)
• Additional (functional) management
• Knowledge and training needed
• Initially cheap
• “Tailor made”
• Rapidly deployable
• Risk of expensive maintenance
But:
• Made by developers
• Additional, non-standard
maintenance
• Additional communication
tester <> developer
• Inflexible
• Poorly reusable
Stubs or Service virtualisation?
23
Mocks Virtual Services
Alternative: Several test environments
24
Production
Research
Copy of
production
Component test
of isolated
services by
system
developers
System test
of isolated
services against
functional and
non-functional
design
Service
Integration test
Acceptance of
individual
services
Test AcceptanceIntegration
TestDevelopment
Development
User Acceptance
Test
Of integrated
system
Service development(internal & external)
Consumer IT Business Operation
Actual use in live
environment
Role of the Test manager
• Is responsible for all test activities
• Compiles a test strategy based on product risks analysis
• Defines requirements to test environments
• Formulates demands on service delivery
25
Test manager is often also "Integration Manager"
First level Acceptance
• Does the service meet the requirements as stated in the repository?
• Is backwards compatibility guaranteed?
• Can the service safely replace the one that's already in the registry?
26
Fist level acceptance by the service owner
27
Today’s Agenda
01 Introducing SOA
02 Test specific items
03
04
SOA governance aspects
The real life experience
SOA Governance
SOA governance is a concept used for activities related to exercising control over services in
a service-oriented architecture (SOA).
IBM and others, state that SOA governance is an extension (subset) of IT governance which
itself is an extension of corporate governance.
Source: https://en.wikipedia.org/wiki/SOA_governance
SOA governance refers to the processes used to oversee and control the adoption and
implementation of service-oriented architecture (SOA) in accordance with recognized practices,
principles and government regulations. SOA governance provides optimum service quality,
consistency, predictability and performance, ensures that personnel follow prescribed policies and
corrects system problems or policy infractions as they occur.Source: https://searchmicroservices.techtarget.com/definition/SOA-governance
28
Service repositories and service registries
29
Source: Boris Lublinsky - Explore the role of service repositories and registries in Service-Oriented Architecture (SOA) IBM-2007
A service repository is the foundation of enterprise SOA
governance, supporting centralized management of all of
services-related information, including design,
implementation, and usage artefacts.
A service registry is the foundation of service location
virtualization, allowing for centralized control over location
and invocation policies for all of the enterprise services.
Two important aspects of SOA Governance
Use of registry and repository
30
Defining, redefining
Services and meta data
modelling
Services
Developping and testing
Services Rolling out
Services Making up
Services
Design time Run time
Modifying, updating
Services and meta data
Supervising
Services
Executing
Services
RegistryRepository
31
Stakeholders and the service repository
The details about your services and the services you use:
• Functionality
• Availability
• Capacity
• Use
• Service owners
• Service Sepcifications
SLA between suppliers and consumers
Business model
32
who will pay for the changes and test them if one consumer needs them
and others don’t.
Passenger Cargo
Business
IT
Agreement on
use and
maintenance
Req
uest
33
Today’s Agenda
01 Introducing SOA
02 Test specific items
03 SOA governance aspects
04 The real life experience
Testing of services in a live environment
The generic presentation of components in an Enterprise Service Bus (ESB)
System A PES EMS CES System B
A: Provider,
source of information
PES:
Providing Adapter
“Translates” A into CDM
According to specs in a mapping sheet
EMS:
Enterprise Messaging
Service
CES:
Consuming Adapter
“Translates” CDM into B
According to specs in a mapping sheet
B: Consumer,
Uses the information
Interface format
as used in System A
Common Data Model
format and behaviour
Standards on EMS Interface format
as used in System B
Service
Mapping A to CDM Mapping CDM to B
Testing of services in a live environment
Multiple consumers per service
System A PES EMS
CES System B
B: Consumer,
Uses the information
Service
Mapping A to CDM
Mapping CDM to B
CES System C
C: Consumer,
Uses the information
Mapping CDM to C
System development related to SOA
Client
needs
ES
B
Inte
gra
tion
Ce
ntr
e
Pro
ject
Exte
rna
l
Pro
gra
mm
ing
pa
rty
Integration
Req’s
Integration
Architecture Design
Intake &
Acceptance
Service Integration
Test
Business
Acceptance Test
Functional and Non-
functional Test
Detailed design &
Build
Delivery
Test
Architecture
& Design
Non-Integration
Req’s
Non-Integration
Development
Functional and Non-
functional Test
Testing of services in a live environment
37
System A PES EMS CES System B
Adapter
system test
Adapter
system test
The developing party in their own SOFa in development environment
ESB Integration Centre in their own Test environment
EIC Delivery test
The client IT department in the Integration Environment (T)
Service Integration Test
End users in the Acceptance Envrionment (A)
Business Acceptance Test.
The Test Process
Generic Test Agreements
Overall test strategy and agreements
• Clear agreements on who test what and when
• Overall acceptance criteria per test phase
• Handover criteria between test phases
Like any test strategy: It prevents "I did not mean or expect that" discussions
38
Summary: testing in SOA environments
39
Accepting parties
▪ The users accept the application
▪ The service-owner accepts
individual SOA services on
functionality and capacity
▪ Operations accepts SOA services
op load, stress and behaviour
Integration testing
▪ Preferably in isolation
▪ End-to-end if stakeholders
consider it necessary and product
risks indicate to
Regression testing
▪ All owners of services that
communicate directly with the
changed one
▪ Owners of the changed services
▪ Operations
▪ Supervising the test process
▪ Advising the stakeholders
▪ Integration of services
The test managers role