steps towards model-based, inference-driven soa testing

simplengineering Service Oriented Architects

November 30, 2011

Steps towards model-based, inference-driven SOA Testing Ariele P. Maesano¹ ², Fabio De Rosa ², Libero Maesano ², Perre-Henri Wuillemin¹ {Ariele.Maesano, Pierre-Henri.Wuillemin}@lip6.fr {ariele.maesano, fabio.de-rosa, libero.maesano}@simple-eng.com

¹ Laboratoire d’Informatique de Paris 6 (UPMC), Paris, France ² SIMPLE ENGINEERING, 31-35, rue St. Ambroise, 75011 Paris, France


30/11/2011 - 2

Table of contents

1. SOA Functional Testing

2. Black-box and Grey-box Test

3. SOA Testing Environment

4. Model Driven Test Engineering

5. BN Inference Approach

6. Discussion & Future work

ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing


30/11/2011 - 3

Motivation In the next 20 years, tens, hundreds, thousands, … applications, systems, devices will be connected and will collaborate without human intermediation - that’s the Brian Arthur’s second economy *

Distributed enterprise architectures involving companies, government and for-no-profit organizations allow the automation of business processes that support daily activities

Service Oriented Architecture (SOA) is a design and implementation style that allows organizations to put in practice dynamic collaborations of loosely coupled systems, in order to achieve flexible, dependable and secure business process automation

SOA is the paradigm able to support the second economy

* W. Brian Arthur, "The second economy." McKinsey Quarterly, October 2011



30/11/2011 - 4

SOA conceptual framework The collaboration among systems is carried out through the exchange of services which are regulated by contracts

Service contracts are formal models of the service functions and of the provider and consumer interfaces and external behaviors (including security and quality of service aspects) Service contracts do not include any information about internals of the systems that comply with them - system implementations are black boxes, hidden and private A services architecture is a network of participant systems, playing roles bound by service contracts, that collaborate in order to achieve business goals



30/11/2011 - 5

SOA testing conceptual framework Validation question: "Are you building the right services architecture?", is translated in the question: "Are you formalizing the right service contracts and participants’ roles (combinations of contract parts)?"

Verification question: "Are you implementing participant systems right, i.e. in compliance with the validated service contracts and participants’ roles?“

A model-driven approach simplifies the validation and verification task:

once the service contract model has been validated, the remainder of the overall validation task is the verification of the correct application of the model mapping and transformation, until the implementation

The last verification question (about compliance between implementation and contract) “stays open”,

SOA Architects and Integrators cannot employ program static check methods or white-box testing

The only available verification methods are black-box testing of individual SUTs (System Under Test) and grey-box testing of interactions among SUTs of the Services Architecture Under Test (SAUT), where they are observable



30/11/2011 - 6

Probabilistic Inference Approach

The ongoing research on SOA testing spans three main areas: the automated generation of test cases from contract models

(positive, negative, quality of service…) the automation of the test execution and evaluation environments inference-based methods and tools in order to help Testers to

efficiently manage test campaigns

There are not yet applications of probabilistic inference to the management of SOA testing campaigns it appears to be adequate for discovery of failures and search and

identification of faulty participants in a services architecture, where the only available information is the match/mismatch between the actual external behavior and the expected one

Bayesian Networks for Services Architecture Testing (BN4SAT) project(*) is centered on the development of a Bayesian network (BN) inference approach and technology to services architecture testing management


(*) The project is conducted in cooperation between the Laboratoire d'Informatique de Paris VI (LIP6 - Université Pierre et Marie Curie - Paris VI - France), the Centre National de la Recherche Scientifique (CNRS - France) and Simple Engineering, a European group specialized in the design of services architectures. The project is partially funded by the Association Nationale de la Recherche et de la Technologie (ANRT - France)


30/11/2011 - 7

Black-box Test

Black-box tests are run against individual SUTs whose internals are hidden

Connections (channels) with other participants are not observed

Only the SUT responses to stimuli are observed


sd TestSuite_Wiring_003

usbg

:BankGateNode

usam

:AccountMngtNode

CheckBalanceResponder/inquiry(m-usbg-usam-001)

CheckBalanceRequester/inform(m-usam-usbg-002)

WithdrawalResponder/invoke(m-usbg-usam-003)

WithdrawalRequester/report(m-usam-usbg-004)




30/11/2011 - 8

Grey-box Test

Grey-box tests are run against a SAUT (network of SUTs)

SUTs are involved in an end-to-end interaction

Channels among SUTs are observed



usatm

:ATM_Node

usbg

:BankGateNode

usam

:AccountMngtNode

FindAccountResponder/inquiry(m-usatm-usbg-001)

FindAccountRequester/inform(m-usbg-usatm-002)

CheckBalanceResponder/inquiry(m-usatm-usbg-003)



CheckBalanceRequester/inform(m-usbg-usatm-004)

CheckCredentialsResponder/invoke(m-usatm-usbg-005)

CheckCredentialsRequester/report(m-usbg-usatm-006)

WithdrawalResponder/invoke(m-usatm-usbg-007)


WithdrawalRequester/report(m-usam-usbg-004)WithdrawalRequester/report(m-usbg-usatm-008)

CheckBalanceResponder/inquiry(m-usatm-usbg-009)



CheckBalanceRequester/inform(m-usbg-usatm-010)


30/11/2011 - 9

SOA testing environment

Testing and Test Control Notation (V.3)* environment

Arbiter, Scheduler, Test components as TTCN-3 scripts

Engine as a plug-in (or a “service”)


* http://www.ttcn-3.org/


30/11/2011 - 10

TTCN-3


deployment BankNet DeploymentTokens_1

usatm4ushwcinterceptor :

ATM4HW_Control

usatm4usbginterceptor

:ATM4BankGate

«SUT»

usatm :ATM_Node


ATM4HW_Control

usatm4usbginterceptor

:ATM4BankGate

usbg4usatminterceptor :

BankGate4ATMusbg4usaminterceptor :

BankGate4AccountMngt

«SUT»

usbg :BankGateNode

usbg4usatminterceptor :

BankGate4ATMusbg4usaminterceptor :


usam4usbginterceptor :

AccountMngt4BankGate«SUT»

usam :AccountMngtNode

usam4usbginterceptor :

AccountMngt4BankGate

usbginterceptor4usam:


usbginterceptor4usatm:

BankGate4ATM

ushwcstimulator:

HW_Control4ATM

usaminterceptor4usbg:


usatminterceptor4usbg:

ATM4BankGate

a4e :

Arbiter4Engine

s4e :

Scheduler4Engine

«executionEnvironment»

:TTWorkbench

usbginterceptor4usam:


usbginterceptor4usatm:

BankGate4ATM

ushwcstimulator:

HW_Control4ATM

usaminterceptor4usbg:


usatminterceptor4usbg:

ATM4BankGate

a4e :

Arbiter4Engine

s4e :

Scheduler4Engine

«TestComponent»

usaminterceptor :

AccountMngtInterceptorNode

«TestComponent»

usatmstimulator :

ATM_StimulatorNode

«TestComponent»

usbginterceptor :

BankGateInterceptorNode

arbiter :Arbiter

scheduler :Scheduler

e4a :

Engine4Arbiter

e4s :

Engine4Scheduler

:Engine

e4a :

Engine4Arbiter

e4s :

Engine4SchedulerAutomated test execution & evaluation environment


30/11/2011 - 11

SOA Testing in WS* platform

SAUT representation = extended UDDI V3 Data Structure + collection of WSDLs

UML Interaction (XMI) + collection of SOAP messages



30/11/2011 - 12

BN Inference Approach

Bayesian network inference is used as a failure discovering driver, by getting the next test case to run in

order to reveal a failure. As troubleshooting driver, by getting the next test to run in order to

locate faulty participants

BN4SAT engine integrates failure discovering and troubleshooting, by managing a well-adjusted dynamic test sequence (the strategy) that, with a minimum number of test case runs: reveals a maximum number of failures and locates a maximum number of faulty participants

The BN4SAT graph is built by the BN4SAT loader directly from: the SAUT deployment UDDI files the WSDL files for all the involved service contracts and the test suite files



30/11/2011 - 13

BN4SAT Metamodel The BN4SAT graph model has six Boolean stochastic variable

types: Entity - business organization, {notFaulty, faulty} Participant - participant system, {notFaulty, faulty} Port - participant required interface, {notFaulty, faulty} ActionType - type of communicative action (message, rpc, rpc reply), {notFaulty, faulty} Action (evidence): test case action {pass, fail} Transaction: test case {pass, fail}



30/11/2011 - 14

Discussion A BN for SOA testing can exhibit a very large size, and can be very dense, therefore with large conditional probability tables (inefficiency in time and space)

We have implemented and assessed different classical inference methods such as value elimination, lazy propagation, Shafer-Shenoy and proved that they are not appropriate to cope in a sustainable way with BN4SAT graphs

Inference by compilation: Bayesian network can be interpreted as a multi-linear function(MLF) and therefore implemented by an arithmetic circuit (AC)


SAUT + test suite specification

Bayesian network

Coarse Conjunctive Normal Form (Coarse CNF)

Refined Conjunctive Normal Form (Refined CNF)

deterministic Disjunctive Normal Negational Form

(d-DNNF)

Arithmetic Circuit

SAUT + test suite specification

Refined Conjunctive Normal Form (Refined

CNF)

deterministic Disjunctive Normal Negational Form

(d-DNNF)

Arithmetic Circuit

Darwiche “classical” method

BN4SAT method (Model-driven inference by compilation)

- No intermediary generation of BN - No CNF optimization step

- Refined CNF in linear time (experimental results) - Push the boundaries with size limitation


30/11/2011 - 15

Thanks

Questions ?



30/11/2011 - 16

SOA testing environment

Services Architecture Under Test - SAUT is a network of SUTs (System Under Test) connected by channels conveying communicative actions such as messages, remote procedure calls (rpc), rpc replies Channels may be observable (grey-box stance). Conversely, SUTs internals are never observable (black-box stance)


deployment BankNet Deployment


ATM4HW_Control

atm4bg :ATM4BankGate«SUT»

usatm :ATM_Node


ATM4HW_Control

atm4bg :ATM4BankGate

am4bg :AccountMngt4BankGate

«SUT»



bg4am :BankGate4AccountMngt

bg4atm :BankGate4ATM

«SUT»

usbg :BankGateNode

bg4am :BankGate4AccountMngt

bg4atm :BankGate4ATM

hwc4atm :HW_Control4ATM

«SUT»

ushwc :HW_ControlNode

hwc4atm :HW_Control4ATM

BankNet SAUT


30/11/2011 - 17

SOA Test Running

Each SUT, in a well established SAUT, is initialized with a text context (the SAUT state), that is coherent with the next selected test suite (i.e., a collection of test cases) is going to run A test run takes place when Tester submits one or more stimuli to one or more SUTs Tester compares the SAUT actual response - the exchange of actions between SUT's following the stimuli - with the expected response - the expected action exchange in a specified SAUT state A test matches (mismatches) if the actual response corresponds to (differs from) the expected one Test run has a verdict: pass - the test matches and there is no evidence of failure; fail - there is evidence of a failure; inconclusive - a decision cannot be reached; error - a run-time error has occurred in the test environment



30/11/2011 - 18

SOA Testing Environment Test Components: specialized components allow reaping SUTs actions, addressing actions to SUTs, comparing SUTs actions with the expected ones, formulating local verdicts and transmitting them to the Arbiter: Stimulators - send stimuli to one or more SUT, in order to trigger

test runs Interceptors - sit transparently between two connected

participants, intercept actions and perform different tasks Emulators - emulate the behavior of SUTs

Arbiter - the Arbiter collects the Test Components local test verdicts and produces final test verdicts Scheduler - the Scheduler drives the execution of the test runs Engine, which is notified by the Arbiter with test verdicts, manages the Scheduler by handling the test strategy on the basis of probabilistic inference on the acquired test verdicts



30/11/2011 - 19

Model-driven Approach

The starting point for testing a SAUT is in the Platform Independent Model (PIM), specifically, the collection of samples A sample is a modeled, schema-compliant occurrence of services interactions, that coordinates either the services deliveries (positive sample), or the services refusals, when the delivery conditions are not satisfied (negative sample) A sample is composed of a: sample interaction, a UML Interaction together with its

MessageTypes instances, and a sample context, a collection of domain objects (instances)

representing the states of the participants' resources in which the interactions take place

The sample collection is the starting point for the design and the implementation of test cases



30/11/2011 - 20

Test Engineering - Negative sample

CUSTOMER

customer#

ABCDEFG

CUSTOMER-NAME

customer# name

ABCDEFG Ariele

ABCDEFG Paolo

CUSTOMER-ACCOUNT

customer# account#

ABCDEFG 123456

ACCOUNT

account# currency state balance

123456 EUR active 900.00

Sample context (fact base)


usbg

:BankGateNode

usam

:AccountMngtNode


WithdrawalRequester/decline(m-usam-usbg-005)

class Sample oracle

«MessageType»

m-usbg-usam-003 :WithdrawalInv okeMessage

acc# = 123456

am = 1000,00

curr = USDollar

dateTime = 2011-06-15T11:28:38.100+01:00

«MessageType»

m-usam-usbg-005 :WithdrawalDeclineMessage

acc# = 123456

am = 1000,00

curr = USDollar

dateTime = 2011-06-15T11:28:14.109+01:00

code = NotEnoughBalance



30/11/2011 - 21

SOA Testing in WS* platform

The SAUT is translated into a set of WSDL files and an extended UDDI V3 Data Structure, in order to capture use links (the observable channels in a specific deployment scenario) - each participant system is represented as UDDI Business Service Sample collection is translated in a test suite (a collection of test cases), by the mapping and mechanical transformation of the PIM into the WS* Interoperability PSM A test case includes: a test case oracle - a XML infoset that represents the interchange of SOAP messages

between SUTs (resulting from the transformation of the sample UML Interaction), together with the collection of the corresponding SOAP envelopes (resulting from the transformation of the Message Type instances exchanged in the interaction)

a test case fact base (test context) - a relational database resulting from the object/relation transformation of the sample context



30/11/2011 - 22

SAUT Description

<?xml version="1.0" encoding="UTF-8"?>

<uddi:businessService businessKey="urn:My_US_BankNet" serviceKey="urn:My_US_BankNet:BankGate">

<uddi:name>BankGate</uddi:name>

<uddi:bindingTemplates>

<uddi:bindingTemplate bindingKey="urn:BankGate-AccountMngt:BankGate4AccountMngt:usbg4usam"

serviceKey="urn:My_US_BankNet:BankGate">

<uddi:description>urn:BankGateNode:usbg</uddi:description>

<uddi:accessPoint useType="endpoint">http://usbg.My_US_BankNet.com/BankGate4AccountMngt/usbg4usam</uddi:accessPoint>

<uddi:tModelInstanceDetails>

<uddi:tModelInstanceInfo tModelKey="urn:BankGate-AccountMngt">

<uddi:instanceDetails>

<uddi:overviewDoc>

<uddi:description>required</uddi:description>

<uddi:description>urn:IAccountMngt</uddi:description>

<uddi:description>urn:AccountMngtNode:usam:BankGate-AccountMngt:AccountMngt4BankGate:usam4usbg</uddi:description>

</uddi:overviewDoc>

</uddi:instanceDetails>

</uddi:tModelInstanceInfo>

</uddi:tModelInstanceDetails>

</uddi:bindingTemplate>

...

</uddi:businessService>

deployment BankNet Deployment

am4bg :AccountMngt4BankGate «SUT»



bg4am :BankGate4AccountMngtbg4atm :BankGate4ATM

«SUT»

usbg :BankGateNodebg4am :BankGate4AccountMngtbg4atm :BankGate4ATM

use links

UDDI v3 Business Service



30/11/2011 - 23

Test Case Oracle - Interaction sd TestSuite_Wiring_002

usbg

:BankGateNode

usam

:AccountMngtNode


WithdrawalRequester/report(m-usam-usbg-004)


<ss20tc:test-case xmlns:ss20tc="urn:simpleSOAD2.0:test-case" id="urn:InterExchangeNet:Scenario01:Wiring_001">

<ss20tc:sequence>

...

<ss20tc:message id="m-usbg-usam-003" sn="m033" from="urn:BankGateNode:usbg"

to="urn:AccountMngtNode:usam" mode="WithdrawalResponder/invoke"/>

<ss20tc:message id="m-usam-usbg-004" sn="m034" from="urn:AccountMngtNode:usam"

to="urn:BankGateNode:usbg" mode="WithdrawalRequester/report"/>

...

</ss20tc:sequence>

</ss20tc:test-case>

XML Infoset describing interaction among SUT



30/11/2011 - 24

Test Case Oracle - Envelope MessageType Instance


<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:ns="urn:ien:bank:services:withdrawal:wsdl:types"

xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing">

<soapenv:Header>

<wsa:MessageID>m-usbg-usam-003</wsa:MessageID>

</soapenv:Header>

<soapenv:Body>

<ns:WithdrawalInvokeMsg xmlns:ns="urn:ien:bank:services:withdrawal:wsdl:types">

<ns:sessionId>

<ns:account>123456</ns:account>

<ns:dateTime>2011-06-15T11:28:38.100+01:00</ns:dateTime>

</ns:sessionId>

<ns:currency>USDollar</ns:currency>

<ns:amount>10000,00</ns:amount>

</ns:WithdrawalInvokeMsg>

</soapenv:Body>

</soapenv:Envelope>

SOAP Message

class Sample oracle

«MessageType»

m-usbg-usam-003 :WithdrawalInvokeMessage

acc# = 123456

am = 1000,00

curr = USDollar

dateTime = 2011-06-15T11:28:38.100+01:00


steps towards model-based, inference-driven soa testing

Technology

model driven test

soa testing spans

service contract model

soa architects

contracts service contracts

testing environment4

modeldriven approach

right service contracts