model based test automation - capgemini sogeti … · model-based test automation reporting &...

© 2014 Tricentis GmbH. All rights reserved. 1

Accelerate. Innovate. Automate.

1 © 2014 Tricentis GmbH. All rights reserved.

Model Based Test Automation Avoiding the maintenance trap Hans Planken

Vice President Alliances EMEA

January 2015


Automation level below 25% even

after 30 years of software testing.

Why?

Do we need a different approach?

Agenda


Evolution of Complexity - Testing Challenges

COM_B18

Mainframe Client/Server Web Mobile

Complexity


What’s the problem? A typical test project.

Source: TRICENTIS® customer projects

Total Cost and Effort

100%

0%

50%

Test Case

Specification

Manual Testing

(Execution)

Automated Testing

(Setup, Maintenance

and Execution)

If you don’t know how much

test coverage you have,

you’ll end up with far too

many test cases!

Not effective enough

Automation levels are low

because of it’s difficult to

automate!

Not efficient enough

Risk Coverage

?


Tricentis Tosca Testsuite

Virtualize – Orchestrated Service Virtualization

Technology Integration

Optimize

Risk Coverage

Optimizer

Manage

Test Data

Provisioning

Automate

Model-Based Test

Automation

Target Applications

Reporting & Dashboard


Virtualize – Service Virtualization & Orchestration

Manage

Test Data

Provisioning

Automate

Model-Based Test

Automation



Aligns business priorities to test planning & execution

Generates the fewest test cases to maximize risk coverage

Shifts focus from Test Coverage to Risk Coverage

Optimize

Risk Coverage

Optimizer


Source: Customer Projects TRICENTIS®

Total effort/costs

100%

0%

50%

Test Case

Specification

25%

Manual

Test

Execution

75%

Automation

Finding

and

Preparing

Test Data

Execution

of

Test Cases

50 - 75% of manual

execution costs! insufficient control of

test data

Synthetic Test Data Engine



Automate

Model-Based Test

Automation


Optimize

Risk Coverage

Optimizer

Manage

Test Data

Provisioning


Provides tailored test data basis for test execution

Reduces effort in manual test execution by 40 – 50 %

Stateful test data management is key for high automation rates




Manage

Test Data

Provisioning

Optimize

Risk Coverage

Optimizer

Automate

Model-Based Test

Automation


Enables extreme automation (rates > 90%)

Covers UI and nonUI test automation

Enables seamless transition from manual to automated testing


Tosca Testsuite – Robust, Cost-Effective, Scalable

Risk Management

Engine: Higher Coverage,

Fewer Tests.

Model-Based Test

Automation:

Robust

Cost-Effective

Scalable

What makes it

different?


Model-Based Automation

Model-Based Test Automation

Robust, Cost-Effective, Scalable


What keeps us from higher

automation levels…

• Prohibitive Maintenance

of Test Scripts

• Missing or inadequate

Object Recognition

• GUI and nonGUI tests require

different teams/skills…

• Autom. Tests require dynamic

and thorough Test Data

Efficiency: Why

are automation

levels so low?

What is the Problem?

MBT_B0

1


The Evolution of Test Automation

Record/

Replay

FRAGILE

1st Generation

Robust

Cost

Effective

Scalable


The Maintenance Problem with Capture/Replay

Example:

“…select invoice with the car registration number VIC-3456”

CheckBox Click,

“/usr/cntlCONTAINER/

shellcont/

shell[2]/chbx[1,3]”

Does your business team understand this command?

Is this command stable?

Conclusion: Capture/Replay does not work!

Checkbox Name Reg Number Invoice

John Smith NSW-1234 90035560

Jim Courier NSW-2345 90035561

John Cook VIC-3456 90035562

Tim Rebhorn VIC-4567 90035563

Dale Howard WA-5678 90035564

Frank Rose WA-6789 90035565


The Evolution of Test Automation

Record/

Replay

FRAGILE

Test

Automation

Frameworks

EXPENSIVE

Model-Based

1st Generation 2nd Generation Next Generation

Script-Based Model-Based

Robust

Cost

Effective

Scalable


• Cognitive Test Cases: business language - not scripts!

• Requires only business SME skills

• Highly stable from a technical as well as from a business point of view

• Independent from technologies, HTML, Java, .NET, SAP, Manual execution, …


CheckBox Click,

“/usr/cntlCONTAINER/she

llcont/

shell[2]/chbx[1,3]”

Example:

Do you understand the business instruction?

Do you remember it?

“…select invoice with the car registration number VIC-3456”


Model Based Test Automation

How does it work?


Model-Based Automation

UI Technology

System Under Test

Model(s)

Test Cases

decoupling

Action Action

non

UI

Business


W/S

OneView: UI/nonUI; Robustness over Change

Model

Systems HTML JAVA .NET HOST SWIFT XML SQL

ST SIT (E2E) UAT CT

SCAN &

SYNCHRONIZE

Test Cases in plain English

CHANGE

SYNCHRONIZE


Consistent User Experience & On-Demand Staffing

TOSCA User Interface

Base Engine decouples

DB

We

b S

erv

ice

s

SW

IFT

Die

se

l

Gas

…

50

0 H

P

Pe

tro

l

GW

T

HT

ML

Test Logic

Technology


Broad coverage or technologies & applications

TOSCA allows for easy tests across any technology and/or platform!

Ad

ap

ter

Ad

ap

ter

Ad

ap

ter

Ad

ap

ter

… > 30 technology specific adapters …

Model Based Test Automation

Customer-specific

extensions

Te

ch

no

log

ies

SA

P

32

70

, E

mu

lato

rs

xB

row

se

r

Ja

va

Cen

tura

/Gu

pta

Win

32

, C

++

, V

B

Po

we

rBu

ilde

r, D

elp

hi

PD

F-D

ocu

me

nts

Sie

be

l, O

racle

Wo

rd, X

L, O

utlo

ok

.NE

T, W

PF

Data

ba

se

s

…

Ava

loq

We

bS

erv

ice

s, X

ML

Mo

bile

RE

ST

JS

ON


Traditional Automation

Traditional automation from a manual to an automated

Test Case…

Manual (Prose)

Automated (Code)


Only Tosca offers seamless test automation

Model

You decide!

Traditional testing: two sets; typically out of sync!


Finding

and

Preparing

Test Data

Execution

of

Test Cases

Source: Customer Projects Tricentis®

Total costs

100%

0%

50%

Test Case

Specification

25%

Pre Tosca

Cost

Manual

Test

Execution

75%

Automation

Test Case

Spec.

Test

Automation

Manual

Testing

Test Case

Design

Risk-Based

Test Structure

Post Tosca

Cost

The Bottom Line

Poor automation coverage due to

(i) maintenance trap

(ii) insufficient control of test data

model based test automation - capgemini sogeti … · model-based test automation reporting &...

Documents