exhaustively test any basic ui or api in 30 minutes: the ideal for any organization serious about...

Exhaustively Test Any Basic UI or API in 30 MinutesThe Ideal for Any Organization Serious About Continuous Delivery

Benjamin Johnson-Ward

DevOps: Continuous Delivery

CA Technologies

Presales Consultant

DO4X85E

@GridTools

#CAWorld

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type of

warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.

For Informational Purposes Only

Terms of this Presentation


Abstract

Testing bottlenecks and the late detection of defects make Continuous Delivery impossible. This talk will examine how this can be resolved, making the exhaustive testing of any basic UI or API possible within 30 minutes.

Benjamin Johnson-WardCA Technologies

Presales Consultant


Agenda

COMMON TESTING CHALLENGES

BREAKING DOWN THE TESTING INTO COMPONENTS

DEMO & SUMMARY

DEMO: FORMERLY MODELLING EXISTING TEST ASSETS

DEMO: AUTO-GENERATE TEST CASES & SCRIPTS

STORING TEST COMPONENTS AS RE-USABLE ASSETS

1

2

3

4

5

6


Why Bother?

Over 60% of IT projects fail1

And 31.1% will be cancelled before completion2

Only 16.2% are completed on-time and on-budget

While testing takes half the time in the SDLC

Testing can eat up to half of the budget3

$312 billion a year is spent on software bugs4

1 & 2 – Standish Chaos Manifesto 20143 – Bloor, 20144 – Cambridge University Judge Business School, 2013


Design Dev. Test. Automation Deploy

A Typical SDLC: Too Manual, Too Linear, and Too Slow


Requirements are Practically Static, are Ambiguous, and are Incomplete.


The Results are Often Poor.

Over 50% of defects are introduced in the design phase


Manual Test Case Design

Currently manual – a time consuming, error-prone process

Is unsystematic, ad hoc, and has no real notion of “coverage”

Over-testing and under-testing – 10-20% coverage with 4 times over-testing

No linkage to test data – process is manual, painstaking and very time-consuming

No flexibility for change requests: a critical weakness in an agile or Continuous

Delivery environment. Changes take longer than the original requirement!


Why Automation is Not a “Silver Bullet” for Testing

Automated testing frameworks are heavily scripted

Script generation is usually done manually

As is script maintenance

Alternative solutions, use: Record Playback

Use script-less automation frameworks (keyword)

But you’re back to Manual test case design


Test cases and scripts are created automatically from “Active” requirements

They are executed automatically

Testing is Model Based for maximum coverage

Use a Component Library of common and optimized tests

Test Data and Virtual End Points are created or found as part of the Automation

Automating the automation

Active Automation

Active Automation


Active Automation with CA Test Case Optimizer

Increase the level of automation

Re-use and standardize common test components

Adapt quickly to change

Maintain high levels of coverage

Support Continuous Development frameworks


Model Based Testing – Step by Step


Linked Processes – Apply the Same Technique


Model Based Testing – Step by Step

CUSTOMER NUMBER:

CUSTOMER NAME:

CUSTOMER COUNTRY:

DATA UPDATED

123456

HUW PRICE

UK


Validation CoverageALL POSSIBLE ALL PAIRS ALL EDGES


Model Based Testing – Process CoverageALL POSSIBLEALL PAIRS

ALL EDGES & PATH HINTS

THE DECISION MODEL


Model Based Testing – Use Libraries of Test Cases


Active Automation with CA Test Case Optimizer

Increase the level of automation

Reuse and standardize common test components

Adapt quickly to change

Maintain high levels of coverage

Support Continuous Development frameworks


Test cases and scripts are created automatically from “Active” requirements

They are executed automatically

Testing is Model Based for maximum coverage

Use a Component Library of common and optimized tests

Test Data and Virtual End Points are created or found as part of the Automation

Automating the automation

Active Automation

Active Automation


Automated Tests: Manual Generation

Automated testing frameworks are heavily scripted

Script generation is usually done manually- As well as the maintenance

Alternative solution: Use scriptless automation frameworks (keyword)

– But you’re back to Manual test case design

Optimal solution: From automated test case generation -> Through automated script generation-> To automated execution

- Automating the automation


Q & A


Recommended Sessions

SESSION # TITLE DATE/TIME

DO4T15SCase Study: Nordstrom Takes Retail Relief for Test Data

Management 11/18/2015 at 11:30 am

DO4T05SVision & Roadmap: The CA Roadmap for Test Data

Management11/18/2015 at 1:00 pm

DO4T26TTech Talk: Fines in the Millions Levied Every Year Coming

Soon—The Business Case for Synthetic Data Generation11/18/2015 at 2:00 pm


Must-See Demos

Test Data Manager

DevOps

Theater 4

DevOps Sim

DevOps

Theater 3

Test Case Optimizer

DevOps

Theater 4

TDM Integrations

DevOps

Theater 4


Follow On Conversations At…

Smart Bar

DevOps

Theater 4

Tech Talks

DevOps

Theater 4


For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15

http://cainc.to/Nv2VOe

http://bit.ly/1wbjjqX

exhaustively test any basic ui or api in 30 minutes: the ideal for any organization serious about...

Technology