it's the 2010's. why are you still manually writing tests?

1

Upload: ca-technologies

Post on 15-Feb-2017

263 views

Category:

Technology


1 download

TRANSCRIPT

Pre-Con Education: It's the 2010s. Why Are You Still Manually Writing Tests?

Chris Kraus

DevOps: Continuous Delivery

CA Technologies

Product Manager

DO4X81E

@ChrisKraus3

#CAWorld

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

For Informational Purposes Only

Terms of this Presentation

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies. The presentation provided at CA

World 2015 is intended for information purposes only and does not form any type of warranty. Some of the specific slides with customer

references relate to customer's specific use and experience of CA products and solutions so actual results may vary.

Certain information in this presentation may outline CA’s general product direction. This presentation shall not serve to (i) affect the rights

and/or obligations of CA or its licensees under any existing or future license agreement or services agreement relating to any CA software

product; or (ii) amend any product documentation or specifications for any CA software product. This presentation is based on current

information and resource allocations as of November 18, 2015, and is subject to change or withdrawal by CA at any time without notice. The

development, release and timing of any features or functionality described in this presentation remain at CA’s sole discretion.

Notwithstanding anything in this presentation to the contrary, upon the general availability of any future CA product release referenced in

this presentation, CA may make such release available to new licensees in the form of a regularly scheduled major product release. Such

release may be made available to licensees of the product who are active subscribers to CA maintenance and support, on a when and if-

available basis. The information in this presentation is not deemed to be incorporated into any contract.

3 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Abstract

What's in a test? We invoke an Application Programming Interface (API)

test hoping to validate business logic and update the system of record,

but why is it so hard to find those systems of record? This introductory

session will explain and demonstrate how CA Continuous Application

Insight (CA CAI) can observe a manual test and then generate tests for

web UI APIs by creating the database validation steps for you. We will

also show you how CA CAI can be used to generate 1000’s of tests vs.

spending days getting the validation right for a single manual test!

Chris Kraus

CA Technologies

Product Manager

4 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Agenda

WHERE ARE THE API’?

FINDING THE APIs WITH CA CAI

WHAT ARE BASELINE, DATA DRIVEN, AND EXPANDED TESTS?

HIGH FIDELITY TESTS – REMOVE HUMAN EDITING OF API TESTS

DEMO

1

2

3

4

5

5 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Where Are the APIs – I Know I Saw Them on a Whiteboard

6 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Finding the APIs

ESB

Mobile applications and

Rich browser applications

Services provided by the ESB

Data being serviced up by

ESB

7 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

What Does CA Continuous Application Insight Consider an API?

Broad definition of “service” or API– Requests data from a app server where

we have a agent

– Lines between agents that follow the transactions

– REST and SOAP requests for data from browser application

8 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

CA Continuous Application Insight Common API’s button

A Common API is one that more than 1 agent has consumed

Focus the analysis on 1st call into the application not the buried “shared code”

The transactions are now from different consumers perspective

9 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

What Are These Tests Generated by CA Continuous Application Insight?

Baseline Test (Expanded)– A baseline test is the most simple replay a observed requests

– Validate the same response code returned

– Graphical Side by Side diff on XML

– A separate test is generated for each observed request and response

– A suite is generated to run all of the tests

– A alternate.config is generated for each baseline

IS THE SYSTEM BEHAVING THE SAME WAY IT DID IN THE PAST?

10 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

11 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

What Are These Tests Generated by CA Continuous Application Insight?

Invoke and Verify Test (Expanded with database validation)– Transaction path can traverse multiple agents

– Single step test 1 invoke and DB steps follow

– Database Validation generated to ensure SOR has been updated

Insert

Update

Delete

DID THE SYSTEM OF RECORD GET UPDATED? DID THE ERROR GET HIDDEN BEHIND THE UI?

12 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

13 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

What Are These Tests Generated by CA Continuous Application Insight?

Data Driven tests – Same as Baseline, but a Large Data Set (LDS) is used to drive test

– When you need to modify the generated test use this – 1 test many rows of data

– Refresh the data with RR pair generation

IS THE SYSTEM BEHAVING THE SAME WAY IT DID IN THE PAST? – BUT I NEED TO UPDATE TEST

14 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

15 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

What Are These Tests Generated by CA Continuous Application Insight?

Functional Test Cases (tests generated from document transactions)– The tests follow the API’s of the user’s journey through the application

– Tests contain multiple steps in the order a user ran them

– High Fidelity test generated

Datasets for input data

Filters for passing data between steps

– Supports invoke and verify (database validation)

API TEST FOLLOWS THE USERS JOURNEY THROUGH THE APPLICATION

16 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

17 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

What Are These Tests Generated by CA Continuous Application Insight?

Functional Test Cases from Browser (Chrome Browser plugin -no agent required)– The tests follow the API’s of the users journey through the application

– Tests contain multiple steps in the order a user ran them

– High Fidelity test generated

Datasets for input data

Filters for passing data between steps

– Supports invoke and verify (database validation) when agent is instrumenting backend

API TEST FOLLOWS THE USERS JOURNEY THROUGH THE BROWSER

18 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

19 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

20 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

High Fidelity Tests – Remove Human Editing of API Tests

Externalize connections to SUT – alternate.config

Create datasets for test input so data can be changed and expanded

Filter response data and parameterize into the next step

Reverse back JDBC calls into validation steps

Test Templates – provide test to handle login, sessions, and customization

21 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Demos

Baseline Test FidelityInvoke and Verify

Functional Test

22 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Q & A

23 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Recommended Sessions

SESSION # TITLE DATE/TIME

DO3T06SThe CA Roadmap for Service Virtualization and the Dev

and Test Portfolio11/18/2015 at 2:00 pm

DO3T38TTech Talk: Extreme Automation—Creating Headless Tests

“Automagically”11/18/2015 at 3:00 pm

DO4T27TTech Talk: Building Regression Tests to Increase Velocity and Prevent Things from “Going Boom” in the Night

11/19/2015 at 11:30 am

24 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Must See Demos

Application Test DemoCreate Better Application Tests

Theater 4

DevOps Sim

DevOps Simulation Experience

Theater 3

Continuous App InsightAutomate Tests for Better Apps

Theater 4

Service VirtualizationDeliver Better Apps Faster with SV

Theater 3

25 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Follow On Conversations At…

Smart Bar

Theaters 3 & 4

Tech Talks

Theaters 3 & 4

26 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15