An agile journey with Nordea - a case study

And then a reflective discussion

My Background ”Why I'm here”

Dan Borring Larsen

Sr. Project Manager, Consultant & Agilist

Working IT industry since 1995 – roles from System

Specialist to Test Manager and final Project Manager

Specializing in helping companies in Agile/QA best

practices (deliver value early). Experience with both

frontend and backend software development on

various platforms. Using the whole-team approach

onshore/offshore. Domain knowledge on investment

banking and business administration. Current assigned as an IT Project

Manager at Nordea in the SWO

department (Christiansbro).

• 04/12/2013

3 •

http://www.linkedin.com/in/borring

dan@bettersoftware.dk

THE MISSION ”what nordea wanted”

• 04/12/2013

5 •

High Quality

High Test awareness

High Process awareness

Fast Time-to-Market

Monthly deliveries

Resource Scalability

Offshore development

Case facts

• 04/12/2013

6 •

The Product (classified) An online solution for Nordea’s customers,

and is launched this year

Project size (IT) 7+ local resources

16 offshore resources

Technology Responsive web design

HTML5, Bootstrap, Less, Apache Wicket, Java

Test Tools Cucumber, Selenium, HP Quality Center, Adobe Edge Inspect,

CA-LISA , Jenkins, Sonar Qube

DAY ONE ”does this boat cross an ocean”

The boat

• 04/12/2013

8 •

Master Test Plan

Master Test Plan

Requirements

Design

Code

Test

Trying to test everything manually

..and way too late!

THE VISION ”rethink”

The Test Vision based on an Agile Approach

• 04/12/2013

10 •

Chapter 16 Quality

Integrate Testing into the Process

Mike Cohn

Test Vision

Automated Unit tests

Automated

Integration tests

Automated

Functional test

UAT

• 04/12/2013

11 •

Test Automation Tailoring the Test vision with the Master test plan

Manual

UAT

Automated Unit Tests

Automated

Integration Tests

Automated

Functional Test

Requirements

Business Sign-off

Test

Resu

lts

Test automation

Manual Test

- Look & feel

(Every 30 minute)

Exploratory

The Test Vision

• 04/12/2013

12 •

The Test Vision Test Vision

Test Objectives • Test more in lesser time

• Test early

• Deliver fast

• Predictable value

• High customer satisfaction

• Fun at work

Test Strategy • Evolution

The Test Vision Master Test Plan

Working quality process over comprehensive documentation

THE CREW ”bring in the right skills”

Test Team

• 04/12/2013

14 •

New competencies

• Test execution by Scripts instead of Keyboards

• From Functional Test to digging deeper (yes unit test)

Test Scripting &

Integration Test

(4 resources)

Business UAT

(2 resources)

Manual Functional Test

(2 resources)

OJT

DIVING ”going deeper”

What is a Feature?

• 04/12/2013

16 •

Behaviours written in the form of Scenarios

Written in simple English and written by non technical people

Feature: Signing up for TextExpo [title]

As a tester [role]

I want to attend TestExpo [feature]

So that I get inspired for my work [benefit]

Scenario: Signing up successfully

Given I go to www.sogeti.dk/Events/TestExpo [context]

When I click the button labelled 'Tilmeld dig her’ [event]

Given there are <free seats> seats [another context]

Then I should see a form with fields: Navn | E-mail | … [outcome]

And I fill in Navn | E-mail | … [another context]

And I click the button 'Tilmelding’ [another event]

I should see 'Du er nu tilmeldt' message. [another outcome]

The order of Given, When, Then, And can be interchanged or used multiple times or sparingly used.

A template for Cucumber, tool used for creating automated acceptance tests

NAVIGATION ”humans get quickly disoriented”

Test Flow

• 04/12/2013

18 •

Requirement walk-

trough by business

Documented

business

requirements

Test Team

Write Feature files

Business

Sign-off

on feature files

Development Team

Sprint Review

Meeting

Test Team

Test feature

files manually

Test Team

Test automation

Candidates that can’t

be automated

Test execution

by CD process

Test execution

before release

CODE Working Software

How does it fit with Sprints

2/4/2014

19

SPRINT 2

SPRINT 2

SPRINT 3

Test Team

Development Team

SPRINT 1

TEST

SPRINT1 CODE

TEST TEST

SPRINT 3

FIX CODE CODE

REQ

B

REQ

A

REQ

C

End-to-End feature flow

• 04/12/2013

20 •

Ready for

Development

Start IT

Development

Business

Demo

Develop

features

Business

UAT

New Version

Required

Assign

Assign

Test Report

- Passed

- Failed

Business IT

Sign-off

Product

Business

Priority

Project

Backlog

Backlog

Feature

Refine

Requirement

Feature

Complete

DoD

Onshore

Sprint

Backlog

Offshore

Sprint

Backlog

Develop

features

Test

Implementation

& Execution

DoD

DoD = Unit tested, Reviewed & features deployed in test environments

Sprint

Demo Dispatch

Sprint

Demo

Write Test

Cases

(feature files)

CRUISE CONTROL ”the quality engine”

Extend Automation with Continuous integration (The Hook)

- End-to-end quality

• 04/12/2013

22 •

Chapter 16 Quality

Integrate Testing into the Process

1. Automate Test (Test vision)

2. Build a pipeline

3. Extend the pipeline

David Farley

How long time does it take to deploy one line of code to production?

Continuous integration

• 04/12/2013

23 •

• Technical Debt

• Unit Tests Coverage

• Unit test success

• Code Compliance

• Compare with the past (trends)

SonarQube™ an open platform to track code quality

Automated Unit tests

Automated

Integration Tests

Continuous integration

• 04/12/2013

24 •

• Functional Test - behavior driven

• It’s based on business requirements and

end-user behaviour

• Features it’s written in plain text (DSL)

• The test is a cycle of interactions with

well-defined outputs, resulting in the

delivery of working tested software

Automated

Functional test

Visualize the process

• 04/12/2013

25 •

RETROSPECT ”so far, so good”

• 04/12/2013

27 •

KPI

Reduced Wait Times

Greater Availability of Services

Decreased Software Cycles

Increased Test Coverage

Shorter Test Cycle Times

Improved Time-to-Market

Fewer Production Defects

• Testers are not waiting for the

builds anymore – single click

deployment

• Regression test is always

running

• Requirements are analyzed up

front by testers due to feature file

creation (increase awareness)

• Developers help testers and

testers help developers

• Testers become more technical

aware

• Testers help deliver software

successfully

FORECAST ”the storm is coming”

Pipeline spinning out of control

• As more test cases and systems are added in the pipeline the complexity will increase!

• Test case execution is often unsuccessful due to failures outside of under-test application (external dependencies)

• Rapidly increasing number of test cases resulting in increased levels of failure

• 04/12/2013

29 •

Number of automated tests

Number of integration points

Automated testing success rate

Environmental complexity

Time

Volume

“I can’t do anything until I have everything… and I never have everything!”

Continuous integration: What are the constraints?

• 04/12/2013

30 •

INCOMPLETE DEVELOPMENT

SYSTEM UNAVAILABLE

NO ACCESS TO 3RD PARTY

INVALID OR MISSING DATA

LIMITED ACCESS TO OTHER SYSTEMS

BURNED DATA

Same problems as every other project I guess!

VISION v2 ”reduce risk of damage”

Objectives for Vision v2

32

• Build an ‘isolated’ test environment using Service Virtualization to provide virtualized dependencies

• Key tasks – Integration into continuous integration

• Record virtual services from the End-to-End (E2E) test and store images in version control

• Extract and deploy CA LISA virtual services from version control.

• Build an isolated test environment configured to use LISA virtual services and execute test cases

Isolated Test environment

The new pipeline with Service Virtualization support

• 04/12/2013

33 •

Code

guidelines

Give full control to “legacy”

and test data

Application

Test

E2E environment used to

capture service images

CA LISA Recorder

CA LISA VSE

VS VS VS

E2E Test

Functional Application

Test

Legacy

systems

Integration

test Code Quality

Compile

Unit test

SVN

Additional step executed

against virtual services

THE END ”of this presentation”

DISCUSSION How many of you work with:

• Agile methods?

• Testing on all levels?

• Test automation?

• Continuous integration?

• Service virtualization?

Does it work for you?

Can we do better than this?

THE END ”every end is is new beginning”