performance testing in an agile world sai …...performance testing in an agile world presented by:...

T5 Track

10/4/2012 9:45:00 AM

Performance Testing in an Agile World

Presented by:

Sai Subramanian

Cognizant Technology Solutions

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Sai Subramanian Cognizant Technology Solutions

As a manager at Cognizant Technology Solutions, Sai Subramanian is responsible for delivering

web performance innovations and performance engineering services to Fortune 500 clients in

North America. During the past six years, Sai has supported major performance initiatives for

well-known healthcare, insurance, and retail clients in varied roles including performance

tester, performance analyst, performance architect, and performance engineer. Sai is a

member of the Computer Measurement Group and has published in its monthly journal,

MeasureIT. Prior to Cognizant, Sai was a researcher at the Software Engineering Institute and

Robotics Institute of Carnegie Mellon University working on software engineering processes

and methodologies that can be applied to robotics application development.

1

Sai Subramanian

Sr.Manager–Cognizant

[email protected]

PERFORMANCE TESTING

IN AN AGILE WORLD

1

What we will discuss today

1

2

3

4

We Willdiscuss today

Performance testing in a Waterfall /Agile

processWhy fitting Performance testing in

Agile is difficult?

A Case Study to explain the different levels of collaboration

Scrum of scrums

approach for programs with

multiple scrums

2

2

Waterfall-Gurukula

ReleaseTestingDevelopmentArchitecture & Design

RequirementsGathering

3

Agile-School

3

4

Performance Testing

TestingWaterfall

Scrum

5

We learnt it the

Hard way…

Fitting Performance in Agile

4

6

The case study-The Project

Life Policies

DI Policies

Annuity Policies

Master Data

Management

(Customer Data)

Policies

Internet

Multiple Channels

Phone

mail

fax

E -mail

Desktop

Single Servicing Environment

IVR / CSR

Mail

Room

Web

Self

Service

End -to -End

Business Process

Management Workflow

Holistic

View

1 Truth

Customer Experience

Servicing Platform

PEGA

------------------------------------------------------------

7

Multiple Scrum Teams

Desktop

team

Web

self-serviceSOA

1. 3 scrum teams running in parallel with their own codebases

during the course of the release

2. Infrastructure team

3. Governance team

5

8

Sample Release Plan

TO

Code

Freeze

Implement

1/1 7/1

2/1 3/1 4/1 5/1 6/1

Code

Freeze

Sprint 1

12 Days

Implement

Sprint 2 Sprint 3 Sprint 4 Sprint 1 Sprint 2 Sprint 3 Sprint 4

Plan

Plan

Sprint 1

Plan

Release 6

3/18/2010

Release 4

6/18/2010

Release 5

Retro

Retro

Release 5Release 4

DAY

9

Roll-out plan & Issues

• Application was rolled out in phases to the CSR’s and Self-service users

• Issues

Adoption

rate was low

Complaints-

overall call

handling time

increased

Training/Performance

(Sadly both!)

6

10

Release Integration-Performance Testing Model

• Role : Member of the infrastructure team

• Challenges faced

Performance team and Core team speak different languages

Bottlenecks are not real/reliable

Integrated rule-sets are available just before the release

Performance team is not identified/accepted as a part of the Core team

• StrengthsLow cost to the project

Flexible demand management for the CoE

11

Integrated Build and Performance Test Release Integration-Performance Testing Model

Sprint-1 Sprint-2 Sprint-3 Sprint-4Scrum-1



Integrated Build

Performance Test

7

12

Student-Teacher Analogy

汉语汉语汉语汉语拼拼拼拼音音音音

எ�ன??

When the student and the teacher speak different languages-it makes it really hard to evaluate

13

TESTING

MODEL

Performance

analyst

should be a

part of the

Sprint

planning

Performance

analysts and

core team

should

understand

each other

Performance

analyst

should be a

part of the

infrastructure

team

Performance

issue, root-

causes =

Unknown

Unknowns

Release Integration-Performance Testing Model Lessons Learned

8

14


எW

音音音音

Student understand the language but may not really have the answer for the questions-known Unknowns

எ�ன=What =

汉语拼音汉语拼音汉语拼音汉语拼音(hanyupinyin)

15

Sprint Integration-Performance Testing Model

• Role : Member of the infrastructure team

• Challenges faced

Performance team cannot keep up with the application delivery

Bottlenecks are identified but the root-causes are not known

Code used for performance testing is not in synch with development

Performance sign-off became a critical path bottleneck to release

• Strengths

Performance Analyst understands the application, performance critical use-

cases, environment and data

9

16

Integrated Build and Performance Test

Integrated Build

Performance Test

Sprint Integration-Performance Testing Model




17

Sprint Integration-Performance Testing Model Lessons Learned

TESTING

MODEL

Performance

Analyst should

be a part of the

scrum teams

Scrum team to

conduct their

own performance

tests prior to

rule-set

integration

Sr.Perf Analyst

should drive

integrated

performance

tests holistically

Performance

issues , root-

causes =

Known

Unknowns

10

18


Known language with continuous evaluation and feedback processes-known Knowns

19

Scrum of scrums approach

• Role: Active members (pigs) of all the scrum meetings that run concurrently

• Sr. Performance Analyst drives the daily performance meeting:

• The agenda will be

Challenges

• Management’s commitment to dedicate time and resources to drive the performance effort

• Organizational culture shift for developers to commit on performance user stories

Strengths

• Performance analysts understands the application, performance critical use-cases, environment and data

• Bottlenecks are identified but the root-causes are also known-Known knowns

Performance tests executed yesterday

Blockers/bottlenecks identified

Readiness for integration

11

2

0

Integrated Build

Performance Test

Integrated build and Performance Test Scrum of scrums model




Scaled-down perf

tests in Dev env

21

Sprint Integration-Performance Testing Model Lessons Learned

TESTING

MODEL

Performance

should be

owned and

evaluated

continuously

Early

performance

analysis with

creative

performance

tests

Improved

performance

with a significant

increase in the

effort and cost

Performance

issues , root-

causes =

Known

Knowns

12

22

Summary

• Management’s realization on the criticality of performance in

Agile projects

• Sr. Performance Analyst required to manage projects with

multiple scrums

• Performance analysts to embed within each scrum teams to

understand the project

• Performance user stories to be collectively owned by the

entire team

• Significant increase in the cost

performance testing in an agile world sai …...performance testing in an agile world presented by:...

Documents