effective testing practices in an agile environment

Effective Testing Practices in an Agile Environment

Raj Indugula [email protected]

www.lithespeed.com

ON-‐DECK AT-‐BAT DONE

What is the problem context?

Can I see it in ac8on?

Concluding thoughts. Any ques8ons?

What are the guiding principles of Agile Tes8ng ?

What are the enabling prac8ces?

Agile Manifesto

Individuals and interactions Processes and tools over

Working software Comprehensive documentation

over

Customer collaboration Contract negotiation over

Responding to change Following a plan over

That is, while there is value in the items on the right, we value the items on the left more.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

http://www.agilemanifesto.org

3

Shared Context drives Quality

Individuals and interactions

Working software

Customer collaboration

Responding to change

4

Quality is a priority

Agile Approach

Maximize value and quality within specified

project constraints.

5

ROI

Customer Value

Sustainability

Quality

5

Integrated Teams & Iterative Delivery Challenges

6

How do we bridge the communication gap between Bus./QA/Dev? How do we ensure that the evolving software does not regress?

Move Quality Upstream

•  Testing not a trailing activity

•  QA leads the Sprint by providing guidance and feedback on the business needs of the emerging product

8

Testing is Continuous, NOT a Phase

http://testobsessed.com/

9

Testing is Collaborative

•  Quality is everyone’s problem, not just of the testers

•  Testing is the responsibility of the whole

team

10

Tests represent Customer Expectations

•  Shared understanding of what it means for a story to be done

11

Quick Feedback

•  Faster feedback loops increase Agility – the ability to respond to change

•  Test automation provides rapid feedback on how the software is behaving

12

“Leave No Broken Windows”

•  Fix bugs as they are found

•  The sooner you find a defect, the cheaper it is to fix

13

It isn't Done until it’s…

“Done Done”

Coded Tested Done

14

Test-Driven

Defining tests with the requirements guides development

http://testobsessed.com/

15







ATDD

TDD

Other prac8ces

Agile Engineering Practices

17

Agile Engineering practices allow teams to move fast, be flexible and deliver high quality software

Bill Wake, http://www.xp123.com

What are acceptance tests?

•  Tests that demonstrate business intent of

system from end user’s point of view

•  Black-box testing

18

What is Acceptance Test Driven Development (ATDD)?

A practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. - Elisabeth Hendrickson

19

Key Characteristics

1.  Elicit details from the business stakeholder(s) about their expectations

2.  Distill acceptance criteria into automatable tests expressed in a natural language

3.  Wire the tests to SUT with “glue” code (as part of implementation)

20

When to Use

•  During Requirements Elaboration “First Cut” once the story is written Card level acceptance criteria

•  During a Three Amigos session (Business, QA, and

Dev) Expand the story card acceptance criteria Complete set of test scenarios

22

User Story

As a <type of user>, I can <immediate goal> so that

<business outcome>.

User Role (Who?)

Desired Function (What?) End Result (Why?)

23 3 C’s – CARD, CONVERSATION, CONFIRMATION

Modern Agile Acceptance Model

Conditions of Satisfaction – Broad Terms

Acceptance Criteria – Further Refined

Examples – Actual scenarios or data

Executable Examples – Ready to automate

24

Acceptance – Conditions of Satisfaction

Testing a Registration Function •  What constraints should we impose? •  Business stakeholders and PO agree that

passwords should not be easy to crack.

Please fill in to register.

Email Address

Password

Register

25

Acceptance – Acceptance Criteria

Acceptance Criteria, the Details The PO works with testers and developers from the team, business stakeholders, users or SMEs to come up with this definition of “not easy to crack”: •  Must be at least 8 characters and no more than 12 •  Must contain only alphanumerics and the period •  Must contain at least one digit •  Must contain at least one alpha character

Please fill in to register.

Email Address

Password ????????????

26

Acceptance – Examples Written •  Examples, Making it Real •  Actual examples are the ultimate way to communicate

requirements

Password Expected Expected Message Comment

abc123 Invalid You password must be at least 8 characters long, and no more than 12 characters long.

abcdefghi Invalid Your password must contain at least one character and one number.

1aaaaaaaaa Valid Why valid?

ajx972dab Valid

27

Acceptance – Testable Examples in Gherkin •  Executable Example, Making it Repeatable •  Examples that can be executed are the final step

Given the “Unregistered User” user has navigated to the “register” page When entering “newuser” in the “Username” field And entering “abc123” in the “Password” field And entering “abc123” in the “Confirm Password” field And pressing the “Register” button Then the text “Thank you for Registering” should appear on page And the URL should end with “use/accountPage”

28

Gherkin – Another Example

•  Plain English with a specific format §  Feature §  Scenario §  Given When Then And But

•  Can be used for both manual testing and automation

29

Automated Acceptance Tests

Tool (Cucumber, SpecFlow, FitNesse)

Specification expressed in

common language

“Glue” code that ties specification

to SUT

30

Business Analyst Develop usage scenarios

Developer

Create and maintain acceptance test “glue” code that ties test specification to system under test

Tester Create and execute tests to simulate usage scenarios; Automate regression testing

•  Should be an integral part of every iteration/sprint, not just a trailing activity

•  Vital not only to prove completeness of a feature in repeatable fashion, but also to prove that software does not regress as it evolves

•  Success depends on cooperation & collaboration

Testing Collaboratively

31

Benefits of ATDD

•  Improved requirements elicitation

•  Consensus between BA/QA/Dev on the story helps prevent bugs & gives clear target for development

•  Reuse of Acceptance Scenarios for all phases of testing

•  Creates clear examples that everyone understands; discovers some problems before any development

32







ATDD

TDD

Other prac8ces

What are unit tests?

•  Developer tests that determine whether the smallest piece of testable software in an application is behaving as expected

•  White-box testing

34

Structure of a Unit Test

@Test!

public void itAddsOnePlayerToTheGame()!

{!

!Game game = new Game();!

!

!Player al = game.AddPlayer("Al");!

!

!Assert.assertEquals(1, game.numPlayers());!

!Assert.assertTrue(game.isPlaying(al));!

!Assert.assertFalse(game.isPlayable());!

}!

Assert

Arrange

Act

35

…all related to same behavior

…title is a behavior

Tests should…

•  Not depend on one another •  Not affect one another •  Leave the system under test unaltered •  Be fast executing •  Be executable in any order, number or

combination of one another

36

What is Test Driven Development (TDD)?

1.  Write a unit test 1.  Use unit testing and mocking framework (JUnit, Mockito etc.)

2.  May need to write some skeleton code to make it compile

2.  Run it and watch it fail (no code yet)

3.  Write code until it passes

4.  Repeat

37

Write a new test

Red (Failing Test)

Write Code

Green (Test Passes)

The tests help determine what code you write (“Incremental Design”)

Write a new test Test fails

Integrate Write code

Refactor code, make sure tests pass

Test passes

Developer heartbeat

Red, Green, Refactor

Refactoring is supported by the tests and good unit test coverage makes refactoring safe to do

38

TDD – Benefits

•  Better design – less coupling

•  Supports change

•  Prevents gold-plating

•  Prevent bugs

•  Discover bugs

39

Test Levels

Isolation Tests

•  True unit tests, focusing on behaviors

•  Many, and fast

Collaboration Tests

•  “End-to-end” tests

•  Find bugs in conversations between collaborating modules

40

Test Doubles (Mocks)

Used to isolate unit tests form external dependencies

Why use them?

•  Ease of setup •  Fast executing •  To work with a subsystem that doesn’t exist •  Simulate various execution paths of external system

41

Test Automation Pyramid

42

Unit Tests/Component Tests

Cucumber, FitNesse, SpecFlow

xUnit, Mocks

Selenium

Developer-centric (Are we building the code right?)

Business-centric (Are we building the right code?)

Adaptation of Mike Cohn's test automation pyramid

Exploratory Testing

GUI Tests

Acceptance Tests

ATDD/TDD Cycle

43

TDD vs. ATDD Historically ATDD did not come about till well after TDD

ATTD §  is a specification and testing practice §  Determines what “Done” means for a story §  Creates a common understanding and shared language

to discuss features (especially with Gherkin) §  Cycle times on the order of hours to days

TDD §  is a coding and design practice §  Provides a mechanism for exploring the solution space §  Creates base of tests to support refactoring §  Cycle times on the order of seconds to minutes

44







ATDD

TDD

Other prac8ces

Version control Tests with Code

Source Code

Unit/Component Tests Acceptance Tests

Source Control Repository

(Git, Subversion…)

46

Requirements gathering Application Development

Source Control

Continuous Integration (CI) Server

Continuously commit changes

and merge changes from others

Pull changes, Build, run tests

Deploy to higher environment

Poll for changes

Development tasks from requirements

test scenarios from business requirements for acceptance criteria

Acceptance Test Environment - Test harness - Tests (automated + manual)

Execute acceptance tests against the deployed application

QA Environment

Deployable artifacts

Build Automation & Continuous Integration

47

Barriers to Adoption

Developer : “I’m a developer, not a tester” Management: “Why does Dev help with testing, when we have QA?”

•  Necessitates behavioral change •  Quality is not free •  Simple, but not easy •  Requires discipline •  Needs generalizing specialists

49

Bridging the Divide

Shared Understanding

Collaborative Testing Quick Feedback

50

“Before beginning a hunt, it is wise to ask someone what you are looking for before you begin looking for it.” —Winnie the Pooh

51

Thanks!

52

Resources

53

Some Tools (open source)

•  Unit testing: JUnit, Mockito, EasyMock

•  Acceptance testing: FitNesse Cucumber

•  GUI testing: Selenium, Sahi

•  Javascript testing: Jasmine, Qunit

•  Testing the database: DBUnit, SQLUnit, DBFit

•  Performance testing: JUnitPerf, JMeter

•  Usability testing: Manual unfortunately

54

Some Tools (open source)

•  Unit testing: MSTest, NUnit, Moq

•  Acceptance testing: SpecFlow

•  GUI testing: Selenium

•  Javascript testing: Chutzpa

•  Testing the database: Tsqlt, SSDT SQL Test

•  Performance testing: JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest

•  Usability testing: Manual unfortunately

55

Agile Software Development

•  Extreme Programming Explained - Kent Beck •  TDD: A Practical Guide – Dave Astels •  Refactoring – Martin Fowler •  Agile Estimating and Planning – Mike Cohn •  Pragmatic Project Automation – Pragmatic Prog.

56

Acceptance Testing

•  Bridging the Communications Gap – Gojko Adzic •  Agile Testing – Lisa Crispin and Janet Gregory •  The Cucumber Book - Wynne M. and Hellesoy A. Gherkin

https://github.com/aslakhellesoy/cucumber/wiki/gherkin Cucumber

http://cukes.info/ https://github.com/aslakhellesoy/cucumber/wiki/

57

TDD/Refactoring

•  TDD: By Example - Kent Beck •  TDD: A Practical Guide – Dave Astels §  The Art of Agile Development – James Shore •  Refactoring – Martin Fowler •  Working Effectively with Legacy Code – Mike Feathers •  Refactoring to Patterns – Joshua Kerievsky •  xprogramming.com, refactoring.com, testdriven.com

58

effective testing practices in an agile environment

Technology