growing manual testers into automators

Growing Manual Testers into Automators

Camille Bell

[email protected]

Twi5er @agilecamille

Agile DC 2015 October 26, 2015

Obvious Limits of Manual Tes6ng

[email protected] 2

Itera6ons

Time

Time Needed

Time Available New Features Regression

Simplifica)on: e.g. Doesn't include interac)on of new & exis)ng features

Delay in Tes6ng (both manual & capture/playback)

[email protected] 3

Dev. Months Test

Dev. Test

Dev. Test

Dev. Test

Days

Weeks

Even in the best case tes)ng is a@er development coding.

•  Keystroke, mouse capture

•  Test briLle (trivial UI changes break tests)

•  Hard coded test data

•  Lots of duplica6on

•  Test modifica6on near impossible (spagheP code when accessible)

•  No control mechanisms (for looping, etc.)

•  Doesn't scale

•  Over 6me degrades so much as to become throw away

•  Etc.

•  Tests wriLen aSer implementa6on

•  Not in business language

•  Doesn't facilitate collabora6on

•  Doesn't drive developer progress (not executable un6l aSer development)

•  Encourages finger poin6ng

•  Not an executable specifica6on

•  Etc.

[email protected] 4

Capture/Playback Tes6ng Fails Because:

These Are All The Same Thing

•  Acceptance Test Driven Development (ATDD)

•  Behavior Driven Development (BDD)

•  Story Driven Development

•  Specifica6on by Example

–  Minor subtle difference don't maLer for most purposes

–  Collaborate to define detailed acceptance criteria before developing applica6on code, turn those into executable acceptance tests (before developing applica6on code), watch them fail (no app) and later pass, when app func6onality implemented

[email protected] 5

Collabora6vely (Spec it/define tests) •  Some6mes called

the 3 Amigos (test, dev, business)

•  Conversa6on based

•  Focus on value

•  Specific examples

•  Concrete data

•  Outside in (business focus drives development)

[email protected] 6

Three Parts of Wri6ng any Test Cucumber 3 As

Ruby 3 As Java

Formal Testers What it is

Given Assemble Arrange Precondi6on All the setup and dependencies for the test

When Ac6vate Act Ac6on/Event Something that happens that will make the outcome different than if it hadn’t happened

Then Assert Assert Expected Results or Postcondi6on

What is expected to happen and any side effects

[email protected] 7

Test Automa6on Has 2 Sides

Business Facing

•  Simple English

•  In business terms •  Promotes collabora6on

•  Uses examples •  Uses concrete data •  Proves business needs met

•  Executable specifica6on •  Modifiable & reusable

Technology Facing

•  Exercises underlying SW

•  Connects to browsers, networks, databases, frameworks, etc,

•  Object Oriented •  Extensive test libraries •  Modifiable & reusable

•  Learnable programming language

[email protected] 8

Business Facing •  User Stories, especially those created though story mapping,

are a good place to start talking about acceptance criteria

•  That can lead to thinking about categories of test scenarios (test 6tles)

•  Given, When, Then comes next

•  Make sure there is concrete test data

•  Let's see how it works

[email protected] 9

Acceptance Criteria will become our specs when they have details

Feature: Cash Withdrawal

In order to get money when the bank is closed

As a bank customer

I want to withdraw cash at the ATM

•  Successful Withdrawal •  Less than balance •  Equal to balance

•  Withdrawal Failed Due to Insufficient Funds

•  Withdrawal Failed Because Cash Dispenser Doesn’t Dispense One Dollar Bills

•  Withdrawal Failed Because Account Closed

First Cut Acceptance Criteria for several tests [email protected] 10

•  Successful Withdrawal (less than balance)

Given my account has starting balance of $1000

When I withdraw $200

Then $200 should be dispensed

And the ending balance of my account should be $800

Given, When, Then format – Business language with unambiguous detail

Detailed Acceptance Criteria

These detailed acceptance criteria are acceptance tests that are executable specs

[email protected] 11

•  Though users only sees the UI, Acceptance Tests exercise behavior of the en6re applica6on stack (network, DB, etc.).

•  However since capture playback causes so many problems, we needed a beLer way to automate, including:

–  Modeling behavior, so it is easier to modify

–  Working in non-‐proprietary programming language testers can learn

–  Having good test architecture and design

–  Exercising good programming prac6ces (D.R.Y., meaningful variable names, etc.)

–  Following good code management (frequent check-‐ins, simple branch structure, frequent merging with trunk/main, etc.)


Technology Facing

Features

Scenarios

Automa6on Libraries (e.g. rspec, page-‐object, wa6r-‐webdriver, etc.)

Support Code (e.g. env.rb, hooks.rb, Gemfile, your custom page objects)

Business Facing

Technology Facing


Step Defini6ons

Cucumber/PageObject Technology Stack

Gherkin (English-‐like business language)

Built-‐in Ruby Libraries for

Tes6ng

Ruby (oSen modified templates)

Steps wriLen in Ruby

Steps

•  Cucumber can be used with other programming languages

•  But the Ruby implementa6on comes with much more out of the box than any other implementa6on

•  For example Ruby libraries already exist to model

–  web pages PageObject

–  iPhone screens ScreenObject

–  3270 green screens TE3270

–  XML Nokogiri

–  etc.

•  In other languages testers have to write a lot of complicated code that's included with Ruby

•  And Ruby is easiest for testers to learn [email protected] 14

Why Ruby?

•  Yes, assuming we're tes6ng a web app

•  The stack includes wa6r-‐webdriver, which is a combina6on of Selenium and Wa6r

–  Which is fully programmable

–  Can be accessed though PageObject

–  And is good

•  NOT the Selenium IDE

–  Which is capture/playback

–  And is bad


Don't We Also Need a Web Interface?

Tester Behavior & System Access •  Testers no longer just use a test tool -‐ they develop test code

•  Testers now will oSen behave like devs. Testers will

–  Write code

–  Run code –  Lookup error messages and technical info on the web

–  Check-‐in code –  Update gems and other SW

–  Install new browsers & other tools –  Etc.

•  So they need the same rights and privileges as devs. For most shops this means:

–  Admin rights

–  Unrestricted web access –  Working with a dev, sys admin or dev-‐op helps for beginning installs


Tools and Infrastructure •  Installa6on details for Windows, Mac and Linux differ

(documented in Cucumber and Cheese)

•  Testers need mul6ple browsers installed (Firefox and Chrome plus IE, Safari or whatever is standard)

•  Select and install an editor or IDE {consider Sublime Text (text editor) or RubyMine (IDE)}

•  Test code needs to be under source control, so connect to Git (modern choice) or other Source Control

•  For Ruby Gem security and other updates 'github.org' and 'rubygems.org' both testers and the CI server should update their Gemfile regularly

•  The CI needs full test stack. If OSs differ (e.g. Windows vs. Linux) some gems may differ. Jenkins is popular CI server.


•  My style:

–  Ensure all infrastructure for every trainee is in place

–  First teach small bit of Ruby intro for non-‐programmers

–  3 day formal class on ATDD/BDD with Cucumber with lots of hands-‐on exercises

–  Start with teaching students how to test exis6ng web pages

–  Evolve to teaching how to define web pages that don't exist yet

•  Resources at end of slides •  Don't just throw a book or web site at

testers, teach them & guide them

Training


Coaching & Daily Prac6ce •  DON'T JUST TRAIN -‐ PRACTICE

•  Technical skills take 6me to learn

•  Requires reinforcement

•  Professional coaches really help

•  Pairing with devs may help ease technical learning

•  Later advanced testers could become internal coaches

•  Prac6ce 3 amigos

•  Mob Programming really helps


Two co-‐workers pair programming

Mob Programming •  Whole team works together

•  On the same thing

•  At the same 6me

•  In the same space

•  Using the same computer

•  Driver rotates •  Everyone par6cipates •  Great for training & prac6ce •  Great for collabora6on •  Everyone up to speed faster •  Some teams mob always

•  Others mob frequently [email protected] 20

Resources

and

References

Resources – Free Ruby Info

Ruby 101 – the basics: (Learn Ruby 2.x not Ruby 1.9)

•  On-‐line tutorials – my favorite is Ruby Monk hLps://rubymonk.com

Learn to navigate the documenta0on:

•  Ruby's built-‐in classes and methods are very rich.

•  Testers should learn to check if the needed func6onality already exists

•  Ruby docs are found at hLp://ruby-‐doc.org/


ALL

SOME

Resources – Free PageObject Info Cheezy's (Jeff Morgan's) Blog

•  Other Cheezy blog posts are also worthwhile hLp://www.cheezyworld.com/2011/07/29/introducing-‐page-‐object-‐gem/

Official documenta0on

•  PageObject is a gem.

•  Like most gems, it and its documenta6on can be found on GitHub (scroll down) hLps://github.com/cheezy/page-‐object


Resources – Free Detailed Demo Cheezy's Demo Tes0ng Puppies Site

•  Uses RubyMine IDE, but other editors or IDEs work fine

•  Covers a later example from Cucumber and Cheese book resource

•  1 hour 10 minute YouTube video hLps://www.youtube.com/watch?v=zpt15XIjpas


Resources – Free Cucumber Info Cucumber Wikis

•  Both wikis focus on headless Cucumber (not in browser with PageObject), but provide useful info on Cucumber and Gherkin

•  Older Wiki – easy to explore format, contains some info not in new wiki hLps://github.com/cucumber/cucumber/wiki

•  Newer Wiki – contains some nice blogs and plug for semi commercial product and support hLps://cucumber.io/


Resources – Ruby Books If Ruby Monk is too challenging, try Learn to Program

•  For those who haven't programmed before (most manual testers)

–  Learn to Program -‐ Second Edi0on by Chris Pine

•  For those who know object-‐oriented programming in some other language (includes more Ruby than most testers will need)

–  Programming Ruby 1.9 & 2.0 by Dave Thomas

•  For a problem-‐based approach to wri6ng small pieces of Ruby that do useful things

–  Everyday Scrip0ng with Ruby by Brian Marick


Available from Amazon or the Pragma6c Bookstore hLps://pragprog.com/

Resources – Cheezy's Book Cucumber with PageObject

•  Covers Cucumber, Gherkin, PageObject, Wa6r, Selenium, etc. used in demo

•  Step by step tutorial with exercises

•  SoScopy only, work in progress, updated periodically

–  Cucumber and Cheese: A Testers Workshop by Jeff Morgan


Available only from Lean Pub hLps://leanpub.com/cucumber_and_cheese

Resources – Cucumber Books Ignore references to headless Rails tes0ng unless using Rails

•  Cucumber with Ruby – General

–  The Cucumber Book: Behavior-‐Driven Development for Testers and Developers by MaL Wynne & Aslak Hellesoy

•  Advanced Cucumber Topics

–  The Cucumber Book: Behavior-‐Driven Development for Testers and Developers by Ian Dees and MaL Wynne


Available from Amazon or the Pragma6c Bookstore hLps://pragprog.com/

Camille Bell

Agile Technical & Process Coaching Retrospec6ves

Agile Boot Camps Agile Training Updated Slides

or just to chat about things agile [email protected]

Twi5er @agilecamille

growing manual testers into automators

Software