Acceptance Testing with Selenium

Topics

• Why test?

• Types of testing– Unit– Acceptance

• Acceptance testing with Selenium

Why test?

• Find bugs

Why test?

• Find bugs

• Other– Performance– Usability– Etc.

Why test?

• Find bugs

• Prove correctness

• Other– Performance– Usability– Etc.

Software Development

• Requirements– What do we want it to do?

• Development– Write code that meets requirements.

• Testing– Does the code meet the requirements?

Testing - Traditional

• Requirements

• Design / Specification

• Coding

• Testing

Testing - Agile

• Requirements

• Write (failing) test

• Write code

• Run test (successfully)

Testing - Agile

• Requirements in the form of a test

• Run test

• Code

• When test passes, you’re done.

Unit Test – Simple Example

Calculator

• Add

• Subtract

• Multiply

• Divide

Unit Test – Simple Example

# Calculator.add() – implementation

function add(first, second)

return first + second

end

Unit Test – Simple Example

# Calculator.add() – test

# 1 + 1 = 2

assert_equal(2, Calculator.add(1, 1))

Unit Test – Simple Example

# Calculator.add() - test

assert_equal(2, Calculator.add(1, 1))

assert_equal(-2, Calculator.add(-1, -1))

assert_equal(42, Calculator.add(0, 42))

assert_equal(1001, Calculator.add(1000, 1))

Unit Test – Simple Example

# Calculator.divide() - implementation

function divide(first, second)

return first / second

end

Unit Test – Simple Example

# Calculator.divide() – test

# 12 / 3 = 4

assert_equal(4, Calculator.divide(12, 3))

Unit Test – Simple Example

5 / 2 = ?

Unit Test – Simple Example

# Calculator.divide() - test

assert_equal(2, Calculator.divide(5, 2))

Unit Test – Simple Example

# Calculator.divide() – implementation

function divide(first, second)

return round(first / second)

end

Unit Test – Simple Example

1 / 0 = ?

Unit Test – Simple Example

# Calculator.divide() - test

assert_equal(0, Calculator.divide(1, 0))

Unit Test – Simple Example

# Calculator.divide() – implementation

function divide(first, second)if (second == 0) then

return 0else

return round(first + second) endend

Unit Test – Simple Example

• It works – the tests pass …

• … but the code is a little verbose …

• … so let’s refactor.

Unit Test – Simple Example

# Calculator.divide() – refactored

# (boolean expr) ? true-result : false-result

function divide(first, second)

return (second == 0) ? 0 : round(first/second)

end

Unit Test – Simple Example

• The tests run successfully

• Working calculator

• Nicely refactored code

• All functionality covered by tests

Calculator.divide() Requirements

First Second Expected

12 3 4

5 2 2

1 0 0

Calculator.divide() Tests

assert_equal(Calculator.add(12, 4), 3)

assert_equal(Calculator.add(5, 2), 2)

assert_equal(Calculator.add(1, 0), 0)

RSpec

describe “Calculator” do

it “should divide 12 by 4 and return 3” doCalculator.divide(12, 4).should == 3

end # rest of Calculator assertionsend

RSpec - Results

Calculator

- should divide 12 by 4 and return 3

- should divide 5 by 2 and return 2

- should divide 1 by 0 and return 0

Finished in 0.014 seconds

3 examples, 0 failures

Questions on unit testing?

Acceptance Testing

• Functional tests are written from a user's perspective. These tests confirm that the system does what users are expecting it to. (IBM)

• … acceptance testing is performed by the customer on a system prior to the customer accepting delivery … (Wikipedia)

Customer Test-Driven Development (CTDD)

• Tests– Customer (browser) perspective– Validate correctness of application– Successful test constitutes acceptance

• Tests created, run, re-run throughout project life

Acceptance Testing

• Finds bugs

• Finds usability issues

• Produces new/changed requirements

Acceptance Testing

• Comes near the end of the project (late)

• Changes are most expensive

• How often is software not “accepted”?

Acceptance Testing - CTDD

• Requirements in the form of a test

• Meeting requirements is an objective, unambiguous activity.

• Consideration given to exceptions and edge cases, not just “happy path”.

Example Requirement

• Add comments field

• Make it required

Requirement

• Add comments field as an HTML text area, 3 rows x 30 columns.

• Put label “Additional Comments” on top of text area

• Make it required. If empty, error message should be “Additional Comments is a required field.”

Requirements as Test

open /record/new

click save

verify text present Additional Comments is a required field

type comments “My comment”

click save

verify text present Record saved

More Requirements

• Comments can be blank for existing data

• Don’t display comments on “Delete Verify” page.

• Comments should be 20 – 80 characters.

Comments can be blank for existing data

open /record/edit?id=42

verify text comment “”

click save

verify text present Record saved

Don’t display comments on Delete - Verify page

open /record/edit?id=9

verify text comment “comment #9”

click delete

verify text not present

“comment #9”

Comments should be 20-80 characters

open /record/edit?id=9

type comment “cmmt”

click save

verify text present

Comments s/b 20-80 chars

type comment (> 19 chars)

click save

verify text not present

Comments s/b 20-80 chars

Requirements as Runnable Test

• It’s real

• Selenium– Selenium Core– Selenium IDE– Selenium Remote Control– Selenium on Rails

Selenium Core

• Installs in public/html (e.g.)

• Test runner is all Javascript

• Write test cases in Selenese (HTML)

• Point browser to your_app/selenium

Selenium IDE

• Firefox plugin

• Record, playback, save, edit scripts

• Works on any (combination of) web sites

Selenium IDE Demo

Selenium Element Selectors

• id=some_id or name=some_name

• dom=document.forms[1].myDropdown

• xpath=//img[@alt=‘Some alt text’]

• link=some link text

• css=cssSelectorSyntax

Selenium Element Selectors

What are you testing?

<a href=“…”, id=“new_link”>Add New</a>

click id=new_link

or

click link=Add New

HTML Recommendations

• Id attributes on all important elements

• Class attributes a close second

• Valid mark-up reduces dom and xpath errors

Selenium Remote Control

… write automated web application UI tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser.

Java, .Net, Perl, PHP, Python, Ruby

C# (NUnit) Exampleusing Selenium;using NUnit.Framework;namespace MyTests { [TestFixture] public class GoogleTest { private ISelenium sel; [SetUp] public void SetUp() { sel = new DefaultSelenium("localhost", 4444, "*firefox", "http://www.google.com"); sel.Start(); } [Test] public void testGoogle() { sel.Open("http://www.google.com/webhp"); sel.Type("q", "hello world"); sel.Click("btnG"); sel.WaitForPageToLoad("5000"); Assert.AreEqual("hello world - Google Search", sel.GetTitle()); } [TearDown] public void TearDown() { sel.Stop(); } }}

C# (NUnit) Example

[Test]public void testGoogle() { sel.Open("http://www.google.com/webhp"); sel.Type("q", "hello world"); sel.Click("btnG"); sel.WaitForPageToLoad("5000"); Assert.AreEqual("hello world - Google Search",

sel.GetTitle());}

Selenium on Rails

• Ruby on Rails plugin

• Runs within Rails test framework

• Write tests in Ruby or Selenese– database access– can abstract tests into reusable functions– and much more

Selenium on Rails Demo

Data Model

roles user_roles users

Selenium on Rails Demo