ua testing with selenium and phpunit - phpbenelux summer bbq
DESCRIPTION
Nothing is as frustrated as deploying a new release of your web application to find out functionality you had doesn't work anymore. Of course you have all your unit tests in place and you run them through your CI environment, but nothing prepared you to a failing javascript error or a link that doesn't work anymore. Welcome to User Acceptance testing or UAT. Before you start putting real people in front of your application, create macros and export them as PHPUnit test classes. Then run them in an automated way just like your unit tests and hook them into your CI. In this talk I will show you how easy it is to create Selenium macros that can be converted into PHPUnit scripts and run automatically on different virtual machines (VM's) so you can test all different browsers on a diversity of operating systems.TRANSCRIPT
UA Tes'ng withSelenium and PHPUnit
PHPBenelux Summer BBQ
2
• PHP Consultant• President PHPBenelux• Conference speaker
Michelangelo van Dam
3
4
Thank you for sponsoring
Today’s goal
• Set up and use Selenium IDE• Record UA tests• Convert to PHPUnit• Run con'nuously• Mul' browser support
5
6
⚠DISCLAIMERS E L E N I U M T E S T S A R E N OT A REPLACEMENT FOR REGULAR UNIT TESTING. THEY ONLY PROVIDE AN ADDITIONAL SET OF TESTS FOCUSED ON USER ACCEPTANCE AND USER EXPERIENCE TESTING.
For more information about unit testing, please see my other material on www.slideshare.net and www.speakerdeck.com. Search for “dragonbe”!
User Acceptance
7
8
“Acceptance testing is a test conducted to determine if the requirements of a specification or contract are met.”
-- source: wikipedia
Checklist for web applica'ons
9
10
Func'onal tes'ng
• Test func'onal requirements-‐ e.g. no access to profile without authen'ca'on
• Test UI elements on the web interface-‐ e.g. buOons, form elements, AJAX controls, …
A word of cau'on!
11
• UA tests only test generated output-‐ not a replacement for unit tes'ng
• UA tests are heavily depending on DOM-‐ changes to the DOM might lead to failing UAT
Browser support
12
Selenium to the rescue
13
Plugin for firefox
14
Get the plugin (demo)
15
Let’s get started
17
Pick a test case
18
Issue #7
19
Verify this issue on PROD
20
21
Fix the issue
22
Run test to see it’s fixed
23
24
25
Save your test as .html
It’s that easy!
26
Automated Tes'ng
27
PHPUnit to the rescue
28
Export to PHPUnit
29
The PHPUnit TestCase
30
<?phpclass Example extends PHPUnit_Extensions_SeleniumTestCase{ protected function setUp() { $this->setBrowser("*chrome"); $this->setBrowserUrl("http://www.theialive.com/"); }
public function testMyTestCase() { $this->open("/"); $this->click("link=login"); $this->waitForPageToLoad("30000"); $this->type("id=email", "[email protected]"); $this->type("id=password", "test1234"); $this->click("id=signin"); $this->waitForPageToLoad("30000"); $this->click("link=Test demo"); $this->waitForPageToLoad("30000"); $this->assertEquals("Done", $this->getText("xpath=//th[5]")); $this->click("link=[EDIT]"); $this->waitForPageToLoad("30000"); $this->assertTrue($this->isElementPresent("id=done")); $this->click("link=sign off"); $this->waitForPageToLoad("30000"); }}?>
Change class name
31
<?phpclass Example extends PHPUnit_Extensions_SeleniumTestCase{ protected function setUp() { $this->setBrowser("*chrome"); $this->setBrowserUrl("http://www.theialive.com/"); }
public function testMyTestCase() { $this->open("/"); $this->click("link=login"); $this->waitForPageToLoad("30000"); $this->type("id=email", "[email protected]"); $this->type("id=password", "test1234"); $this->click("id=signin"); $this->waitForPageToLoad("30000"); $this->click("link=Test demo"); $this->waitForPageToLoad("30000"); $this->assertEquals("Done", $this->getText("xpath=//th[5]")); $this->click("link=[EDIT]"); $this->waitForPageToLoad("30000"); $this->assertTrue($this->isElementPresent("id=done")); $this->click("link=sign off"); $this->waitForPageToLoad("30000"); }}?>
class MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase
The PHPUnit TestCase
32
<?phpclass MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase{ protected function setUp() { $this->setBrowser("*chrome"); $this->setBrowserUrl("http://www.theialive.com/"); }
public function testMyTestCase() { $this->open("/"); $this->click("link=login"); $this->waitForPageToLoad("30000"); $this->type("id=email", "[email protected]"); $this->type("id=password", "test1234"); $this->click("id=signin"); $this->waitForPageToLoad("30000"); $this->click("link=Test demo"); $this->waitForPageToLoad("30000"); $this->assertEquals("Done", $this->getText("xpath=//th[5]")); $this->click("link=[EDIT]"); $this->waitForPageToLoad("30000"); $this->assertTrue($this->isElementPresent("id=done")); $this->click("link=sign off"); $this->waitForPageToLoad("30000"); }}?>
protected function setUp() { $this->setBrowser("*iexplore"); $this->setBrowserUrl("http://www.theialive.com/"); $this->setHost('192.168.56.101'); $this->setPort(12666); }
Meaningful method name<?phpclass MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase{ protected function setUp() { $this->setBrowser("*iexplore"); $this->setBrowserUrl("http://www.theialive.com/"); $this->setHost('192.168.56.101'); $this->setPort(12666); }
public function testMyTestCase() { $this->open("/"); $this->click("link=login"); $this->waitForPageToLoad("30000"); $this->type("id=email", "[email protected]"); $this->type("id=password", "test1234"); $this->click("id=signin"); $this->waitForPageToLoad("30000"); $this->click("link=Test demo"); $this->waitForPageToLoad("30000"); $this->assertEquals("Done", $this->getText("xpath=//th[5]")); $this->click("link=[EDIT]"); $this->waitForPageToLoad("30000"); $this->assertTrue($this->isElementPresent("id=done")); $this->click("link=sign off"); $this->waitForPageToLoad("30000"); }}?>
33
public function testMarkTestAsDone()
startSeleniumStandAlone.BAT
34
"C:\Program Files\Java\jre7\bin\java.exe" -jar "C:\Users\user\Downloads\selenium-server-standalone-2.28.0.jar" -port 12666
Now run your tests
35
How it runs on the node
37
Advantages
39
• You can start tes'ng immediately• Even test “hard to test” kind of situa'ons• More nodes for parallel tes'ng• Tes'ng different browsers and plaborms• Con'nuous Integra'on possible
Selenium Grid Setup
40
Selenium Testing
CI Server Windows"HUB"
Linux client"NODE"
CI executes tests
Windows HUB launchesSelenium node clients to execute tests
Windows Server collectsfeedback from the Citrixclient nodes and reportsback to CI Server
Windows client"NODE"
Mac OS X client"NODE"
Continuous User Acceptance Testing
Next Steps
41
Mul' Browser support
42
Base TestCase
43
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class TestCase extends PHPUnit_Extensions_SeleniumTestCase{ //const TEST_HUB = '217.21.179.192'; const TEST_HUB = '192.168.56.101'; const TEST_PORT = 12666;
const USERNAME = '[email protected]'; const PASSWORD = 'test1234'; const BASURL = 'http://www.theialive.com';
public static $browsers = array ( array ( 'name' => 'Internet Explorer 8 on Windows 7', 'browser' => '*iexplore', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Firefox on Windows 7', 'browser' => '*firefox', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Google Chrome on Windows 7', 'browser' => '*googlechrome', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), );
protected function setUp() { $this->setBrowserUrl(self::BASURL); }}
Base TestCase
44
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class TestCase extends PHPUnit_Extensions_SeleniumTestCase{ //const TEST_HUB = '217.21.179.192'; const TEST_HUB = '192.168.56.101'; const TEST_PORT = 12666;
const USERNAME = '[email protected]'; const PASSWORD = 'test1234'; const BASURL = 'http://www.theialive.com';
public static $browsers = array ( array ( 'name' => 'Internet Explorer 8 on Windows 7', 'browser' => '*iexplore', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Firefox on Windows 7', 'browser' => '*firefox', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Google Chrome on Windows 7', 'browser' => '*googlechrome', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), );
protected function setUp() { $this->setBrowserUrl(self::BASURL); }}
array ( 'name' => 'Internet Explorer 8 on Windows 7', 'browser' => '*iexplore', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT,),
Base TestCase
45
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class TestCase extends PHPUnit_Extensions_SeleniumTestCase{ //const TEST_HUB = '217.21.179.192'; const TEST_HUB = '192.168.56.101'; const TEST_PORT = 12666;
const USERNAME = '[email protected]'; const PASSWORD = 'test1234'; const BASURL = 'http://www.theialive.com';
public static $browsers = array ( array ( 'name' => 'Internet Explorer 8 on Windows 7', 'browser' => '*iexplore', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Firefox on Windows 7', 'browser' => '*firefox', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Google Chrome on Windows 7', 'browser' => '*googlechrome', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), );
protected function setUp() { $this->setBrowserUrl(self::BASURL); }}
array ( 'name' => 'Firefox on Windows 7', 'browser' => '*firefox', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT,),
Base TestCase
46
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class TestCase extends PHPUnit_Extensions_SeleniumTestCase{ //const TEST_HUB = '217.21.179.192'; const TEST_HUB = '192.168.56.101'; const TEST_PORT = 12666;
const USERNAME = '[email protected]'; const PASSWORD = 'test1234'; const BASURL = 'http://www.theialive.com';
public static $browsers = array ( array ( 'name' => 'Internet Explorer 8 on Windows 7', 'browser' => '*iexplore', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Firefox on Windows 7', 'browser' => '*firefox', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), array ( 'name' => 'Google Chrome on Windows 7', 'browser' => '*googlechrome', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT, ), );
protected function setUp() { $this->setBrowserUrl(self::BASURL); }}
array ( 'name' => 'Google Chrome on Windows 7', 'browser' => '*googlechrome', 'host' => self::TEST_HUB, 'port' => self::TEST_PORT,),
Modify MarkTaskDoneTest <?php/** * Class MarkTaskDoneTest * * @group Selenium */require_once 'TestCase.php';class MarkTaskDoneTest extends TestCase{ public function testMarkTestAsDone() { $this->open("/"); $this->click("link=login"); $this->waitForPageToLoad("30000"); $this->type("id=email", TestCase::USERNAME); $this->type("id=password", TestCase::PASSWORD); $this->click("id=signin"); $this->waitForPageToLoad("30000"); $this->click("link=Test demo"); $this->waitForPageToLoad("30000"); $this->assertEquals("Done", $this->getText("xpath=//th[5]")); $this->click("link=[EDIT]"); $this->waitForPageToLoad("30000"); $this->assertTrue($this->isElementPresent("id=done")); $this->click("link=sign off"); $this->waitForPageToLoad("30000"); }}
47
Require the TestCaseand extend it
Running test
48
49
Benefits
• run your tests on mul'ple browsers• detect flaws in specific browsers (e.g. IE6)-‐ adapt your apps to solve these flaws
50
51
Con'nuous Integra'on
52
More informa'on
53
seleniumhq.org
54
phpunit.de
55
http://www.phpunit.de/manual/3.5/en/selenium.html
Credits
56
• apple store: hOp://www.flickr.com/photos/jtjdt/3571748777• checklist: hOp://www.flickr.com/photos/alancleaver/4439276478• flat 're: hOp://www.flickr.com/photos/anijdam/2468493546/• first place: hOp://www.flickr.com/photos/evelynishere/
3417340248/• gears: hOp://www.flickr.com/photos/wwarby/4782904694• steps: hOp://www.flickr.com/photos/ben_salter/1407168763• browsers: hOp://www.flickr.com/photos/richoz/3791167457• gears: hOp://www.flickr.com/photos/freefoto/5982549938• danger: hOp://www.flickr.com/photos/armchairbuilder/
7345497766• informa'on: hOp://www.flickr.com/photos/twicepix/2650241408/• elephpant: hOp://www.flickr.com/photos/drewm/3191872515
Contact
57
Michelangelo van DamZend Certified Engineer
email: [email protected]: michelangelovandamtwitter: @DragonBe
tel EU: +32 15 34 52 90tel US: 202 559-7401
www.in2it.be
facebook.com/in2itvof | @in2itvof
Contact us forConsultancy - Training - QA - Webdesign
Thank you
59