unit testing php-unit - phing - selenium_v2

Automated Unit Testing using PHPUnit, Selenium and Phing

Tricode Professional Serviceswww.tricode.nl

01-05-2008Sander van Beek & Patrick van Dissel

“One test is worth a thousand expert opinions”-- Bill Nye, The Science Guy

Automated Unit Testing

- Put Angels on Your Shoulders- Unit Testing

- Use It Before You Build It- PHPUnit

- Different Makes a Difference- Selenium

- Automate, Automate, Automate- Phing

Put Angels on Your Shoulders

Unit Testing is a software verification and validation method where the programmer gains confidence that individual units of source code are fit for use

A unit is the smallest testable part of an application, e.g. a method

Unit testing is regression testing, making sure made changes are not a step backwards

Unit Testing

Without Unit Testing:You write code and stick in some print statements to see the value of a few key variables. You run the code, maybe in a debug mode, and look at the results manually, fix problems that come up

Then you throw away the print statements or exit the debug mode, and move on to the next item

Unit Testing

With Unit Testing:With Unit Testing you take that familiar process and instead of throwing away debug code, you save it, and continue to run it automatically

Instead of manually inspecting the interesting variables, you write code to check for specific values in specific situations

Unit Testing

Benefits:- Provides instant feedback

Code gets exercised repeatedly. As you change and rewrite your code, test cases will check that you haven’t broken existing contracts. You can quickly identify and fix any problems

- Makes your code robustTesting helps you think through the behavior of the code, exercising the positive, negative and exceptional cases

Unit Testing

Benefits (continued):- Can be a helpful design tool

Unit testing can help you achieve a pragmatic and simpler design

- Is a confidence boosterYou’ve tested your code and exercised its behavior for a variety of different conditions:

this will give you the confidence when faced with new, high pressure tasks on tight deadlinesYou can change anything as long as the tests stay green!

Unit Testing

Benefits (continued):- Can act as probes when solving problems

Tests go red when something is wrong. If something is wrong and tests stay green, that means that new test-cases need to be created for that specific situation

- Are reliable documentationUnit tests can serve as accurate, reliable documentation

When following an easy naming-convention you can even create documentation based on unit tests

Unit Testing

Benefits (continued):- Are a learning aid

You can write unit tests against an API to facilitate your learning. The tests not only help you understand the behavior of the API but also help you quickly find any incompatible changes that might be introduced later

Unit Testing

Limitations:- Public API only

Unit tests are focused on the public API only!

Protected and private API can be tested, but wrappers are needed to get this to work. It’s a bit more work, but advised for very important/complex methods

- Tests are not created and updated by themselves

Creating and keeping unit tests up-to-date is not an automatic process. You must to have the discipline to create and update the tests yourself!

Unit Testing

Limitations (continued):- Not everything can be tested

By default, unit tests are focused on testing behavior of single units. Integration, performance, GUI and user acceptance tests are not the focus or supported by unit testing alone

Using unit testing in combination with other testing tools can extend the reach of the tests. More and more extensions are added to unit testing frameworks to support these tools

Unit Testing

Some best practices:- Don't assume the order in which tests within a test case

run- Avoid writing test cases with side effects- Do not load data from hard-coded locations on a file

system- Keep tests in the same location as the source code- Name tests properly- Keep tests small and fast- Test Behavior, Not Methods- Apply the “Too Simple to Break” Rule- Use Object-Oriented Best Practices In Your Test Cases

Use It Before You Build It

PHPUnit is a unit testing framework for PHP5

It is one of the xUnit family of frameworks that originated with Kent Beck's SUnit for Smalltalk

Wikipedia has a nice list of unit testing frameworks:http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

PHPUnit

Static Test structure

PHPUnit

Runtime Test structure

PHPUnit

Basic concepts of xUnit:– Test fixture

The set of preconditions or state needed for a test to succeed. Also known as a test context.

– Test caseA single test method within a test suite.The setUp() and tearDown() methods of the test suite are executed before and after each test case, respectively

– AssertionEvaluate the result of the test case

– Test suiteA collection of test cases, andA collection of test suites

A simple Test Suite

class ArrayTest extends PHPUnit_Framework_TestCase{ public function testNewArrayIsempty() { $array = array(); $this->assertEquals(0, count($array)); }}

A simple Test Suite

class ArrayTest extends PHPUnit_Framework_TestCase{ protected $fixture; protected function setUp() { parent::setUp(); $this->fixture = array(); } protected function tearDown() { parent::tearDown(); $this->fixture = null; } public function testNewArrayIsempty() { $this->assertEquals(0, count($this->fixture)); }}

Writing tests

- A test suite is a class that extends from PHPUnit_Framework_TestCase

- A test case is a method within a test suite, which:- Method name starts with ‘test’, or- Is annotated with ‘@test’

- The setUp() method of a test suite can be overriden and should be used to setup the fixture. setUp() is called before each test case

- The tearDown() method of a test suite can be overriden and should be used to cleanup the fixture.tearDown() is called after each test case

Executing tests

DEMO- From command-line- From Zend Studio

- Generating a result report- Code-coverage

Data providersclass DataTest extends PHPUnit_Framework_TestCase{ /** * @dataProvider provider */ public function testAdd($a, $b, $c) { $this->assertEquals($c, $a + $b); }

public function provider() { return array( array(0, 0, 0) , array(0, 1, 1) , array(1, 0, 1) , array(1, 1, 3) // fails ); }}

Testing exceptionsclass ExceptionTest extends PHPUnit_Framework_TestCase{ /** * @expectedException InvalidArgumentException */ public function testException() { throw new InvalidArgumentException(); }}

PHPUnit converts PHP errors, notices and warnings to exceptions, respectively:

- PHPUnit_Framework_Error- PHPUnit_Framework_Error_Notice- PHPUnit_Framework_Error_Warning

Testing performance

class PerformanceTest extends PHPUnit_Extensions_PerformanceTestCase{ public function testPerformance() { $this->setMaxRunningTime(2); // 2 seconds sleep(1); // 1 second }}

Testing output

class OutputTest extends PHPUnit_Extensions_OutputTestCase{ public function testExpectFooActualFoo() { $this->expectOutputString('foo'); print 'foo'; } }

PHP's Output Buffering feature is used to provide thefunctionality that is necessary for this

Advanced usage

- Code Coverage Analysis- Grouping of tests

@group- Mark tests incomplete- Mark tests skipped- Behavior-Driven development

stories- Agile documentation

--story, --tap, --testdox- Test doubles

Stubs and Mocks- Skeleton generation

- from test case to class- from class to test case

Documentation

Want to learn more?– PHPUnit pocket guide

www.phpunit.de/manual/ – Testing patterns

www.xunitpatterns.com

At the end you will receive:- A PHPUnit Cheatsheet for version 3.3- A Unit Testing best practices document- A Test-Driven-Development:

- Rhythm reference guide- Quick reference guid

SeleniumWeb application automated testing suite


24-04-2009Sander van Beek

Selenium

Selenium is a suite of tools to automate web app testing across many platforms.

Selenium...• runs in many browsers and operating systems • can be controlled by many programming

languages and testing frameworks.

Selenium

Selenium suite has 3 main tools:• Selenium IDE

– Create tests

• Selenium RC (remote contol)– Run tests in different browsers

• Selenium Grid– Scale out, speed up

Selenium

Selenium IDE

• Firefox plug-in

• Record tests (demo)

• Export them to different formats– HTML– C#– Java– PHP– Ruby– Perl– Python

Selenium IDE

Commands:• Actions

open, click, refresh

• Accessors Get state and store it

• Assertions Verify state. Have 3 modes:

- assert (halt on error)- verify (generate warning on error)

- waitFor (wait for condition to become true)

Selenium RC

• Run distributed Selenium tests on one (remote) server

Selenium RC

• Supported browsers:– Firefox 2, 3– IE 7, 8– Safari 2, 3– Opera 8, 9– Others: partially

• Supported operating systems:– Windows– Linux– OS X,– Solaris– Others (as long as they run supported browsers)

Selenium Grid

• Selenium in parallel on multiple servers (or VM’s)

Selenium Grid

• http://saucelabs.com/: Selenium on EC2

• TestSwarm: http://ejohn.org/blog/javascript-testing-does-not-scale/

Questions

More info: http://seleniumhq.org/


14-12-2008Sander van Beek

Phing An automated build system based on Apache ant

Phing

• PHing Is Not GNU make; it's a project build system based on Apache Ant. You can do anything with it that you could do with a traditional build system like GNU make, and its use of simple XML build files and extensible PHP "task" classes make it an easy-to-use and highly flexible build framework. Features include file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS operations, tools for creating PEAR packages, and much more.

Phing

• Automated build tool

• Types of automated builds:– On demand– Scheduled (nightly build)– Automated (continuous integration)

Phing

• Why ‘build’ PHP?– Improve product quality – Eliminate redundant tasks – Minimize "bad builds" – Eliminate dependencies on key personnel – Have history of builds and releases in order to

investigate issues– Save time and money - because of the reasons listed

above

Concepts

• Targets & taks

• The build file: build.xml– Project

• Target1– Task1– Task2



• Targets can be dependant on each other

• One default target (run when no task is specified on command line)

Task types

• Core– PhingCall, input, phpeval, condition, echo, exec

• File system– Copy, delete, move, mkdir, touch, available,

resolvePath

• Transformation– XsltTask, ReflexiveTask

• Packaging– pearPackageTask, zipTask, tarTask, dbdeploy,

ioncube

Task types

• Version control– svnCheckout, svnExport, svnUpdate, svnLastRevision

• Quality assurance– phpunitTask, zendCodeAnalyser, phpDocumentor,

phpCodesniffer, coverageXXX, phpLint

• Filters– Replacing, stripping comments/whitespaces, line

numbering, tidy and more

• Easy to add new tasks

Filesets

• Can be used in many tasks

<fileset dir="/etc" > <include name="httpd/**" /> <include name="php.ini" /> <exclude name=“doc/**" /></fileset>

• **/* <- Ant glob syntax

Property files

• Create environment specific builds

• E.g. build.properties, contains ‘key=value’ lines. E.g:database.user=upcpl

• Can be used to replace properties in source code - e.g. ${database.user}

<property file="build.properties" /> <reflexive> <fileset dir=“config"> <include file=“config.php”/> </fileset> <filterchain> <expandproperties /> </filterchain></reflexive>

Running phine

• Simply run ‘phing’ in the directory containing build.xmlOptionally specify a target as the first parameter

• Can also be run automated, e.g. from cruisecontrol or hudson

unit testing php-unit - phing - selenium_v2

Documents