joomla! testing - j!dd germany 2016

Joomla! TestingYves Hoppe

@yveshoppe

@compojoom

Javier Gomez

excellent presentation! congratulations.

Why?

Deliver better and reliable application

Validate there is no regression

Make sure your application runs as expected

Trust your own application

Makes debugging much easier

TestsIn Joomla!

Unit testsIntegration testsSystem testsJavaScript testsCodeStyle (phpcs)

Joomla! Automated Testing Team

Unit testing

In Unit testing ()

we test a small and specific amount of code

Sample method

public function isTheAnswer($number){

if ($number === 42) {

return true; } return false;}

Sample test

class TestCase extends \PHPUnit\Framework\TestCase{

public function testIsTheAnswer(){

$this->assertTrue(isTheAnswer(42));$this->assertFalse(isTheAnswer(1));

// … Best practice: One assertion per test}}

Javier Gomez

nice and funny! :)

The issue / reality

public function getTheAnswer($number){ if (is_array($number)) throw new Exception('Arrays not accepted');

if (!is_numeric($number)) // Check

// Decimal, Binary, Hex - @todo add lower / upper case check if ($number === 42 || $number === '00101010' || $number == '2A' || $number == '0x2A') return '<span class="answer-right">The number is the answer to

everything</span>'; elseif ($number == 13) return '<div class="answer-partial">What do you get when you multiple 6 x

9=</div>'; elseif ($number == 666) return 'WTF?!';

return false;}

Benefits of Unit testing

Issues can be discovered early

Acts as documentation

Forces you to write simpler and more readable code

Verifies the design

Fastest and no GUI necessary

Javier Gomez

another possible benefit: They are the fastests tests to run (and write)

Drawbacks

Does not show the absence of errors

Hard to set up useful and realistic tests

Writing unit tests after the development can be tough

Does only cover code

Unit tests in Joomla!

https://github.com/joomla/joomla-cms/tree/staging/tests/unit

PHPUnit 4.8.27

Within today 6119 different tests with 10526 assertions

We need you

Running the Joomla! unit testsClone the current staging repo from GitHub

git clone --depth 1 https://github.com/joomla/joomla-cms

Navigate with the command line to the cloned joomla instancecd joomla-cms

Install dependencies with composer (get it before)composer install

Run the tests with:‘libraries/vendor/bin/phpunit’

https://github.com/joomla/joomla-cms

Value for Joomla! Extension Developers

Downsides

Complicated to set up with Joomla!

You have to mock and bootstrap everything

Lot of work

System Tests

In System Testing (Browser testing)

we test against the installed and running application

Benefits

Tests the whole application

Real world testsTest more code

with less lines

System tests vs Unit tests

Unit tests are mainly used to test framework classes

System tests make sure the application works correctly from a user point

System tests does not require you to understand the code (or even the programming language)

System tests test the whole system not only a small part

They also cover JavaScript, CSS and markup changes

It’s not System or Unit testing - they complement each other!

System tests in Joomla!

https://www.youtube.com/watch?v=ZHo0TuA5Qjs

https://www.youtube.com/watch?v=ZHo0TuA5Qjs

Google Summer of Code 2016

Prital Patel

● Setup of browser automated tests for Joomla!

● Powered by Gherkin● For com_users● And com_content

http://youtube.com/v/ZHo0TuA5Qjs

Software used

https://github.com/joomla-projects/gsoc16_browser-automated-tests

Sample Codeception testpublic function administratorCreateWeblink(\Step\Acceptance\weblink $I) { $I->wantToTest('Weblink creation in /administrator/'); $I->doAdministratorLogin();

$I->amOnPage('administrator/index.php?option=com_weblinks'); $I->waitForText('Web Links', '30', ['css' => 'h1']); $I->checkForPhpNoticesOrWarnings();

$I->clickToolbarButton('New'); $I->waitForText('Web Link: New', '30', ['css' => 'h1']); $I->fillField(['id' => 'jform_title'], $this->title); $I->fillField(['id' => 'jform_url'], $this->url); $I->clickToolbarButton('Save & Close');

$I->waitForText('Web Links', '30', ['css' => 'h1']); $I->see('Web link successfully saved', ['id' => 'system-message-container']);

$I->see($this->title, ['id' => 'weblinkList']);}

Running the Joomla! Browser testsClone the current staging repo from GitHub

git clone --depth 1 https://github.com/joomla-projects/gsoc16_browser-automated-tests

Navigate with the command line to the cloned instance codeception foldercd gsoc16_browser-automated tests/tests/codeception


Configure ‘tests/codeception/acceptance.suite.yml’ and run the tests with:‘tests/codeception/vendor/bin/robo run:tests’

https://github.com/joomla-projects/gsoc16_browser-automated-tests

Value for Joomla! extension developers

BenefitsFast to setup and you don’t need to mock Joomla!

JoomlaBrowser has common Steps for Joomla, like:

Installation, uninstallation of extensions

Navigation, Checking for PHP notices

Creating menu items

Fast to write and easy to learn

JavaScript Tests

In JavaScript Testing (Browser based)

We cover the JavaScript files of our application.

JavaScript tests in Joomla!

Google Summer of Code 2016

Ruchiranga Wickramasinghe

● Setup of an automated test system

● Coverage reporting● For /media/system/js● Tests are merged in Joomla!

Core now

Software used

Jasmine 2.4.1, Jasmine-jQuery 2.1.1, Karma 0.13.22 and Firefox 48

Within today we have 196 different tests

Running the JavaScript! testsClone the current staging repo from GitHub


Navigate with the command line to your joomla instance javascript tests foldercd joomla-cms/tests/javascript

Install dependencies with npm (get it before)npm install

Run the tests with:‘tests/javascript/node_modules/karma/bin/karma start karma.conf.js’


Value for Joomla! Extension Developers

Benefits

No need for mocking Joomla!

No running Joomla! Installation needed

Easy to write and setup

CodeStyle tests

Code Styleand Metrics tests(Static Analysis)

check .

Javier Gomez

In case you want to use ISTQB organization language, they call this "Static Analysis":The objective of static analysis is to find defects in software source code and software models. Static analysis is performed without actually executing the software being examined by the tool; dynamic testing does execute the software code. Static analysis can locate defects that are hard to find in dynamic testing. As with reviews, static analysis finds defects rather than failures. Static analysis tools analyze program code (e.g., control flow and data flow), as well as generated output such as HTML and XML.The value of static analysis is:o Early detection of defects prior to test executiono Early warning about suspicious aspects of the code or design by the calculation of metrics, suchas a high complexity measureo Identification of defects not easily found by dynamic testingo Detecting dependencies and inconsistencies in software models such as links o Improved maintainability of code and designo Prevention of defects, if lessons are learned in developmentTypical defects discovered by static analysis tools include:o Referencing a variable with an undefined valueo Inconsistent interfaces between modules and components o Variables that are not used or are improperly declaredo Unreachable (dead) codeo Missing and erroneous logic (potentially infinite loops)o Overly complicated constructso Programming standards violationso Security vulnerabilitieso Syntax violations of code and software modelsStatic analysis tools are typically used by developers (checking against predefined rules or programming standards) before and during component and integration testing or when checking-in code to configuration management tools, and by designers during software modeling. Static analysis tools may produce a large number of warning messages, which need to be well-managed to allow the most effective use of the tool.Compilers may offer some support for static analysis, including the calculation of metrics.

Why Code Style matters

Communication is the Key

Code looks consistent

Clear rules for developers

Make Errors obvious

Code Style in Joomla!

http://joomla.github.io/coding-standards/

PHP_CodeSniffer (phpcs)

and PHPQA Tools

PHP_CodeSniffer 1.5.6 (working on upgrade)

Running the PHP_CodeSniffer checksClone the current staging repo from GitHub


Navigate with the command line to your joomla instance cd joomla-cms


Run the tests with:‘libraries/vendor/bin/phpcs --extensions=php --standard=build/phpcs/Joomla .’


Automated testing

Putting the teststogether to a bigautomatic safety net!

Javier Gomez

lol

Why?

Repeatable

Detect issues for every commit / pull request

Saves you time and money

Automated tests in Joomla!

Software used

Directly see issues at your Pull Request

Runs for every change and pull request against coreTravis:

Multiple PHP Versions:

- PHP 5.3, 5.4, 5.5, 5.6, 7.0- 7.1, HHVM (allowed to fail)

JavaScript tests (Node.js)

PHP_CodeSniffer (PHP 5.6)

Conclusion

Call for volunteers!

The Joomla! Automated Testing Working Group is looking for new members, which want to make

Joomla! better

https://volunteers.joomla.org/teams/automated-tests-working-group

https://volunteers.joomla.org/teams/automated-tests-working-group

Questions?Thank you for listening!

@yveshoppe

joomla! testing - j!dd germany 2016

Engineering