Businessassurance$ Testing

WHITEPAPER APRIl 2015

www.hcltech.com

AuthOr:Arish Arbab is a Software Engineer at HCL Singapore Pte Limited, having expertize on Agile GUI/API Automation methodologies.

Smart BDD Testing Using Cucumber and JaCoCo

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 2

TABLE OF CONTENTS

INTRODUCTION 3

PROBLEM FACED 3

SOLUTION APPROACH 4

BENEFITS 6

IMPROVEMENTS 7

APPLICABILITY TO OTHER PROJECTS 7

UPCOMING FEATURES 7

REFERENCES 8

APPRECIATIONS RECEIVED 8

ABOUT HCL 9

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 3

INTRODUCTIONBehavioral Driven Development (BDD) testing uses natural language to describe the “desired behavior” of the system that can be understood by the developer, tester and the customer. It is a synthesis and refinement of practices stemming from TDD and ATDD. It describes behaviors in a single notation that is directly accessible to domain experts, testers and developers for improving communication. It focuses on implementing only those behaviors, which contribute most directly to the business outcomes for optimizing the scenarios.

PROBLEM FACEDBy using the traditional automation approach, and if given a summary report of automated tests to the business analysts, then it is guaranteed that the report would be met with a blank expression. This makes it tricky to prove that the tests are correct—do they match the requirement and, if this changes, what tests need to change to reflect this? The whole idea behind BDD is to write tests in plain English, describing the behaviors of the thing that you are testing. The main advantage of this is that the tests reflect the user stories and the business requirements of the application. Therefore, the results generated can be read by a non-technical person, e.g., a project sponsor, a domain expert, a business analyst, and the tests can be validated against the requirements for better operational efficiency.

To achieve the BDD approach, there is an open source tool—Cucumber—that parses the scenarios/tests that are written in Gherkin language (GWT). This tool also interacts with the API/Junit/Source code of an application, using its step definitions test code. The best option to get the business intent into our tests is to have clear deliverables and acceptance criteria in the user stories. These can be then translated into feature tests containing scenarios written in Gherkin Language.

BDD is the “outside-in” software development methodology, which has changed the tester’s role dramatically in the recent years and bridges the communication gap between business and technology. This innovative approach has the potential to shape a new organization standard with new SDET roles.

Problems faced while implementing the new framework:

y Justifying the coverage done by this framework becomes difficult as the complexity of the application code increases

y Exposing APIs/Source Code was difficult as developers disagreed

y Converting the software requirement into GWT format at the later stages of the software development utilized more efforts

y Requiring testers with good technical/coding knowledge

y Extending the Junit by using the mocking techniques or spring-based test was a hindrance when the complexity of the code increased

y Convincing the BTMs to review the scenarios on every day was difficult

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 4

SOLUTION APPROACHTraditionally, software requirement was written in simple English sentences, but to convert those requirements into GWT format required an extra effort that should be properly handled. As GWT approach results in better quality of product, requirement should be written in GWT format by the business analyst at the early stages of software development rather than writing the requirement in domain-specific language. Set of scenarios should depict a particular user story. Implementing this approach in a later stage may increase the cost of a project.

Example of writing the scenarios in the abovementioned format would be:

Traditional software requirement: As a user, I want to login to the application with valid credentials.

Simple Cucumber Feature File in Gherkin Format:

Feature: Login to the system,

User should be allowed to login with valid credentials

Scenario: Login Success

Given The login page is opened

When I input valid username and password in the textboxes

And I click the login button

Then I logged in successfully.

Step definitions for matching scenarios:

Similarly for “When” and “Then,” step definitions are created.

Corresponding step definitions are then written, and the result is displayed in the above format that assures developer/tester/BA and customers that a particular software requirement has been successfully implemented.

Common method of testing an application at the API level requires its API or web services. However, if the application has neither the API available nor the developers who want to create it for testing purposes, then the problem arises to test the code written by developer without any extra development effort. To overcome this problem, Junit classes that are written by developers can be used to call the respective service methods. However, if the unit testing is being done using mocking technique, then it would be difficult to test the actual functionality of the application. So, in order to use the Junit methods effectively, dependency injections techniques can be used at the testing end. Such usage would call the Junit method by passing a particular dependency, which, in turn, calls the real service method.

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 5

BDD approach—detailed explanation of using Cucumber by leveraging Junit

The pictorial representation of Cucumber automation framework:

Data(.xlsx) DatabaseTest Result

HTML Reports, Screen shots, logs of failed scenarios etc.

Feature File(Scenarios)

Cucumber Main(Driver)

Cucumber Step Definitions

Junit Classes containing unit test methods

Model Service Methods

DB, Test DataAnd Reports

Cucumber

Junit Classes

App Service Methods

Flow of the BDD framework using Cucumber is as follows-

y The test data, which is to be used in the scenarios, is stored in Excel sheet or MySql Database.

y The test data is then passed to feature file, containing scenarios written in GWT format. Scenarios are denoted by certain tags.

y Driver class decides which scenario among the entire features files should be executed by proving a proper tag value. Before running a particular feature, dependency used in application is injected to avoid mocking unit testing framework.

y Appropriate matching method defined in the Cucumber step definition file is called.

y From the step definition, appropriate unit test method of the parameterized Junit class is called and, thus, test data is passed into it.

y Unit test method, which, in turn, calls the real service method by passing the test data to it.

y Response is then returned from the service method to unit test method, which, in turn, passes the information to Cucumber step definition file.

y Expected response is validated against the returned (actual) response in the step definition.

y Logs are created upon failure or success and an auto HTML report is created, depicting the passed and failed scenarios in appropriate colors.

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 6

Now, if we have the API available, then, instead of calling the Junit test method, the API method is called directly and rest of the flow remains the same.

Then the question arises, “Whether or not we are covering all the areas of an application with our test as the scenarios written are very behavioral specific?” Here the need for the code coverage arises. JaCoCo is one of the tools, which can be integrated with Cucumber Test, and the code coverage report generated can be easily read by customers/developers/business analysts and testers. The traditional way of testing may work out with testers, having less technical knowledge. Nevertheless, as we can clearly point out by seeing the above solutions and exposure of code to tester. The need for a tester with good coding skills is prerequisite to implement this approach successfully.

JaCoCo report

Cucumber test execution can be integrated into CI (Jenkins) server and can be executed depending upon the need. Now, when we have everything in place—scenarios, test code and test results—it becomes difficult to convince the BTMs to review the scenarios on a daily basis. There is another open-source tool available that overcomes these difficulties: Pickles, open-source living document generator. This tool generates the document consisting of the scenarios, which can be easily reviewed by the customer as it supports advance search capabilities.

BENEFITS

Key Benefits

y Cucumber and JaCoCo are open-source tools, which provide equal benefits that other paid tools provide.

y Around 80% reduction in the test execution time as compared to manual testing.

y Cucumber can be integrated with existing tools like Selenium, TestComplete and QTP.

y Quality of an application increased as the defect is found at an early stage of development.

y Cucumber can be used to test REST and SOAP Web services, thereby, easily adjusting in traditional way of API automation.

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 7

y Unlike other API testing tools, it produces more detailed, easy-to-understand test execution report for client, developer, BA and tester regardless of their technical knowledge.

y One of the eminent features of Cucumber is that the separation on three levels of feature, steps and world makes it easier to reuse certain parts of the code.

y Scenarios can be written in different languages, including French, German, Chinese and Hindi.

y Cucumber with Code Coverage tool (JaCoCo) provides efficient and advanced way of testing applications.

Available for nine programming languages, Cucumber is an open-source tool for BDD. The tool is available in languages such as Ruby, JVM-based languages, JavaScript and .NET (.NET languages are supported through the SpecFlow tool.).

IMPROVEMENTS

The key learning is as follows:

y Team gained knowledge on API automation using smart BDD tools (Cucumber and JaCoCo).

y Team was able to understand the requirements better due to the constant interaction with BAs, and the interaction with developers helped the team increase its technical knowledge.

y Detailed test results report was generated in the UI automation by integrating Selenium with Cucumber.

y Exposure to continuous integration servers increased, which helped in scheduling the overnight batch execution.

y Team collaboration and communication techniques—used for involving customers—such as Pair Programming and Code Reviews became frequent.

APPLICABILITY TO OTHER PROJECTS y BDD testing is not restricted to any domain, and, hence, can be used in any domain.

y It can be easily adjusted/merged to the traditional automation approach.

y By leveraging Cucumber, API-based projects can also use these techniques.

y Junit/Nunit test method can be used along with Cucumber to achieve BDD.

y Cucumber dependencies are openly available, which can be easily integrated with any project.

UPCOMING FEATURES y Collaborate across the globe with a real-time shared editor

y Access the latest version of your executable specifications anytime

SMART BDD TESTING USING CUCUMBER AND JACOCO | ApRIl 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 8

y Link to issue trackers

y Edit incorrect specifications instantly, and see the results as they appear

REFERENCEShttp://en.wikipedia.org/wiki/Cucumber_(software)

http://cukes.info/

http://www.ibm.com/developerworks/library/a-automating-ria/

https://github.com/cucumber/cucumber/wiki/Cucumber-JVM

http://en.wikipedia.org/wiki/Behavior-driven_development

http://guide.agilealliance.org/guide/bdd.html

APPRECIATIONS RECEIVEDProof of Concept was accepted and greatly appreciated by the client. Appreciation for this practice was also provided by the head and the project manager.

ABBREVIATIONSAbbreviation ExpansionTDD Test Driven DevelopmentATDD Acceptance Test Driven DevelopmentJaCoCo Java Code CoverageUI User InterfaceSDET Software Development Engineer in TestGWT Given-When-ThenBA Business AnalystHTML Hyper Text Markup LanguageCI Continues Integration API Application Program InterfaceJVM Java Virtual MachineBTM Business Technology Manager

Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration, applied innovation and new generation partnership models that put your interest above everything else. Right now 105,000 Ideapreneurs are in a Relationship Beyond the Contract™ with 500 customers in 31 countries. How can I help you?

ABOUT HCL

About HCL Technologies

HCL Technologies is a leading global IT services company working with clients in the areas that impact and redefine the core of their businesses. Since its emergence on the global landscape, and after its IPO in 1999, HCL has focused on ‘transformational outsourcing’, underlined by innovation and value creation, offering an integrated portfolio of services including software-led IT solutions, remote infrastructure management, engineering and R&D services and business services. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare & Life sciences. HCL takes pride in its philosophy of ‘Employees First, Customers Second’ which empowers its 100,240 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 5.7 billion, for the Financial Year ended as on 31st December 2014 (on LTM basis). For more information, please visit www.hcltech.com

About HCL Enterprise

HCL is a $6.8 billion leading global technology and IT enterprise comprising two companies listed in India – HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India’s original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 105,699 professionals of diverse nationalities, who operate from 31 countries including over 505 points of presence in India. HCL has partnerships with several leading global 1000 firms, including leading IT and technology firms. For more information, please visit www.hcl.com