specflow— bridge between business and development · pdf file... is an agile software...

Business Assurance & Testing

WHITEPAPER APRIl 2015

www.hcltech.com

SpecFlow— Bridge between Business and Development

AuthOrs:

Anurag Sharma [email protected]

Is a software engineer at HCL Technologies India.Currently responsible for automating the test cases at API Level and providing the API Automation solution to End-Users.

Samarth Gupta [email protected]

SpecFlow—An Open-source or Paid Tool

SPECFLOW—BRIDGE BETWEEN BUSINESS AND DEVELOPMENT | APRIL 2015

© 2015, HCl TECHNOlOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, All RIGHTS RESERVED. 2

TABLE OF CONTENTS

INTRODUCTION 3

PROBLEM FACED/ DURING IMPLEMENTATION OF GOOD PRACTICES 3

SOLUTION APPROACH 3

BENEFITS 7

LEARNING/IMPROVEMENTS 8

APPLICABILITY TO OTHER PROJECTS 8

REFERENCES 8

APPRECIATIONS RECEIVED 8

ABBREVIATIONS 8

ABOUT HCL 9



INTRODUCTIONBehavior Driven Development (BDD) is an agile software development technique focused on improving a key factor in the successful development of any software product.

BDD comes with the idea of filling the gap between the business and the development techniques. BDD centers on obtaining a clear understanding of desired software behavior through discussion with stakeholders.

PROBLEM FACED/ DURING IMPLEMENTATION OF GOOD PRACTICESSpecFlow is an open-source .NET tool that lets you write specifications using 100% Cucumber-compatible Gherkin syntax. The tool has a number of advantages such as:

y It integrates with Visual Studio (VS), which means you get File New templates for creating new feature files and step definitions.

y It gives complete VS debugger support, so you can set breakpoints on Given/When/Then lines in your .feature files and step through their execution.

y You can implement your step definitions in any .NET language.

y When you compile a project containing SpecFlow feature files, the output is a NUnit test assembly. So, you can use your favorite NUnit-compatible test runner or existing CI infrastructure to run the specifications with no additional configuration.

CONSTRAINTS

y As for the IDE, SpecFlow has full integration with Visual Studio® 2008, 2010 and 2012, but all these IDEs use paid licenses.

y To make automation more effective, we use TeamCity as our Continuous Integration platform with SpecFlow, which also requires yearly subscription.

y Visual Studio Express Edition can be used free of charge even for commercial purposes. The limitations on the edition’s functionality are borderline ridiculous. For example, it doesn’t support Microsoft’s own unit testing framework, and all add-ins are banned (like SpecFlow and ReSharper). It also doesn’t work with Microsoft’s own version control system (TFS).

SOLUTION APPROACH

INTEGRATING WITH VISUAL STUDIO EXPRESS

Install SpecFlow and NUnit

Before we get started, we need to install VS2010 Express, SpecFlow and NUnit.

1. Install VS2010 C# Express if you don’t already have it

2. Install the latest SpecFlow version from the SpecFlow



3. Install the latest NUnit version (Nunit 2.6.3) from the NUnit download page

4. Add the SpecFlow directory (C:\Program Files\TechTalk\SpecFlow or similar) and the NUnit directory (C:\Program Files\NUnit 2.5.9\bin\net-2.0 or similar) to your Windows path.

Adding SpecFlow templates to Visual Studio 2010 Express

1. While installing SpecFlow to VS2010 (non-express), it creates templates that can be accessed through the “New Item” menu. Fortunately, it is fairly trivial to add these to VS2010 Express

2. Just download the templates, unzip the directory and place the .zip files

3. (SpecFlowFeature.zip, SpecFlowStepDefinition.zip, etc.) into your Visual Studio 2010 Express C# templates directory (usual pathname—C:\Users\username\Documents\Visual Studio 2010\Templates\ItemTemplates\Visual C# or similar)

This means you now have access to the SpecFlow templates while adding a new item to your C# project.

Generating SpecFlow NUnit tests from Feature Files

When you add a SpecFlow feature to the full version of VS2010, it automatically generates a unit test file behind the feature file, so that it’s always up to date. This doesn’t work in the Express edition, so you have to use the specflow.exe file to do this for yourself. Fortunately, this can be done in a batch file and then added as an “external tool” to VS2010 Express, so you get the same functionality but not quite as neat.

1. Enable Expert Settings in VS2010 Express (if not already enabled), by choosing Tools->Settings->Expert Settings.

2. Create a batch file in your project directory called generatespecflowtests.bat with the following content:

Create an external tool setting the same as we did for the generate unit tests but with the parameters defined on this dialog:

– Command: Link to your batch file

– Arguments: $(ProjectFileName)

– Initial Directory: $(ProjectDir)

– Use Output Window: checked



Running SpecFlow NUnit tests from Feature Files

To automatically run SpecFlow NUnit tests and capture a pretty HTML result, we have to create another batch file.

1. Create a batch file in your project directory named runspecflowtests.bat or similar.

2. Include below content in your batch file.

Create an external tool setting the same as we did for the generate unit tests but with the parameters defined on this dialog:

– Command: Link to your batch file

– Arguments: $(TargetName)$(TargetExt)$(ProjectDir)$(ProjectFileName)$(BinDir)TestResult.xml

– Initial Directory: $(BinDir)

– Use Output Window: checked

Result:

Result will be published in HTML format, which will include all the test execution steps of each scenario in the Given/When/Then format (detailed). It will show the result summary-wise, feature-wise and scenario-wise:



Integrating with Jenkins

It is possible to Integrate SpecFlow with Jenkins to create a detailed HTML report.

1. Set up a new project in Jenkins

2. Add the NUnit Plugin and HTML Publisher Plugin to Jenkins

3. In your project, go to Configure -> Build -> Add a build step

4. In the dropdown, scroll down to -> Execute Windows Batch Command

5. Ensure this step is placed after your MSBuild step

6. Add the following, replacing the variables:

7. Click on Add post post-build action and select Publish HTML reports. Fill the path of HTML file created locally and the title with which you want to show the report

8. Click on Add post post-build action and select Publish Nunit test result report. In thet test report XMLs field, put TestResult.xml



9. Whenever you build the project, we will get the result in Jenkins as follows:

BENEFITSQUANTITATIVE BENEFITS—WORTH USD 29,302

0

5000

10000

15000

20000

25000

30000

35000

Total Cost involved in 1year (In $, effort cost)

Lumpsum Tool Costinvolved (In $, per year),For QTP version 10 by

HP

Grand cost involved (In $,per year)

Approach with out 'SpecFlow' i.e.via GUI Automation Testing

Approach with 'SpecFlow' i.e. viaAPI Testing

QUALITATIVE BENEFITS

y Tools used are Open Source – Around 80% reduction in test execution time as compared to manual testing

– Exposure to Continuous Integration Servers helps in scheduling the overnight batch execution



– BDD testing is not restricted to any domain, Feature file can be easily understood by any stakeholders

– Produces more detailed, easily understood test execution report

– SpecFlow can be integrated with existing tools like Selenium, TestComplete and QTP

– SpecFlow supports many languages

– SpecFlow dependencies are openly available, which can be easily integrated with any project

LEARNING/IMPROVEMENTSIt is easier to adopt, and it is best suited for the agile testing project; however, it can be equally used in other types of projects as well. It can be integrated with other tools as well.

APPLICABILITY TO OTHER PROJECTSHaving no dependency on the domain type, it can be used in any type of projects. It can easily be implemented.

REFERENCEShttp://www.specflow.org/

APPRECIATIONS RECEIVED Highly appreciated from the client, and it has suggested to implement the same for other projects in the same account.

ABBREVIATIONS

Abbreviation ExpansionBDD Behavior Driven Development

Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration, applied innovation and new generation partnership models that put your interest above everything else. Right now 105,000 Ideapreneurs are in a Relationship Beyond the Contract™ with 500 customers in 31 countries. How can I help you?

ABOUT HCL

About HCL Technologies

HCL Technologies is a leading global IT services company working with clients in the areas that impact and redefine the core of their businesses. Since its emergence on the global landscape, and after its IPO in 1999, HCL has focused on ‘transformational outsourcing’, underlined by innovation and value creation, offering an integrated portfolio of services including software-led IT solutions, remote infrastructure management, engineering and R&D services and business services. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare & Life sciences. HCL takes pride in its philosophy of ‘Employees First, Customers Second’ which empowers its 100,240 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 5.7 billion, for the Financial Year ended as on 31st December 2014 (on LTM basis). For more information, please visit www.hcltech.com

About HCL Enterprise

HCL is a $6.8 billion leading global technology and IT enterprise comprising two companies listed in India – HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India’s original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 105,699 professionals of diverse nationalities, who operate from 31 countries including over 505 points of presence in India. HCL has partnerships with several leading global 1000 firms, including leading IT and technology firms. For more information, please visit www.hcl.com