acceptance test driven development with specflow and friends christopher bartling joel levandoski...
Post on 18-Dec-2015
226 views
TRANSCRIPT
Acceptance Test Driven Development
with SpecFlow and Friends
Acceptance Test Driven Development
with SpecFlow and Friends
Christopher Bartling
Joel Levandoski
Christopher Bartling
Joel Levandoski
Contact informationContact information
Christopher Bartling
Twitter: @cbartling
Joel Levandoski
Twitter: @joellevandoski
Christopher Bartling
Twitter: @cbartling
Joel Levandoski
Twitter: @joellevandoski
Administrative stuffAdministrative stuff
Presentation and demo are available on BitBucket.org
https://bitbucket.org/cebartling/presentations
Presentation and demo are available on BitBucket.org
https://bitbucket.org/cebartling/presentations
TopicsTopics
Acceptance Test Driven Development (ATDD)
SpecFlow for .NET
Behavior Driven Development (BDD) tool
WatiN and WebAii
Live demonstration
Questions
Acceptance Test Driven Development (ATDD)
SpecFlow for .NET
Behavior Driven Development (BDD) tool
WatiN and WebAii
Live demonstration
Questions
Acceptance Test Driven DevelopmentAcceptance Test Driven Development
ATDD
Acceptance tests are executable specifications of desired behavior and functionality of the system
Expressed in language of the problem domain
Automated
Regression suite
Development is driven from the outside-in
ATDD
Acceptance tests are executable specifications of desired behavior and functionality of the system
Expressed in language of the problem domain
Automated
Regression suite
Development is driven from the outside-in
ATDD vs. TDDATDD vs. TDD
TDD is extremely valuable, but you need more
Achieve great unit test coverage and still not deliver value to the customer
ATDD focuses on complete features and functionality
ATDD: macro view
TDD: micro view
TDD is extremely valuable, but you need more
Achieve great unit test coverage and still not deliver value to the customer
ATDD focuses on complete features and functionality
ATDD: macro view
TDD: micro view
SpecFlow for .NETSpecFlow for .NET
BDD testing framework
Integrates nicely with Visual Studio
Acceptance tests manifest themselves as features and scenarios
SpecFlow generates automated test from features
SpecFlow tests run as normal xUnit tests
Visual Studio test runner, build server
BDD testing framework
Integrates nicely with Visual Studio
Acceptance tests manifest themselves as features and scenarios
SpecFlow generates automated test from features
SpecFlow tests run as normal xUnit tests
Visual Studio test runner, build server
FeaturesFeatures
Describe some piece of functionality of the system
Maintained in a .feature file
Plain readable text
Understandable by all parties, including business
Gherkin format, popularized by Cucumber
Features contain one or more scenarios
Describe some piece of functionality of the system
Maintained in a .feature file
Plain readable text
Understandable by all parties, including business
Gherkin format, popularized by Cucumber
Features contain one or more scenarios
Feature file exampleFeature file example
Feature: Calculate Net Present Value
In order analyze the profitability of a project
As a project manager
I want to be able to calculate the project’s Net Present Value
Scenario: A project is rejected
Given a project to evaluate
When data is entered reflecting a failing project ROI scenario
Then the net present value calculator will reject the project
Feature: Calculate Net Present Value
In order analyze the profitability of a project
As a project manager
I want to be able to calculate the project’s Net Present Value
Scenario: A project is rejected
Given a project to evaluate
When data is entered reflecting a failing project ROI scenario
Then the net present value calculator will reject the project
ScenariosScenarios
• A scenario describes a single acceptance test for a feature
• Most features are composed of multiple scenarios
• SpecFlow generates a test for each scenario
• The name of the test is generated from the scenario title
• A scenario describes a single acceptance test for a feature
• Most features are composed of multiple scenarios
• SpecFlow generates a test for each scenario
• The name of the test is generated from the scenario title
Scenario exampleScenario example
Scenario: A project is rejected when its NPV is negative
Given a project to evaluate
When data is entered reflecting a failing ROI scenario
Then the net present value calculator will reject the project
Scenario: A project is rejected when its NPV is negative
Given a project to evaluate
When data is entered reflecting a failing ROI scenario
Then the net present value calculator will reject the project
Scenario stepsScenario steps
• Used to compose a scenario
• Custom code to automate your application
• Describe preconditions, triggering action, and verification of outputs for the acceptance test
• Given: preconditions
• When: triggering action
• Then: behavior and state verifications
• Used to compose a scenario
• Custom code to automate your application
• Describe preconditions, triggering action, and verification of outputs for the acceptance test
• Given: preconditions
• When: triggering action
• Then: behavior and state verifications
Scenario step exampleScenario step example
Given a project to evaluate
is matched to the following step definition binding…
[Given(@"a project to evaluate")]
public void GivenAProjectToEvaluate() {
. . .
}
Given a project to evaluate
is matched to the following step definition binding…
[Given(@"a project to evaluate")]
public void GivenAProjectToEvaluate() {
. . .
}
Set up and tear downSet up and tear down
• Attributes for before and after events
• TestRun
• Feature
• Scenario, ScenarioBlock
• Step
• Specificity achieved when used with tags
• Attributes for before and after events
• TestRun
• Feature
• Scenario, ScenarioBlock
• Step
• Specificity achieved when used with tags
TagsTags
• Markers that can be applied to features and scenarios
• Useful for selectively mixing in behavior
• Used to categorize scenarios
• Some unit test frameworks support this categorization tagging
• Custom tags you define
• Markers that can be applied to features and scenarios
• Useful for selectively mixing in behavior
• Used to categorize scenarios
• Some unit test frameworks support this categorization tagging
• Custom tags you define
Tag exampleTag example
@WatiN
Scenario: Save valid sample size mid range
Given the user enters 10 as sample size
When the user selects save
Then the value is stored
@WatiN
Scenario: Save valid sample size mid range
Given the user enters 10 as sample size
When the user selects save
Then the value is stored
Tag example used with BeforeScenarioTag example used with BeforeScenario[BeforeScenario("WatiN”)]
public void BeforeScenarioUsingWatiN()
{
...
}
[BeforeScenario("WatiN”)]
public void BeforeScenarioUsingWatiN()
{
...
}
BackgroundBackground
• Common preconditions for all scenarios in a feature
• Contains one or more scenario steps
• Executed before any other steps in the scenario
• SpecFlow generates a method from the background element in the feature file in the test class
• Invoked from each scenario test case in the test class
• Common preconditions for all scenarios in a feature
• Contains one or more scenario steps
• Executed before any other steps in the scenario
• SpecFlow generates a method from the background element in the feature file in the test class
• Invoked from each scenario test case in the test class
Background exampleBackground example
Background:
Given that the welcome page is displayed
Scenario: Add a comment to a book being reviewed
. . .
Background:
Given that the welcome page is displayed
Scenario: Add a comment to a book being reviewed
. . .
Scenario outlinesScenario outlines
• Data-driven scenarios or scenario templates
• Consists of
• Scenario template specification with data placeholders
• Set of examples providing values for placeholders
• SpecFlow generates parameterized test logic for the scenario outline and individual test method for each example set
• Data-driven scenarios or scenario templates
• Consists of
• Scenario template specification with data placeholders
• Set of examples providing values for placeholders
• SpecFlow generates parameterized test logic for the scenario outline and individual test method for each example set
Scenario outline exampleScenario outline example
Scenario Outline: Score calculation tables
Given a new bowling game
When I roll the following series: <rolls>
Then my total score should be <total score>
Examples:
| game | rolls | total score |
| beginners game | 2,7,3,4,1,1,5,1,1,1,1,1,1,1,1,1,1,1,5,1 | 40 |
| one single spare | 2,8,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 | 29 |
Scenario Outline: Score calculation tables
Given a new bowling game
When I roll the following series: <rolls>
Then my total score should be <total score>
Examples:
| game | rolls | total score |
| beginners game | 2,7,3,4,1,1,5,1,1,1,1,1,1,1,1,1,1,1,5,1 | 40 |
| one single spare | 2,8,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 | 29 |
ScenarioContextScenarioContext
• ScenarioContext.Current.Pending()
• Causes the step definition to return pending
• Used to signal a stubbed step definition
• ScenarioContext.Current dictionary
• Allows you to carry context/state across step definitions participating in scenario
• ScenarioContext.Current.Pending()
• Causes the step definition to return pending
• Used to signal a stubbed step definition
• ScenarioContext.Current dictionary
• Allows you to carry context/state across step definitions participating in scenario
WatiNWatiN
Open source browser automation tool for IE and Firefox
Similar to Selenium, WebDriver, Watir
Browser abstraction
Find elements using CSS selectors
Interact with AJAX web elements
Handle popup dialogs (native and HTML)
Open source browser automation tool for IE and Firefox
Similar to Selenium, WebDriver, Watir
Browser abstraction
Find elements using CSS selectors
Interact with AJAX web elements
Handle popup dialogs (native and HTML)
WebAiiWebAii
Browser automation tool from Telerik
Automates both web 2.0 and Silverlight applications
HTML element wrappers
WaitForElement(s) support when using AJAX
Identifying elements using LINQ
Invoke JavaScript directly from test code
Browser automation tool from Telerik
Automates both web 2.0 and Silverlight applications
HTML element wrappers
WaitForElement(s) support when using AJAX
Identifying elements using LINQ
Invoke JavaScript directly from test code
Best practicesBest practices
• Write high-level specifications
• Specifications should remain stable over time
• Build a scripting interface for manipulating your system under test (SUT)
• Focus specifications on isolated behaviors
• Think of specifications in Given-When-Then format
• Use the Page Object pattern
• Write high-level specifications
• Specifications should remain stable over time
• Build a scripting interface for manipulating your system under test (SUT)
• Focus specifications on isolated behaviors
• Think of specifications in Given-When-Then format
• Use the Page Object pattern
SmellsSmells
• Specifications are constantly changing
• Specifications are composed of “sequential command executions”
• Specifications have a lot of instrumentation or fixture code
• Specification examples exhibit same structure
• Specifications are constantly changing
• Specifications are composed of “sequential command executions”
• Specifications have a lot of instrumentation or fixture code
• Specification examples exhibit same structure
Net Present Value Calculator DemoNet Present Value Calculator Demo
Used in capital budgeting
Measures the excess or shortfall of cash flows, in present value terms, once financing terms have been covered
Demo has two implementations
ASP.NET MVC
Silverlight
Used in capital budgeting
Measures the excess or shortfall of cash flows, in present value terms, once financing terms have been covered
Demo has two implementations
ASP.NET MVC
Silverlight
Questions?Questions?
Literature citedLiterature cited
http://www.concordion.org/Technique.html
http://www.telerik.com/automated-testing-tools/webaii-framework-features.aspx
http://watin.org/
http://www.specflow.org/
http://code.google.com/p/selenium/wiki/PageObjects
http://www.concordion.org/Technique.html
http://www.telerik.com/automated-testing-tools/webaii-framework-features.aspx
http://watin.org/
http://www.specflow.org/
http://code.google.com/p/selenium/wiki/PageObjects