BDD USING SPECFLOW(CUCUMBER IN .NET)

PROBLEM

• ACCORDING TO A NUMBER OF STUDIES, AROUND HALF OF ALL SOFTWARE PROJECTS FAIL TO

DELIVER IN SOME SIGNIFICANT WAY.

• THE 2011 EDITION OF THE STANDISH GROUP’S ANNUAL CHAOS REPORT FOUND THAT 42%

OF PROJECTS WERE DELIVERED LATE, RAN OVER BUDGET, OR FAILED TO DELIVER ALL OF THE

REQUESTED FEATURES, AND 21% OF PROJECTS WERE CANCELLED ENTIRELY.

• SCOTT AMBLER’S ANNUAL SURVEY ON IT PROJECT SUCCESS RATES USES A MORE FLEXIBLE

DEFINITION OF SUCCESS, BUT STILL FOUND A 30–50% FAILURE RATE, DEPENDING ON THE

METHODOLOGIES USED. THIS CORRESPONDS TO BILLIONS OF DOLLARS IN WASTED EFFORT,

WRITING SOFTWARE THAT ULTIMATELY WON’T BE USED OR THAT DOESN’T SOLVE THE

BUSINESS PROBLEM IT WAS INTENDED TO SOLVE. - BDD IN ACTION

BEHAVIOR-DRIVEN DEVELOPMENT (BDD) IS A SET OF SOFTWARE ENGINEERING PRACTICES DESIGNED TO HELP TEAMS BUILD AND DELIVER MORE VALUABLE, HIGHER QUALITY SOFTWARE FASTER.

IT DRAWS ON AGILE AND LEAN PRACTICES INCLUDING, IN PARTICULAR, TEST-DRIVEN DEVELOPMENT (TDD) AND DOMAIN-DRIVEN DESIGN (DDD).

BUT MOST IMPORTANTLY, BDD PROVIDES A COMMON LANGUAGE BASED ON SIMPLE, STRUCTURED SENTENCES EXPRESSED IN ENGLISH (OR IN THE NATIVE LANGUAGE OF THE STAKEHOLDERS) THAT FACILITATE COMMUNICATION BETWEEN PROJECT TEAM MEMBERS AND BUSINESS STAKEHOLDERS.

KEY BENEFITS

• LIVING DOCUMENTATION – DOCUMENTATION CHANGES AS BEHAVIOR CHANGES AND IS IN

SOURCE CONTROL

• *COLLABORATION (3 AMIGOS QA/BA/DEV)

• EASY FOR NEW TEAM MEMBERS TO LEARN THE SYSTEM

• REAL TIME PROGRESS REPORTING – FEATURES ARE COMPLETE WHEN TESTS TURN GREEN

• END OF SPRINT DEMOS – MORE PRODUCTIVE AS WE CAN SHOW EXACTLY WHAT SYSTEM IS

DOING

• FASTER RELEASE CYCLE – TIES INTO CONTINUOUS INTEGRATION ENVIRONMENT AND COVERS

REGRESSION SCENARIOS

TOOL WE ARE USING TO ASSIST WITH OUR BDD IMPLEMENTATION

FEATURE

AS A [X]

I WANT [Y]

SO THAT [Z]

SCENARIO

•GIVEN DESCRIBES THE PRECONDITIONS FOR THE

SCENARIO AND PREPARES THE TEST ENVIRONMENT.

•WHEN DESCRIBES THE ACTION UNDER TEST.

• THEN DESCRIBES THE EXPECTED OUTCOMES.

GIVEN

THE PURPOSE OF GIVENS IS TO PUT THE SYSTEM IN A KNOWN STATE BEFORE

THE USER (OR EXTERNAL SYSTEM) STARTS INTERACTING WITH THE SYSTEM (IN

THE WHEN STEPS). AVOID TALKING ABOUT USER INTERACTION IN GIVENS. IF

YOU WERE CREATING USE CASES, GIVENS WOULD BE YOUR PRECONDITIONS.

WHEN

THE PURPOSE OF WHEN STEPS IS TO DESCRIBE THE KEY ACTION THE USER PERFORMS (OR,

USING ROBERT C. MARTIN’S METAPHOR, THE STATE TRANSITION).

THEN

THE PURPOSE OF THEN STEPS IS TO OBSERVE OUTCOMES. THE OBSERVATIONS SHOULD BE

RELATED TO THE BUSINESS VALUE/BENEFIT IN YOUR FEATURE DESCRIPTION. THE OBSERVATIONS

SHOULD ALSO BE ON SOME KIND OF OUTPUT – THAT IS SOMETHING THAT COMES OUT OF THE

SYSTEM (REPORT, USER INTERFACE, MESSAGE) AND NOT SOMETHING THAT IS DEEPLY BURIED

INSIDE IT (THAT HAS NO BUSINESS VALUE).

AND, BUT

THE AND AND BUT KEYWORDS CAN BE USED TO JOIN SEVERAL GIVEN, WHEN, OR THEN STEPS

TOGETHER IN A MORE READABLE WAY:

GIVEN I HAVE A CURRENT ACCOUNT WITH $1000

AND I HAVE A SAVINGS ACCOUNT WITH $2000

EXAMPLE FEATURE/SCENARIO

FEATURE: SCORE CALCULATION

IN ORDER TO KNOW MY PERFORMANCE

AS A PLAYER

I WANT THE SYSTEM TO CALCULATE MY TOTAL SCORE

SCENARIO: GUTTER GAME

GIVEN A NEW BOWLING GAME

WHEN ALL OF MY BALLS ARE LANDING IN THE GUTTER

THEN MY TOTAL SCORE SHOULD BE 0

MIKE COHN’S TEST AUTOMATION PYRAMID

CURRENT STATE OF AUTOMATION AT NORDSTROM

EXECUTABLE SPECIFICATIONS

• SPECFLOW AND SOAP SERVICE LAYER AUTOMATION FRAMEWORK

• SPECFLOW AND SELENIUM WEBDRIVER AUTOMATION FRAMEWORK

• SPECFLOW AND RESTSHARP/REST SERVICE LAYER AUTOMATION FRAMEWORK

LOW LEVEL EXECUTABLE SPECIFICATIONS

• SPECFLOW UNIT TESTS ARE CREATED/MAINTAINED BY DEVELOPERS BUT REVIEWED BY 3

AMIGOS

SCENARIO OUTLINE

SCENARIO OUTLINE: EARNING INTEREST

GIVEN I HAVE AN ACCOUNT OF TYPE <ACCOUNT-TYPE> WITH A BALANCE OF <INITIAL-BALANCE>

WHEN THE MONTHLY INTEREST IS CALCULATED

THEN I SHOULD HAVE EARNED AT AN ANNUAL INTEREST RATE OF <INTEREST-RATE>

AND I SHOULD HAVE A NEW BALANCE OF <NEW-BALANCE>

EXAMPLES:

| INITIAL-BALANCE | ACCOUNT-TYPE | INTEREST-RATE | NEW-BALANCE

| 10000 | CURRENT | 1 | 10008.33

| 10000 | SAVINGS | 3 | 10025

| 10000 | SUPERSAVER | 5 | 10041.67

EMBEDDED TABLES

SCENARIO: TRANSFERRING MONEY BETWEEN ACCOUNTS WITHIN THE BANK

GIVEN I HAVE THE FOLLOWING ACCOUNTS:

| ACCOUNT | BALANCE |

| CURRENT | 1000 |

| SAVINGS | 2000 |

WHEN I TRANSFER 500.00 FROM CURRENT TO SAVINGS

THEN MY ACCOUNTS SHOULD LOOK LIKE THIS:

| ACCOUNT | BALANCE |

| CURRENT | 500 |

| SAVINGS | 2500 |

BACKGROUND

BACKGROUND:

GIVEN I HAVE A "SALE" TRANSACTION TYPE

AND I HAVE A "POS" TRANSACTION DATA SOURCE

AND I HAVE A "USD" CURRENCY CODE

AND I HAVE A VALID STORE NUMBER

HOOKS

• [BEFORETESTRUN]/[AFTERTESTRUN]

• [BEFOREFEATURE]/ [AFTERFEATURE]

• [BEFORESCENARIO] OR [BEFORE]/ [AFTERSCENARIO] OR [AFTER]

• [BEFORESCENARIOBLOCK]/[AFTERSCENARIOBLOCK]

• [BEFORESTEP]/[AFTERSTEP]

HOOK EXAMPLE

TAGS

SHARING BETWEEN STEPS

•CONTEXT INJECTION

•FEATURECONTEXT

•SCENARIOCONTEXT

SPECRUN

• (HTML) REPORT AS PRIMARY OUTPUT

• FAST INTEGRATION TEST EXECUTION BY PARALLELIZATION

• ADVANCED EXECUTION METRICS: EXECUTION TIME, MEMORY USAGE, MEMORY LEAKING, BENCHMARKING

• DETECTION OF “RANDOM FAILURES”

• TEST VARIATIONS (E.G. RUNNING THE SAME SUITE FOR IE AND FIREFOX)

• SUPPORT FOR APPLYING DIFFERENT TEST CONFIGURATIONS

• PROVIDE FAST FEEDBACK ABOUT FAILURES (PRIORITIZE FAILING TESTS FIRST, STOP AFTER A NUMBER OF

FAILURES, ETC.)

PRETTY TEST RESULT REPORTS OUT OF THE BOX

CAN EASILY SWITCH BETWEEN UNIT TEST PROVIDERS

EASILY PARALLELIZE TEST RUNS

PICKLES

*PICKLES* CAN PRODUCE OUTPUT IN DIFFERENT FORMATS:

• STATIC HTML: A SET OF HTML FILES WITH MINIMAL JAVASCRIPT

• DYNAMIC HTML: A JAVASCRIPT-RICH SINGLE PAGE APP WITH INTEGRATED SEARCH

• WORD: MICROSOFT WORD

• EXCEL: MICROSOFT EXCEL

• JSON: A CUSTOM JSON FORMAT

BDD IN ACTION

Q&A

SOURCESHTTP://WWW.SPECFLOW.ORG/

HTTP://MARTINFOWLER.COM/BLIKI/UBIQUITOUSLANGUAGE.HTML

HTTP://DANNORTH.NET/INTRODUCING-BDD/

HTTPS://GITHUB.COM/TECHTALK/SPECFLOW-

EXAMPLES/BLOB/MASTER/BOWLINGKATA/BOWLINGKATA-

NUNIT/BOWLING.SPECFLOW/SCORECALCULATION.FEATURE

HTTP://GUIDE.AGILEALLIANCE.ORG/GUIDE/UBIQUITOUS.HTML

HTTP://MARTINFOWLER.COM/BLIKI/TESTPYRAMID.HTML

HTTPS://WWW.MANNING.COM/BOOKS/BDD-IN-ACTION

HTTP://WWW.SPECFLOW.ORG/PLUS/RUNNER/

HTTPS://WWW.NUGET.ORG/PACKAGES/PICKLES/

HTTPS://SITES.GOOGLE.COM/SITE/UNCLEBOBCONSULTINGLLC/THE-TRUTH-ABOUT-BDD