(almost) everything i know about testing i learned playing poker

T17 Personal Excellence 5/5/16 13:30

(Almost) Everything I Know about Testing I Learned Playing Poker

Presented by:

Matthew Eakin

Centric Consulting

Brought to you by:

350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ [email protected] -‐ http://www.stareast.techwell.com/

Matthew Eakin Centric Consulting With more than twenty years of technical, leadership, and planning experience, Matthew Eakin is the national automation architect (Ruby/Cucumber) with the software quality assurance and testing practice of Centric Consulting. His experience in all aspects of the SDLC, combined with development skills, has helped Matthew to be an effective agile testing practitioner and coach. He speaks extensively on Gherkin Scripting and is in high demand as a behavior-‐driven development and acceptance test-‐driven development coach. Matthew now leads his workshopsâ€”Agile Testing, Gherkin Scripting, and A Manual Testers Guide to the Ruby/Cucumber Frameworkâ€”nationally and teaches customized variations of the programs at client sites.

NOTICE: Proprietary and Confidential

This material is proprietary to Centric Consulting, LLC. It contains trade secrets and information which is solely the property of Centric Consulting, LLC. This

material is solely for the Client’s internal use. This material shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express

consent of Centric Consulting, LLC.

© 2016 Centric Consulting, LLC. All rights reserved

(Almost) Everything I know

about testing I learned playing

poker

Matthew Eakin, National Automation Architect [email protected] @MatthewEakin MatthewEakin

At Centric, we deliver unmatched client experiences that center on

collaboration. Working together, we implement lasting solutions that work.

Solutions that matter.

mailto:[email protected]

Does anyone want to play some poker?

Need 6 “volunteers”

To play Texas Hold ‘Em

.

Texas Hold ‘Em & Software Testing

Some testing concepts we are going to see in action: 1.Knowledge of the domain

2.Risk Analysis

3.Planning vs. Writing a Test Plan

4.Adjusting your efforts as you go

5.Knowledge gathering

6.Estimation

7.Resource allocation

What we are going to do…

At least 2 hands of Texas Hold ‘Em

Hand 1: “Agile”

•Constant evaluation of risk

•Constant knowledge gathering

•Constant planning and adjusting

Hand 2: “Waterfall”

•Risk evaluation conducted up-front (even though our knowledge is limited)

•Knowledge is gathered up-front (even though our knowledge is limited)

•Planning conducted up-front (even though our knowledge is limited)

•Adjusting is discouraged. You MUST follow the Plan!!

Hand 3 and beyond: You choose the approach…

Rules of Texas Hold ‘Em (in a nutshell)

•All players are dealt 2 cards face down. You can look at them, but don’t

show anyone.

•A round of betting will follow

•5 additional cards will be dealt into the “Community” face up. •3 cards will be dealt first followed by a round of betting

•1 card will be dealt followed by a round of betting

•1 card will be dealt followed by a final round of betting

•Make the best 5 card hand

•Hand order: •Royal flush – 10-A all same suit

•4 of a kind – 4 cards match, i.e. 9-9-9-9-3

•Full House – 3 of one card, 2 of another, i.e. K-K-K-5-5

•Flush – all cards same suit, i.e. 2-3-6-8-Q all clubs

•Straight – all cards in a row, i.e. 4-5-6-7-8

•3 of a kind – 3 cards match, i.e. K-K-K-5-7, 2-2-2-7-9

•2 pair – i.e. K-K & 6-6 in your hand

•Pair – 2 cards match, i.e. K-K, 6-6

•High card in hand

Hand 1:

Agile

Knowing your Table/Team

What is their Knowledge of the Domain?

Relationship to testing

What is your testing teams knowledge of each

domain?

•Knowledge of Testing

•Knowledge/experience with the process (agile or waterfall)

•Technical Knowledge

•Knowledge of the Application

The Deal

•Everyone is dealt 2 cards

•At this point you know very little

about the deck of cards or what

cards other players could potentially

have.

9

Assessing Risk

Risk Assessment: Key concept to both Poker &

Software Testing

•In Poker: assess your probability of winning the hand with your cards

•Probability of your cards

•Probability of opponents cards (what you think they have)

•Probability with chip stacks

•Conduct risk analysis on every card drop

•Conduct risk analysis on every bet

•Conduct risk analysis on every player action

Relation to testing

Testing is Risk Minimization

•In Testing: use risk analysis to determine what to build/test first

•Conduct risk analysis on product backlog

•Conduct risk analysis on sprint backlog

•Conduct risk analysis on every User Story

•Conduct daily risk analysis on work in progress

Betting

•As each player bets you begin to

know more about:

• Their skill level

• Their aversion to risk

• What they have in their hand

•Hint: players won’t bet unless they

think they can win the hand, right?

WRONG!!! • Beware the bluff

12

• Everyone bets

Relation to Testing

13

Resource Allocation

• As a project progresses you learn more about:

• the risks associated with these changes of the application,

• your testing team

• the entire project team

• You might want to allocate more/less resources to one area of the app

• You might want to focus more on automation

Bluffing

• With every code drop you learn more about the code base/application

• Is the project really 80% done?

• Are the developers bluffing?

• Does this change the risk? Change resource allocation?

The Flop

•The flop •3 cards are dealt face-up. All players can use these cards to create the best 5

card hand possible.

•You now know a little more about the deck of cards.

•You conduct another risk assessment of your now 5 cards and determine if

you should bet or fold.

14

Relation to Testing

15

Planning

“Planning is everything. Plans are nothing.”

–Field Marshal Helmuth Graf von Moltke

• Plan for (almost) every possible outcome. When it happens you are

ready for it

• Do you have the tools/equipment/skills to handle each outcome?

• What will you do when only 50% of your scripts pass? What if 100% on

first pass?

• What if development delivers code late? What if scope changes? What if

your test environment isn’t available on day 1 of testing?

• Many “risks” are identified in a test plan. But what are you going to do if

they actually happen?

• Can you plan out the rest of the hand?

The Flop - betting

16

• Everyone bets

– As each player bets continue to conduct risk assessments on them. Does their betting indicate they have a good hand? A bad hand?

The Turn & the Turn Bet

•The Turn •1 card is dealt face-up (4 total in the

community cards).

17

• Everyone bets

– Can you figure out what hand everyone has yet?

– Can you predict your final hand?

Relation to Testing

18

Predicting – a hypothesis – what do you think will happen?

• As a project progresses can you predict how/when it will end?

• Can you predict your final hand?

• Can you predict opponents final hand?

• Tools to help you predict

• Burn-up/Burn-down charts

• Run Reports

• Defect Reports

• Very important for testers to be able to predict the future

• Discuss bad news early

• Critical team discussions

The River & the Final Bet

•The river •The final card is dealt face-up into the

community cards (there are now 5).

19

• Final bets

– Your last chance to get everyone's $$$

The Show

•Before we show our hands… •Can you guess each players hand?

•Who do you think won the hand?

•The show •Everyone shows their cards.

•Best hand takes the pot.

20

What we learned…

•Understand your teams knowledge of the domain

•Constant risk analysis

•Adjust resources as needed

•Planning is critical

•Can you predict the outcome of a project accurately?

21

Hand 2:

Waterfall

The 2nd Hand – the “Waterfall Hand”

•We are going to play this hand as if we were testing in a

waterfall project.

•All planning will be done up-front rather than as we go.

•All risk analysis will be conducted up-front rather than as we

go.

•Resources will be allocated up-front rather than as we go.

•Your knowledge of the requirement is assumed to be 100%.

23

The Test Plan

•Each player is given a Test Plan which they must follow

EXACTLY for the entire round.

•Plan will include: •What every other players cards will be

•Cards in the Board will be revealed.

•Did you win?

24

Sample Test Plan

Deal:

Player 1 will be dealt Q, 10

Player 2 will be dealt 9, 4




Player 6 (you) will be dealt A, K

Flop: Cards will be 5, 8, 9

Turn: Card will be Q

River: Card will be J

You will end up with a Straight A, K, Q, J, 10

You will lose to Player 4 who has a Straight Flush 5 , 6, 7, 8, 9

Relation to Testing

26

Estimating in Waterfall

• Every Test Manager has had to provide a resource estimate for a project

they know almost nothing about.

• Have your estimates every been accurate? +_ 50%?

• Estimates are very difficult when there is so much unknown

• Based on past experiences you can start getting closer on you

estimates

• Tools can help

Betting

•In waterfall you allocate ALL

resources up-front

27

• Everyone bets for all rounds

The Deal

•Everyone is dealt 2 cards •Players must stick to their Test Plan.

28

• Are there any problems with the Test Plan?

– Too bad, you must stick to the Test Plan!!!

• Did anything happen that was not covered in the Test Plan?

– How did you react?

The Flop

•3 cards are dealt to the Board •Players must stick to their Test Plan.

29

• Are there any problems with the Test Plan?

– Too bad, you must stick to the Test Plan!!!

Change Requests

30

A change request has been introduced. Since the code delivered was not as expected, numerous questions were asked of the business/BA. Some requirements have been changed and new ones have been introduced. Result:

• Each player can now change their Test Plan

– Conduct a new Risk Assessment on each Player.

– Evaluate your own stack and create a new betting plan for the Turn and River.

The Turn & the Turn Bet

•The Turn •1 card is dealt face-up (4 total in the

community cards).

31

• Everyone bets

– Can you figure out what hand everyone has yet?

– Can you predict your final hand?

The River & the Final Bet

•The river •The final card is dealt face-up into the

community cards (there are now 5).

32

• Final bets

– Your last chance to get everyone's $$$

The Show

•Before we show our hands… •Can you guess each players hand?

•Who do you think won the hand?

•The show •Everyone shows their cards.

•Best hand takes the pot.

33

What we learned…

•Writing out a Test Plan based on critical assumptions we have

very little information about is dangerous.

•A Test Plan does not fit every situation.

•Following a Test Plan, even though it doesn’t match reality, will

result in disaster for the tester.

34

NOTICE: Proprietary and Confidential

This material is proprietary to Centric Consulting, LLC. It contains trade secrets and information which is solely the property of Centric Consulting, LLC. This

material is solely for the Client’s internal use. This material shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express

consent of Centric Consulting, LLC.

© 2016 Centric Consulting, LLC. All rights reserved

Thank You

Matthew Eakin, National Automation Architect [email protected] @MatthewEakin MatthewEakin

mailto:[email protected]

Test Plan Player 1

Deal:

Player 1 will be dealt K, 10





Player 6 will be dealt A, K




You will end up with a Straight K, Q, J, 10, 9

You will lose to Player 4 who has a Straight Flush 5, 6, 7, 8, 9

Test Plan Player 2

Deal:










You will end up with a Straight Flush 5 , 6, 7, 8, 9

You WIN!!!!!

Test Plan Player 3

Deal:










You will end up with a Pair 9, 9, Q, J, 8


Test Plan Player 4

Deal:











You WIN!!!!!

Test Plan Player 5

Deal:










You will end up with a Pair 9, 9, Q, J, 8


Test Plan Player 6

Deal:











You WIN!!!!!