1

Powerful New Hexawise Features

Agenda

2

• Intro and “Quickie Demo”

- What is Hexawise? What doesn’t it do?

- How does pairwise testing and related methods work?

- Advantages and Limitations of the “Old” Hexawise

• Hexawise “2.0” - The Most Powerful New Features We’ve Ever Released:

- Requirements Traceability

- More Powerful Test Design Capabilities

- Faster Test Script Documentation (including Expected Results)

- Better Auto-Scripting

• Demo of New Features

• Wrap Up with Questions and Answers

- Suggestions for Good Sources of Information

What is Hexawise?

3

Define Requirements

Export Test Scripts into Test

Mgmt. Tool

It’s a test design tool.

Testers use it to do these two things better & faster.

DocumentTest Scripts

Select Test Conditions

ExecuteTest Scripts

Find Bugsand Verify Behavior

But that’s definitely not all Hexawise does….

What is Hexawise? (Cont’d)

4

Export Test Scripts into Test

Mgmt. Tool

… Hexawise also improves test EXECUTION efficiency & effectiveness.

The tests you generate with Hexawise are more powerful than the tests you would select by hand. Specifically, tests generated using Hexawise are optimized to:

• Be as varied as possible,• Minimize wasteful repetition, and • Maximize test coverage in a manageable number of tests.

ExecuteTest Scripts

Find Bugsand Verify Behavior

What DOESN’T Hexawise Do?

5

Hexawise is not a test management tool (like Quality Center).

It’s also not a tool that automates test execution (like QTP).

When and Where is Hexawise used?

6

Export Test Scripts into Test

Mgmt. Tool

ExecuteTest Scripts

After testers generate tests in Hexawise, they usually export them to a tool like Quality Center.

Hexawise generates tests for teams that do both: - Manual testing, and - Automated testing.

Poll Results: Most Popular Hexawise Benefit?

7

How? Test generation algorithms maximize testing coverage.

DocumentTest Scripts

Select Test Conditions

ExecuteTest Scripts

Find Bugsand Verify Behavior

How? If you want to, you can execute fewer, more powerful tests.

How? Hexawise automates many test design and test

documentation steps.

1. You document tests faster…

2. You complete test execution faster…

3. You consistently find more bugs.

4. Your team makes more informed, data-driven decisions.

How? From much more precise coverage analysis reporting.

Where Do Testers Use Hexawise?

8

Everywhere.

Is Hexawise Applicable to My System?

9

Yes.

“When testing anything: "No matter what kind of interface it has, you can always identify interesting things to vary."

- Elisabeth Hendrickson – in her brand new book “Explore It!”

Agenda

10

• Same Easy to Use Features as Always, to Increase Your…

- Test selection speed

- Test documentation speed

- Testing thoroughness• “Quickie Demo”

- Advantages and Limitations of the “Old” Hexawise• Plus The Most Powerful New Features We’ve Ever Released:

- Requirements Traceability

- More powerful Test Design Capabilities

- Faster Documentation with Expected Results

- Better Auto-Scripting• Demo of New Features

Charlie’s Chocolates – Release 7

11

The “interesting things to vary”…

12

Select Test Conditions

Simply (A) Enter test conditions, and (B) click here.

As before, to do this

How does Hexawise prioritize tests?

Short version:

1) Pairwise: In this example – using the default “pairwise” coverage strength - Hexawise will select 16 tests from 576 total possible ones. Those 16 tests include at least one test of every possible pair of Values.

2) Rationale: The vast majority of your defects that sneak by undetected into production will be triggered if you tested for every possible pair of Values.

3) Manually-Selected Tests Have Many Holes: If you started to test this way tomorrow, even with fewer tests, you would find many more defects than you currently do because you would be closing thousands of gaps that exist today.

4) First Draft vs. Final: These 16 are a “first draft” set of powerful tests. Not a final set. You can and should modify them based on your knowledge.

5) Turn the Coverage Dial Up or Down: You can easily generate however many tests you want depending upon your time vs. thoroughness preferences: a dozen?, a hundred?, all 576?

For more information, data, and real-world case studies, see: pairwisetesting.com

13

14

… testers would transform powerfully optimized sets of test conditions like these…

DocumentTest Scripts

Before, to do this

15

…into “test scripts” like these below using the Auto-Script feature. (Note that these have both: (A) detailed instructions and (B) test conditions).

Advantages & Limitations: “Old” HexawiseTest Script Documentation:

While we were able to

Quickly select and document new test scripts…

And re-write the tester instructions “instantly” for each test…

Those test scripts…

Did not include Expected Results.

16

Advantages & Limitations: “Old” HexawiseRequirements / Ability to Include Specific Complex Combinations:

While we were able to

Quickly generate new sets of highly-varied test scripts…

And those tests had maximum coverage possible with minimum repetition…

We did not have the ability to…

Include specific combinations of Values to test for our

complex requirements.

17

So What’s New in “Hexawise 2.0”?

18

Charlie’s Chocolates – Release 8Let’s see how “Hexawise 2.0” handles changes for the next release:

New Features to Test in Release 8 of Charlie’s Sales Platform:

1. Charlie now accepts MasterCard

2. Charlie now offers 2-day shipping

Additional “Trouble Area” to Test More Thoroughly in Release 8:

3. Single item orders are working well; when multiple items of an item are

ordered, odd things have been happening in Release 7.

19

20

We need to add some new inputs (as usual)…

… which will lead to an entirely new set of “scrambled” tests which need new ERs.

21

… tons of new Expected Results!

Good news: we can now generate Expected Results automatically!

22

Which means creating test scripts like these is now easy and fast:

23

Once you generate your simple rules, hundreds (or thousands!) of these Expected Results will be generated with the push of a button… without a single mistake.

So far so good, but how can we test a convoluted warning message like this?!

24

http://37signals.com/svn/posts/3239-a-great-customer-experience-nutscom-goes-the-extra-mile

This “warning” feature can only be tested by specifying many specific Values to appear in a test.

This example (a result of both great customer service and good testing) was “seen in the wild” by one of my heroes:

Charlie’s Chocolates – Release 8And how does “Hexawise 2.0” handle these additional testing goals?

Two Requirements to Test (from new Business Rules):

1. Fizzy Lifting Drinks ordered on November 11 should get an 11% discount.

2. “The complex conditional melted chocolate notification”

• If customer selects 2-day delivery,

• … and the type of food being shipped is made of chocolate,

• … and it would actually take 3-days to deliver (b/c of a Sunday or Holiday),

• … and it is predicted to be hot enough to melt chocolate outside,

• …. Then notify the customer there is a risk the chocolate might melt.

25

Here’s how to think about entering Requirements in Hexawise:

26

Ask “what is the Expected Result of the requirement?”

Ask “What is the smallest number of test inputs you need to specify in order to see if the requirement is true or not?”

Enter those test inputs (and only those) into the Requirements screen.

First –

Second –

Third –

Requirements

27

“Fizzy Lifting Drinks ordered on November 11 should get an 11% discount.”

1. Expected Result =?

“should get an 11% discount.”

2. Smallest Number of Values to trigger the discount = ?

2:

Type of Candy must = Fizzy Lifting Drinks

Date Ordered must = November 11th.

3. Now just enter those two Values (and only those two

values!) into the Requirements screen and enter the ER!

Add the Values in Screen:

28

Requirements Feature (Cont.)

29

How many specific Values do we need to trigger it?

4. So simply select these four values from the drop down menus

on the Requirements screen and add your Expected Result:1. Customer selects 2-day delivery, and

2. The type of food being shipped is made of chocolate, and

3. It would actually take 3-days to deliver (b/c of a Sunday or

Holiday), and

4. A weather prediction system predicts it will be hot.

What is the Expected Result we want to trigger?

“Then notify the customer there is a risk the chocolate might melt.”

To add a test for the second “complex melting chocolate notification”

requirement, we follow the same process:

“Traditional” Recommended Sources

Explanations of how Pairwise Testing (and similar, more sophisticated test design methods) consistently pack more testing coverage into a manageable set of tests than hand-selected tests

• pairwisetesting.com

Questions about features (e.g., “How do I…?”)• help.hexawise.com

Training / understanding test design concepts• training.hexawise.com

An introductory, fact-based article for skeptical managers• A scholarly, evidence-based

article and explanation of benefits from a 10 project study that I co-wrote with 3 PhD’s

30

Fun Recommended Sources

• A fun presentation about pairwise software testing - powerful test design

• Nutty video announcement of our new “Hexawise 2.0” features

31

http://www.youtube.com/watch?v=3F1eQag6XUg

hexawise.com/?p=531

Sign up for a free account now:

32

hexawise.com/free

Try Hexawise out yourself.