how to avoid being fooled by percent changes

What the board room can learn from the shop floor

or, how to avoid being fooled by percent changes

By Ben Joneshttp://dataremixed.com

Meet Jim

(Jim says Hi)

Jim runs a company selling…tractors.

Tractor Sales are down 3.6% this year

-3.6%

Jim wants answers…

Meet Jim’s two sales guys, Joe and Larry

Joe Larry

Larry’s numbers are up 2.9%…

Larry

+2.9%

Joe’s numbers are…not.

Joe-8.3%

What should Jim do?

(Jim is polishing up his resume…)

but what if it was all just a big misunderstanding?

Instead of just looking at % change “Year

Over Year”…

-3.6%

…what if we plotted units sold by month?

and what if we applied some brand new

techniques?

http://amzn.to/dOWKBwWalter Shewhart written in 1939

to understand the variation in the data:

x-bar (or average)

UCL – Upper Control Limit

LCL – Lower Control Limit

“Common Cause” variation

“Special Cause” variation

“Special Cause” variation

So wait, only LAST month is a “down” month…

…and WHO do you think caused THAT?

Joe Larry

Probably Joe, right?

Joe-8.3%

But if we look at Joe’s numbers over time…

-8.3%

2009: 96 Units

But if we look at Joe’s numbers over time…

-8.3%

2010: 88 Units

Joe

We see no statistically significant change!

“Common Cause” variation

But Larry can’t be the reason, his numbers are

up, right?

Larry

+2.9%

Let’s look at Larry’s numbers over time…

2009: 69 Units

Let’s look at Larry’s numbers over time…

2010: 71 Units

Looks like Larry has some explaining to do about

December…

Larry“Special Cause” variation

These two approaches tell two very different stories…

Percent Change “YOY”

Control Charts

Blaming Joe would be mistaking NOISE for a

SIGNAL

Rewarding Larry would be missing the opportunity

to identify a SIGNAL

Either way, Jim’s business loses an opportunity to

LEARN and GROW…

…and the fate of our sales guys’ careers hangs

in the balance…

Joe Larry

The Morals of the Story:• DON’T just compare % change

• DON’T assume a % change is statistically significant just because you don’t like it.

• DO use a Control Chart to filter out noise in a data set and identify signals

• NEVER adjust the y-axis to cross the x-axis at anything other than 0 (did you catch that trick? The column charts grossly over-exagerate the changes)

For bar charts, always set the y-axis to 0

y-axis starts at 157 y-axis starts at 0

Appears to be 4X less

Got all that, Jim?

More about Control Charts (in business)

• Understanding Variation– Donald J. Wheeler

Presentation by Ben Joneshttp://dataremixed.com