Testing Antipatterns

Matt RobinsonPuppet Labs

Anti - What?

Antipattern (Dark

Pattern):A pattern that is commonly used but is ineffective and/or counterproductive

Who has an extensive test

suite?

Who feels like they should test more?

Who dislikes, even dreads, writing

tests?

Testing Can Be Fun

TextTextTextText

http://xkcd.com/583/

For example:

Red Green TDD Addiction

Testing Framework Games

(although these can be a distraction)

Not looking like an idiot - that’s fun

right?

Why Automated Testing?

•Prevents regressions

•Improves confidence

•Documents requirements

•Catches errors before codes ships

•You fill in the blank _________

Overheard Reasons Not to

Test•Don’t trust the tests

•Who tests the tests? Quis custodiet ipsos custodes?

•Makes refactoring harder

•Not enough return on investment (ROI)

•Too hard and/or time consuming

Antipatterns that have bitten me the

most

Not Running Tests Often Enough

If it’s Hard and Important Do it

More Often

testing, deploying, releasing, exercising,

meditating, writing, public speaking

How Often?• Before deployment or release

• After every commit: CI (Continuous Integration) - Jenkins, Cruise Control, BuildBot, Cerebrus, CI Joe

• Before every commit (or push): discipline, git hooks

• Every save: autotest, watchr

Not Fixing Broken Tests Before

Committing New Code

The Slowpoke

How Slow Is Too Slow?

Unit Test Suite Took 8 Hours to Run - On One

Project

Single Test Took 2 Minutes to Start

So slow that rewriting it in PHP

would make it faster

How to Get Faster•Make your implementation code faster

•Run your tests in parallel

•Optimize your tests (new framework, trick garbage collector, rewrite culprits)

•Mark some tests as slow and only run them occasionally

The Mockery

Why mock when the real things will do?

Testing at the Wrong Level

unit, smoke, functional, integration, acceptance

Success Against All Odds

(test never fails)

Testing Implementation

Instead of Behavior

Testing Implementation

Testing Behavior

Order Dependent Failures

You’ve got too much global state

Excessive Setup

What are we testing again?

•The Dodger - Never tests desired behavior

•The Stranger - Misplaced tests

•Local Hero - Worked on my dev box

•Generous Leftovers - Fills your hard drive

•The Giant - You may have a God object

•The Loudmouth - Spams your test logs

Other Antipatterns

Summary

Avoid Antipatterns and Testing Can Be

Fun

Links

http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/

http://jasonrudolph.com/blog/testing-anti-patterns-how-to-fail-with-100-test-coverage/

http://blog.rubybestpractices.com/posts/gregory/034-issue-5-testing-antipatterns.html

http://c2.com/cgi/wiki?AntiPatternsCatalog

http://martinfowler.com/articles/mocksArentStubs.html