quality @ANGIESLIST5/28/2014

1. DEVELOPMENT TESTING

DEVELOPMENT TESTINGCODE functions and performs as expected

$0.10 defects

DEVELOPMENT TESTINGunit | api | isolated functional | benchmark

+thousands run per commit+pre merge

+assume function will break

DEVELOPMENT TESTING"catch bugs on your laptop"

isolated | local db | small dataset | no external dependencies

DEVELOPMENT TESTINGKEYS TO success

+sub-10 minute build+enforced code coverage in build

+functional/benchmark test coverage+TDD

2. RELEASE TESTING

RELEASE TESTINGCODE integrates and performs as expected

$1,000 defects

RELEASE TESTINGfunction | api | behavior | benchmark

+managed test environment+weekly production snapshot

+assume integrations and data will break

RELEASE TESTINGPRE-PROD e2e integration

+integration testing with real data+prod db snapshot

+external dependencies on

RELEASE TESTINGPRE-PROD Performance

+established benchmarks+production snapshot (scrubbed)

+test high impact, revenue critical functions

RELEASE TESTINGKEYS TO success

+no performance degradation under load+feature validation and verification

+demos to stakeholders+release speed (fix => production)

3. PRODUCTION TESTING

DOGFOODING

+share release candidates with AL employees+mobile and web

MONITORING+notification on thresholds (for dev)

PRODUCTION TESTINGKEYS TO success

+no disruption for AL employees+quick turnaround on monitoring failures/warnings

ANGIE'S LIST

Jeff Rogers@jrojers

jrojers@gmail.com