Continuous Delivery and DevOps

Moving from Staged to Pervasive Testing

Intelligent Testing Conference 2015

Rob Lambert@rob_lambert

www.thesocialtester.co.uk

SlowBoring

Ineffective testing

I’ve spent a lot of time doing

It’s no surprise then that we didn’t ship very often

Test case management system of DOOM

Mind the gap

A simple calculation

Anyone who can click a mouse

became a tester….doesn’t that

say something about the testing?

I can’t believe it took so long to test

Slow

Boring

Ineffective

I don’t mind helping out but my word, testing is so boring

We ran 10 million tests and we still had issues!

I joined NewVoiceMedia just before the last giant release.

We vowed never to do a year long release again.

Let’s do something different

Why stop doing big releases?

This is what we shipped(One tower)

This is what our customers now needed (a whole bridge)

Photo by Vaidotas Mišeikis - Clifton Suspension Bridge at Dusk - February 19, 2011

When your customer’s needs move quickly, so too must your release process.

The ultimate feedback loop

How did we make the change?

We created a vision

1. Adopt Agile2. Prioritise work3. Bring Dev and Ops together - DevOps4. Everyone tests. Test all the time.5. Become “one” with data

The VisionWeekly Releases

1. Removed testers from the centre of the universe

Testers are important, but no more than any other member of the team

Instead we put “Testing” at the centre of our universe

2. Focus on automating checks

BDDTDDCIUI Testing

3. Ate our own product

Dog-fooding (Pre-Production)

Event logs, internal customer feedback and regression tests

4. Tested and monitored after release

TRUSTOwn usageMonitoring

Every team supports their work in to production

Data from production tells us how our product is behaving

This is immensely powerful in guiding your testing

5. Emphasised discovery over assumptions

Exploratory Testing

What is the system telling us?

Testing that is data informed

6. Learned.

• 5 Whys on everything• Continuous Improvement• Training and personal development

“It’s like studying for a degree but getting paid for it”

7. Dealt with fires

• “If it hurts. Keep doing it”

• Political• Squash “Naysaying”

and “Silent Plotting”

Everyone knows how to test.

But it’s not that sort of testing you need.

You need testing that is relevant.

• At least one release each week, sometimes more

• We have more to improve though• Feedback from live is guiding our testing• Feedback from our testing is immediate

Did it work? And is that it now?

Thank you. Any Questions.NVMwww.newvoicemedia.com

How To Remain Relevant – The Bookwww.howtoremainrelevant.com

Cultivated Management Seminar Daywww.cultivatedmanagement.com

Blogwww.thesocialtester.co.uk

Twitter@rob_lambert