the evolution of qa at just eat
DESCRIPTION
Презентация доклада Jim Beattie на конференции SQADays-14 English Day, Львов 7 ноября 2013TRANSCRIPT
![Page 1: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/1.jpg)
THE EVOLUTION OF QA AT JUST EATJIM BEATTIE – HEAD OF I18N, JUST EAT
![Page 2: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/2.jpg)
2
INTRO
About Me +About JUST EAT
Automation…
4 QA improvementsused by JUST EAT
1 brand new thing
Bleeding Edge!
Why things have changedFor QAs
![Page 3: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/3.jpg)
JIM - BACKGROUND
![Page 4: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/4.jpg)
JUST EAT: LOCAL TAKEAWAY ONLINE
The JE network
• 36,000 restaurants
• 13 countries
• 7M registered members
• $1BN per year generated for
restaurants!
• 1000 employees
QA• 6 QAs in UK (down from 10) • 5 in Kyiv• Everyone is responsible for
Quality
![Page 5: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/5.jpg)
5
HOW THINGS HAVE CHANGED…
![Page 6: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/6.jpg)
6
OLD SCHOOL ENGINEERING PROCESS
![Page 7: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/7.jpg)
7
ENGINEERING IS DIFFERENT NOW
Agile!!!• 1 – 3 week sprints
• Lower risk of failure
• Reduced impact of failure
• Releases are different
• Different philosophy
![Page 8: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/8.jpg)
8
WHAT DOES THIS MEAN FOR QA?
Trust more
Resist change
MISSION
ATTITUDE
Reduce
riskNot about
perfection
![Page 9: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/9.jpg)
9
CONCLUSION – ENGINEERING HAS CHANGED
JUST EAT QA NEEDED TO CHANGE AS WELL
![Page 10: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/10.jpg)
10
SO… HOW CAN WE CHANGE?
![Page 11: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/11.jpg)
11
AUTOMATION: BIGGEST RECENT CHANGE IN QA
Is automation the silver bullet?
Great, there’s no
manual testing to
do!
![Page 12: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/12.jpg)
12
CONCLUSION - AUTOMATION
START WITH AUTOMATION
TEST THE GAPS
![Page 13: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/13.jpg)
13
OK… APART FROM AUTOMATION,HOW DID WE CHANGE?
![Page 14: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/14.jpg)
14
WHAT ABOUT EVERYTHING ELSE
Efficient process
Using users
Exploratory testing(test the gaps)
Using crowds
![Page 15: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/15.jpg)
15
BE EFFICIENT!
![Page 16: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/16.jpg)
16
BE EFFICIENT!
Documentation:• Test Strategy• Test Approach• Test Plan• Entry / Exit Criteria
• Test Cases• Test Execution Matrix
• Test Evidence
What’s the value?
Any drawbacks?
![Page 17: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/17.jpg)
17
LOOK FOR THE VALUE
The value is in the contentBetter content comes from collaboration
Everyone’s got
an iPhone
(or Android!)
![Page 18: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/18.jpg)
18
CONCLUSION – BE EFFICIENT
CHUCK NORRIS DOESN’T WRITE TEST PLANS.HIS TESTS PLAN THEMSELVES.
![Page 19: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/19.jpg)
19
EXPLORE!
![Page 20: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/20.jpg)
20
EXPLORATORY TESTING
The value is in human intelligence
Where’s the gap???
Quality not docs
Break my stuff! Win a prize!Have some fuuuunCelebrate
at show and tells
![Page 21: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/21.jpg)
21
CONCLUSION – EXPLORATORY TESTING
YOU ARE MORE CREATIVE
THAN A ROBOT
PROVE VALUE AND EARN TRUST
![Page 22: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/22.jpg)
22
TAKE EXPLORATION TO ALPHA
![Page 23: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/23.jpg)
23
THE JUST EAT BUG BASH
LunchtimeNo docsEveryone WelcomeGhetto Style!
Great results every time!
![Page 24: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/24.jpg)
24
CONCLUSION – EXPLORATORY TESTING
MORE PEOPLE FIND MORE
BUGS
AND YOU GET FREE PIZZA
![Page 25: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/25.jpg)
25
CROWDSOURCING
1) URL (or APP)
2) Optional Spec
3) GO!
10,000 QAEs
Pay Per Bug
Great value
Great coverage
Drawbacks?BackendFalse positivesLow value bugsGreat coverage?
![Page 26: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/26.jpg)
26
CONCLUSION – CROWDSOURCING
THE CROWD ARE YOUR
FRIENDS
HELP THEM TO IMPOVE
PRODUCT QUALITY
![Page 27: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/27.jpg)
27
AFTER ALPHA COMES…
![Page 28: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/28.jpg)
28
BETAS
Measure KPIsMeasure server statsCollect feedback
Drawbacks?Feedback?What if…
Release to 2 servers
![Page 29: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/29.jpg)
29
BETAS – FEEDBACK
It’s good to get
feedback:-)
![Page 30: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/30.jpg)
30
CONCLUSION – BETA
WE DID IT AND
NOTHING BAD
HAPPENEDWE DON’T EVEN LABEL IT BETA
![Page 31: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/31.jpg)
31
CONCLUSIONS FROM THE HEART
![Page 32: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/32.jpg)
32
WHAT’S NEXT FOR QA?1 BRAND NEW THING ;-)
![Page 33: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/33.jpg)
33
BENEFITS WITH AUTOMATION AND MANUAL
Automation:• Repeatable• Quick• Programmable
• Triggered via CI
Manual / crowds:• Intelligent• Not so brittle
Let’s get married xxx
![Page 34: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/34.jpg)
34
RAINFOREST APP
![Page 35: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/35.jpg)
35
CONCLUSION – QAAAS
AUTOMATION MEETS
OUTSOURCING!
IS THIS THE FUTURE?
![Page 36: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/36.jpg)
36
RECAP
Automation…
3 QA advances used by JUST EAT• CUT WASTE• EXPLORE• USE GROUPS
QAAAS
Why things have changedFor QAs
![Page 37: The evolution of QA at JUST EAT](https://reader035.vdocuments.net/reader035/viewer/2022062406/558fbe4e1a28ab6e7e8b47ca/html5/thumbnails/37.jpg)
37