experimental product development
TRANSCRIPT
![Page 1: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/1.jpg)
i290 lean/agile product management unit 6: experimental product development
@jezhumble https://lapm.continuousdelivery.com/
This work © 2015-2016 Jez Humble Licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
![Page 2: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/2.jpg)
identify experiments to test hypotheses
understand how to do outcome-based planning
describe hypothesis-driven development
understand why small batches are important
define A/B testing and the culture it enables
learning outcomes
![Page 3: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/3.jpg)
EpicTheme
Story
![Page 4: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/4.jpg)
impact mapping
Gojko Adzic, Impact Mapping
![Page 5: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/5.jpg)
working backwards
http://www.allthingsdistributed.com/2006/11/working_backwards.html
![Page 6: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/6.jpg)
@jezhumbleJeff Gothelf “Better product definition with Lean UX and Design” http://bit.ly/TylT6A
hypothesis-driven delivery
We believe that
[building this feature]
[for these people]
will achieve [this outcome].
We will know we are successful when we see [this signal from the market].
![Page 7: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/7.jpg)
![Page 8: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/8.jpg)
COST OF EXPERIMENTS
8
Production Software
SPEED
COST
new services
feasibility spike
service substitution
integration
Quantitative forecasting
real-time price experiment
Data sampling and modeling tests
Sketches & Paper Prototypes
Interactive Prototype
Software demo
Interviews & surveys
micro-niche
Wizard of Oz
VIABILITY (BUSINESS) | DESIRABILITY (CUSTOMER) | FEASIBILITY (TECH)
![Page 9: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/9.jpg)
exercise
• choose a hypothesis from your assignment
• design an experiment to test your hypothesis
• what do you expect the results to be?
• what result will confirm your hypothesis?
• what result will disprove your hypothesis?
• how soon can we get the result?
![Page 10: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/10.jpg)
“Etsy’s Product Development with Continuous Experimentation” Frank Harris and Nellwyn Thomas | http://bit.ly/19Z5izI
![Page 11: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/11.jpg)
“Etsy’s Product Development with Continuous Experimentation” Frank Harris and Nellwyn Thomas | http://bit.ly/19Z5izI
![Page 12: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/12.jpg)
“Etsy’s Product Development with Continuous Experimentation” Frank Harris and Nellwyn Thomas | http://bit.ly/19Z5izI
![Page 13: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/13.jpg)
Jon Jenkins, “Velocity Culture, The Unmet Challenge in Ops” 2011 | http://bit.ly/1vJo1Ya
![Page 14: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/14.jpg)
do less
“Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3 were successful at improving the key metric!”
“Online Experimentation at Microsoft” | Kohavi et al | http://stanford.io/130uW6X
![Page 15: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/15.jpg)
“I think building this culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate.”
http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html
![Page 16: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/16.jpg)
@jezhumble
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
WORK IN SMALL
BATCHES
![Page 17: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/17.jpg)
higher quality
you can stop at any time with a working system
faster feedback (assuming people pay attention)
higher motivation
less rework
working in small batches
Don Reinertsen, Principles of Product Development Flow, ch5.
![Page 18: Experimental Product Development](https://reader031.vdocuments.net/reader031/viewer/2022021919/5879a7e31a28ab082c8b73b1/html5/thumbnails/18.jpg)
further reading
https://www.infoq.com/presentations/controlled-experiments
http://xp123.com/articles/resources-on-set-based-design/
Tom DeMarco & Tim Lister, Waltzing with Bears
Humble et al, Lean Enterprise ch 9
Gojko Adzic, Impact Mapping