8 rollouts a day
TRANSCRIPT
![Page 1: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/1.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
8 Rollouts a dayKristian Köhntopp
![Page 2: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/2.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Most interesting!
![Page 3: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/3.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
This is what we do
![Page 4: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/4.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
This is what we do
![Page 5: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/5.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
This is what we do
• Most hotels worldwide are not in a chain.
• Many hotels communicate by fax.
• Guests need help.
• Self-Service okay, Personal if needed.
• 110+ countries, 40+ languages.
![Page 6: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/6.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Problem statement
![Page 7: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/7.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Most likely your problem is different
Problem statement
![Page 8: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/8.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
• Rapid growth - constantly:
• Last years maximumreached early in the new year
• Year after year
Problem statement
![Page 9: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/9.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
• The growth pattern is predictible and stable.
• Due to growth, good practice is constantly obsolete.
• New bottlenecks appear all the time, the larger part of the year we operate in 'blue water'.
Problem statement
![Page 10: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/10.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Problem statement
The Mission Statement:
Create the required
organisational
personal
technical
structure to match the expected size in time.
![Page 11: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/11.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Problem statement
There are no constants. Embrace Change.
![Page 12: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/12.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
About booking
• 2006:
• Under 40 persons in IT.
• They know the code, because they wrote it. And the CPAN modules it uses.
• Conditional code in production. Collection of statistics.
• No Ops team.
![Page 13: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/13.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
About booking
• Long startup time for new devs.
• Unstable, slow rollouts.
• Code hard to test, too many variants.
• Code hard to test, changing too fast.
• Drowning in Ops work, too many variants.
![Page 14: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/14.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 15: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/15.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 16: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/16.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 17: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/17.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 18: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/18.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 19: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/19.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
![Page 20: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/20.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
• Validate requirements: Do they actually help improving the biz?
![Page 21: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/21.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
HIghest Paid Persons OpinionExperiments kill HIPPOs
![Page 22: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/22.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
![Page 23: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/23.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
• Data proves:
• HIPPO ideas are 0% better than improvements by random strangers.
![Page 24: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/24.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
• Data proves:
• HIPPO ideas are 0% better than improvements by random strangers.
• Conclusion:
• The only reliable way to improve is to run many experiments.
![Page 25: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/25.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
• Data proves:
• HIPPO ideas are 0% better than improvements by random strangers.
• Conclusion:
• The only reliable way to improve is to run many experiments.
• Velocity: Roll out code as fast as possible,
• and prove it with real users.
![Page 26: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/26.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
http://blog.ads.pof.com/2012/04/03/ throw-everything-you-know-about-ads-out-the-window-pics-inside/
![Page 27: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/27.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
http://blog.ads.pof.com/2012/04/03/ throw-everything-you-know-about-ads-out-the-window-pics-inside/
Experiments
![Page 28: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/28.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
http://blog.ads.pof.com/2012/04/03/ throw-everything-you-know-about-ads-out-the-window-pics-inside/
Experiments
![Page 29: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/29.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
http://blog.ads.pof.com/2012/04/03/ throw-everything-you-know-about-ads-out-the-window-pics-inside/
Experiments
![Page 30: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/30.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Experiments
• »Experiment« Framework
• Conditional Code,
• measure impact on conversion, performance, etc.
![Page 31: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/31.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Velocity
• Focus is on capturing quantitative (business) performance data.
• and a low Time To Rollout.
![Page 32: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/32.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 33: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/33.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Proven requirements
ExperimentalImplementation
![Page 34: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/34.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Proven requirements
• Successful Experiment:
• Full On, even if it is painful! It is earning money!
• Optimize that code.
• Trailing performance team sees only business-positive code.
![Page 35: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/35.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Proven requirements
• Better insight than tests: The best simulation of a real user is…
• There is a term for this: Testing In Production
http://www.thetestingplanet.com/2011/11/the-future-of-software-testing-part-one-testing-in-production/http://www.setheliot.com/blog/wp-content/uploads/2012/03/STP2012_503_A_to_Z_Testing_in_Production-v2.0.pdf
![Page 36: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/36.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Testing in production
• Experiment framework
• decoupling deployment/activation
• ramped activation
• exposure control
![Page 37: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/37.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Testing in production
• Load/Capacity test in production
• determine true capacity, true failure mode
• use a probe to determine latency
• concentrate real user load on few frontends, observe probe latency
![Page 38: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/38.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Testing in production
• Telemetry
• modern browsers can report page load times back
• add additional system data
• Data mining
• real time monitors
• offline data mining
![Page 39: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/39.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Decision
• Velocity. Scalability. Performance.
• Prove business impact.
• Scale the method. Create more great old ones.
![Page 40: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/40.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Scale the method
Wait.What method?
![Page 41: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/41.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
![Page 42: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/42.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
Risk compensation is an effect whereby individual people may tend to adjust their behaviour in response to perceived changes in risk. …
Another way of stating this is that individuals will behave less cautiously in situations where they feel "safer" or more protected.
http://en.wikipedia.org/wiki/Risk_compensation
![Page 43: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/43.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
Education
Visibility
Empowerment
![Page 44: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/44.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
![Page 45: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/45.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
Education: You should know your trade.
We're painting you blue.
![Page 46: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/46.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
Visibility: Effects of changes should be obvious.
Linear, continuous systems.
![Page 47: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/47.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Shared Spaces
Empowerment:
Take away all mechanism that makes decisions = Give back responsibility
![Page 48: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/48.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Booking Rule #1
If you break it,
will you even notice?
![Page 49: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/49.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Booking Rule #2
If you break it,
can you fix it?
![Page 50: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/50.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Craftsmanship
The ability to debug a user story front to back,
from Biz to deployment.
![Page 51: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/51.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Culture of Failure
A good fail is fast and early.
There is a lesson to take away from a good fail.
Bad fails are extended suffering and pain.
![Page 52: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/52.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
So what exactly?
• Hire experienced people
• Real-world goal vs. SE academicscode earning money vs. beautiful code
• with a proven track record in OSS
• working in a cooperative environment, self-organizing, self-driven personality
![Page 53: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/53.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
So what exactly?
• We are not building on a green lawn in the middle of nowhere
![Page 54: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/54.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
So what exactly?
![Page 55: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/55.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
So what exactly?
• We are not building on a green lawn in the middle of nowhere,
• but we are also very aware of the fact that code has a finite lifetime.
• In fact, more than 90% of our code changes are thrown away.
![Page 56: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/56.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Components
• Experimentation
• Testing in Production
• Shared Spaces
• Education, Visibility, Empowerment
• A culture of failure
![Page 57: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/57.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
So how does this relate to DevOps?
![Page 58: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/58.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
![Page 59: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/59.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Kill all sysadmins!
• BOFH is a poisonous term
![Page 60: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/60.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Kill all sysadmins!
• BOFH is a poisonous term
• Infrastructure Developer
• automate infrastructure tasks
![Page 61: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/61.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Kill all sysadmins!
• BOFH is a poisonous term
• Infrastructure Developer
• automate infrastructure tasks
• Inhouse Operations Expert
• support developers before and after failure
![Page 62: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/62.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
![Page 63: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/63.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Infrastructure Developer:
![Page 64: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/64.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Infrastructure Developer:
• Make sure the same features work at a 10x scale.
![Page 65: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/65.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Infrastructure Developer:
• Make sure the same features work at a 10x scale.
• Make sure things work with zero manual interactions.
![Page 66: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/66.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Infrastructure Developer:
• Make sure the same features work at a 10x scale.
• Make sure things work with zero manual interactions.
• Mindset differs from feature dev a lot.
![Page 67: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/67.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
![Page 68: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/68.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Methods differ
![Page 69: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/69.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Methods differ
• Scrum-ish vs. Kanban-ish vs. Firefighting
![Page 70: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/70.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Methods differ
• Scrum-ish vs. Kanban-ish vs. Firefighting
• All have prio lists
• Feature dev: release oriented, timeboxed
• Infrastructure dev: mission oriented, # of things in flight
• Translator, Agent: look at top10 things in flames right now, pick any
![Page 71: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/71.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
![Page 72: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/72.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Inhouse Operations Expert
• Feature dev works on feature front to back, biz to deploy
![Page 73: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/73.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Inhouse Operations Expert
• Feature dev works on feature front to back, biz to deploy
• Infrastructure expert offer local special knowledge, possible solutions and their 'prices'
• Enable dev to make informed decision
![Page 74: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/74.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Inhouse Operations Expert
• Feature dev works on feature front to back, biz to deploy
• Infrastructure expert offer local special knowledge, possible solutions and their 'prices'
• Enable dev to make informed decision
• Dev then still may choose to break things
![Page 75: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/75.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
![Page 76: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/76.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Example:
• DBAs are not 'owning' their databases
• Dev can get advice on schema design, query help, etc at any timπe
• Dev can also elect to go forward against this advice
![Page 77: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/77.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations POV
• Example:
• DBAs are not 'owning' their databases
• Dev can get advice on schema design, query help, etc at any timπe
• Dev can also elect to go forward against this advice
• The experts do not have control
![Page 78: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/78.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Lock in
• We are in fact locked in…
• Code changes must be small to debug, so we MUST roll out often.
• "Book": No rollout for x months, took weeks to reintegrate.
![Page 79: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/79.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Lock in
• Dependence on functional and fast monitoring:
• Eventlogging breaks: We cannot roll out.
• Monitors all over the place.
• Obsessed with data-based reasoning.
![Page 80: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/80.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Lock in
• Individual changes must be small and rolled out fast:
• We cannot use GIT branches, they are tools to accumulate large diffs.
• Unified codebase everywhere.
• We need to roll out systems even if they do not change.
![Page 81: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/81.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Lock in
• We are load testing weekly:
• Codebase changes quickly.
• A lot of really bad code is put life in experiments, has capacity impact.
• Load testing?
• In production, playing w/ LB weights, of course.
![Page 82: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/82.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
It works for us
"Have you broken the site already?""No.""What am I paying you for?" -- CEO to new dev over lunch
![Page 83: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/83.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Craftsmanship
• Hire experts: "Fachwissen" They know all possible design choices and the price tags for each of them.
• Teach history: "Sachwissen"The choices we made.
![Page 84: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/84.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Craftsmanship
• C-Level endorsement.
• Teach business to devs, dev to business people.
![Page 85: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/85.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Craftsmanship
• No cop out: You can get counselling, but the choice to roll out is yours.
• You wrote it. You are the only expert on that code we have. Your call.
• Avoiding risk compensation.
![Page 86: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/86.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
End-to-end responsibility
![Page 87: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/87.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
End-to-end responsibility
• DevOps is magic:
![Page 88: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/88.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
End-to-end responsibility
• DevOps is magic:
If you treat adult expertsas responsible adult experts, they will handle themselves as responsible adult experts.
![Page 89: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/89.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
No biz vs. dev
• We are doing many small projects.
• We are creating tight feedback loops.
• We are working interdisciplinary. Business people sit with devs, as needed. Operations people sit with devs, as needed.
![Page 90: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/90.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations as a service
![Page 91: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/91.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations as a service
• Primary goal: Velocity.
• Everything else we do as needed.
![Page 92: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/92.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations as a service
• Primary goal: Velocity.
• Everything else we do as needed.
• We are deploying imperfect, badly performing or otherwise broken code, crap schema designs, or break internal rules. Tough luck. Deal.
![Page 93: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/93.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Operations as a service
• Primary goal: Velocity.
• Everything else we do as needed.
• We are deploying imperfect, badly performing or otherwise broken code, crap schema designs, or break internal rules. Tough luck. Deal.
• Some people can't. They quit.
![Page 94: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/94.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Should you do it?
• Should you do it?
• Website. We control the rollout.
• High growth environment.
• 8 months/year in blue water.
• »Slack is good.« Baby fat eaten by growth rapidly anyway.
![Page 95: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/95.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Hell yes
• Treat your responsible adults as responsible adults, and they will handle themselves as responsible adults.
• Kill HIPPOs with data.
• Measure. Real world data. For that, you need to be live.
![Page 96: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/96.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
Live by the rules
If you break it, will you even notice?
If you break it, can you fix it?
![Page 97: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/97.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
http://booking.com/jobs
Join us!
![Page 98: 8 Rollouts a day](https://reader034.vdocuments.net/reader034/viewer/2022050719/55c17c43bb61eb21798b459d/html5/thumbnails/98.jpg)
Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices
http://booking.com/jobs
Join us!