devops roadtrip - denver

DENVER - SEATTLE - SAN FRANCISCO - MINNEAPOLIS - NEW YORK CITY

#DevOpsRoadTrip

#DevOpsRoadTrip

JASON HAND |DevOps Evangelist

• 18 years of experience as a developer, system administrator, and support specialist

• Fully emerged into the world of agile development and the DevOps movement

• DevOps Evangelist (VictorOps)• Author of “ChatOps For Dummies”• Organizer – DevOpsDays Rockies• Contributor to Wired, TechBeacon,

InfoQ, and many other technology publications

• Co-host of “Community Pulse” – podcast on building Community in Tech

@jasonhand

Agenda12:00 - 1:00 - Registration & Lunch 1:00 - Opening Remarks | Jason Hand, DevOps Evangelist, VictorOps 1:15 - Expert Panel + Q&AFrom Start to Finish Continuous Improvement: The DevOps Journey at CraftsyMatthew Boeckman, VP of Infrastructure at CraftsyThe Why and How of DevOps at SendGrid –Mary Moore-Simmons, Engineering and Operations Manager at Sendgrid 2:15 - BREAK

#DevOpsRoadTrip

2:30 - Topical Breakout Sessions 3:30 - BREAK 3:40 - Avoiding the DevOps PitfallsKurt Bittner, Principal Analyst - Forrester ResearchAmy Demartine, Sr. Analyst - Forrester Research 4:25 - Closing Remarks

4:45 - Happy Hour & Stadium Tours

DevOps History 101

#DevOpsRoadTrip

MATTHEW BOECKMAN | Vice President , Infrastructure

• Serial entrepreneur with 18 years in technical and operations management

• Works in enterprise systems architecture, process optimization, and operational intelligence

From Start to Finish Continuous Improvement:

The DevOps Journey at Craftsy

Matthew BoeckmanVP - Infrastructure

Craftsy@matthewboeckman

#30 on Forbes' 2015 list of Most Promising Companies10+MM registered members, 11+MM enrolled courses350 course enrollments/hour

Wow DevOps!

Such OnCall

Very CI/CD

Many Puppet

Culture

ToolsOncall

Reward FailureCelebrate SuccessBlameless Postmortems

Personal ResponsibilityAvoid Process

Configuration Management

CI/CD

Git

Clouds

Dev shares oncall

Actionable alerts

Runbooks

Runbooks:● System overview● Escalation path● Metrics● Alert descriptions● Common failure conditions● Known recovery procedures● Incident history

Incident

Post-Mortem

Tools

Runbook

Reward

Thank you!@matthewboeckman

#DevOpsRoadTrip

MARY MOORE-SIMMONS Engineering Operations Manager

• Engineering Operations Manager

• Began career in chemical engineering before switching to the world of DevOps

• Focuses on meaningful impact on company success in the fast-paced world of software engineering

The Why and How of DevOps at SendGridMarch 2016

Why DevOps?

Scale, yo! It’s hard!

20102.1K Customers

4.8B Emails

20117.3K

Customers20.6B Emails

2012 14.3K

Customers54B Emails

201322.6K Customers

99.8B Emails

201428.5K Customers

164B Emails

Today34K Customers

>200B Emails in 2015

Now: 1B Emails/day

Champions of Quality

But I Want to be Friends!

Once Upon a Time…. It Was Terrible

Now Everything is Unicorns and Rainbows

…..Kindof

Constant Vigilance! (Analysis)

Dedicate People to the Problem

Set Goals, Get Leadership On Board

Q A&

Questions?

THANKS FOR PUTTING UP WITH SILLY JPGS

#DevOpsRoadTrip

BREAK TIME

Breakout Sessions Devs On-Call, How and Why to Get Started

Kurt Bittner, Principal Analyst - Forrester Research Doing DevOps Safely (governance & security

mindfulness) Amy Demartine, Sr. Analyst - Forrester Research

Learning from Failure Jason Hand, DevOps Evangelist, VictorOps

Measure Everything: What to & What Not to Alert On Matthew Boeckman, Craftsy

What does your “DevOps Ecosystem” look like? Brad Norris, VP, VictorOps

#DevOpsRoadTrip

#DevOpsRoadTrip

BREAK TIME

#DevOpsRoadTrip

KURT BITTNER| Principal Analyst

• Principal Analyst Serving Application Development and Delivery

• Focuses on extracting broader business value from organizational and cultural shifts

• Former CTO at Ivar Jacobson International

#DevOpsRoadTrip

AMY DEMARTINE| Senior Analyst

• Senior Analyst Serving Infrastructure & Operations Professionals

• Currently researching strategy, design, organization, and implementation of modern service delivery

• Formerly worked with BMC and HP on bringing new enterprise software products to worldwide markets

Avoid the DevOps PitfallsAmy DeMartine, Senior AnalystKurt Bittner, Principal Analyst

© 2015 Forrester Research, Inc. Reproduction Prohibited 74

Common DevOps Pitfalls To Avoid1. Going too far too fast

2. Lack of learning culture

3. Measuring people with old system metrics

4. Not being able to reduce mean time to know

5. Keeping change management the same

6. Making DevOps all about Dev and Ops

7. Letting complexity continue

Pitfall 1: Going too far too fast


Anti-Patterns› Impatience; wanting to “skip to the solution”

›Wanting different results but not really wanting to change

›Not taking time to understand barriers

›Thinking of “DevOps” as a methodology


Risks of “Big Batch” Change› A lot of energy, time and money are

invested before any result is achieved

› Forced change meets resistance

› Coordination complexity

› Progress stops when external energy is no longer being supplied

› Inevitable disappointment when progress is less than expected

› “Blame the approach”, or even “Blame the person who got us into this”

Benefits of “Small Batch” change› Quick results with small investment

› “Opt-in” change lets people choose when and how to change

› Low complexity as long as dependencies are minimized

› Scale-up when ready

› Not everyone needs to change, at least at first

› Lead by example

Big batches of work don’t work for application delivery;Why would they work for Agile transformation?


http://alivecampus.com/wp-content/uploads/2014/08/shortcut.jpg


How To Scale DevOps:ht

tp://

ww

w.n

etso

lutio

nsin

dia.

com

/blo

g/w

p-co

nten

t/upl

oads

/201

3/08

/Cor

e-A

gile

-Tea

m.jp

g

Expand Team By Team…

…


While Building A Supporting Ecosystem›Common pipeline, process, and tools

›Common platforms and services

›Communities of practice

›Servant-leadership

›A customer-obsessed culture

Pitfall 2: Lack of a learning culture

Failure not seen as opportunity to learn

Source: “Six Trends Shape DevOps Adoption, Q1 2015”, Forrester report

Ops is not getting included in life cycle


Pitfall 3: Measuring people with old system metrics


38% of I&O DevOps respondents didn’t know if customer experience was measured at their company.



Too many I&O pros aren’t measured on customer experience goals.

Pitfall 4: Not being able to reduce mean time to know


Single Source Of Truth Lacking In Many Orgs – 95% only most of the time or less

Source: April 15, 2015 “Six Trends That Will Shape DevOps Adoption”, Forrester report


…Leads To Customers Finding the Problem First



Where do we waste time, energy and resources?

MTTI MTTK MTTF MTTV

MTTRMean Time To Repair

Mean time to identify

Mean time to know

The War Room

Mean time to fix

Mean time to validate

Why a war in the war room?• Wrong tools for the tasks• Obsolete or inadequate monitoring (infrastructure monitoring rather than

performance management)

• Too many specialized tools

• Too much data, and the wrong types

• Silo mentality• Exoneration rather than cooperation

• No real process

• Wrong skills applied to the wrong level

Pitfall 5: Keeping change management the same


Changes Require Too Much Touch by 72%


Pitfall 6: Making DevOps all about Dev and Ops


1 out of every 16 open source component

download request is for a component with a known vulnerability

97% of the successfully exploited vulnerabilities in 2014 trace back to 10

common vulnerabilities and exposures, eight of which have been patched for 10 to 12

years

median number of days that threatening groups were

present on a victim's network before detection was 205 in

2014, and only 31% of victims discovered the breach

themselves

47% of web applications have unprotected file issues due to web

server misconfigurations


The Delivery-Centric Organization

IntegratedProductTeams

Communities of Practice

Servant-Leadership

LOBCMO

CIO …


Operational Platform Services

Customers

Software Platform Services Teams

Shared Service Teams

Integrated __Product Teams___

Business Offering Teams

Project Teams Sourc

ing

• One or more products• Integrated customer experience• Integrated business processes

• Single release• Shared resources• Finite scope

• Ongoing releases• Dedicated resources• Aligned with business offering

funding

• Ideally deliver via automated self-service• Resources shared across products and

projects• Ideally staffed or automated for “zero wait-

time”

• Common components and services

• Common architectural patterns

• Computing as-a-utility

The Modular Customer-Focused Application Delivery Organization


skillDev I&O QA Architects CX Product

ManagersCyber

Security

Discovering outcomes ✔︎ ✔︎

Designing capabilities ✔︎ ✔︎ ✔︎ ✔︎

Writing code/scripts ✔︎ ✔︎ ✔︎ ✔︎ ✔︎

Testing code/scripts ✔︎ ✔︎ ✔︎ ✔︎ ✔︎

Assessing usability ✔︎ ✔︎ ✔︎

Optimizing experiences ✔︎ ✔︎ ✔︎ ✔︎

Optimizing application & operational platforms ✔︎ ✔︎

Reviewing analytic data ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎ ✔︎

Securing applications ✔︎ ✔︎ ✔︎ ✔︎

Architecting solutions, patterns, and IaC ✔︎ ✔︎ ✔︎

Product Team Skills and Roles


Successful change is bottom-up, with support from leadership to clear obstacles and change culture

Patterns› Clearing obstacles, including people who get in the

way

› Measuring cycle time, customer satisfaction and throughput to determine effectiveness

› Promoting a culture of shared accomplishment across Business, Application Development, QA and Ops

› Encouraging innovation and experimentation, and tolerating mistakes so long as learning results

› Dissolving the walls between roles and functional areas

› “The customer doesn’t always know what they want, but they know what they don’t want when they see it.”

Anti-patterns› The executive cheerleader - encouraging but not

involved in the game

› Measuring utilization to determine effectiveness

› Promoting a culture of individual accomplishment and “accountability”; promoting “heroes”

› Rewarding predictability, and punishing mistakes

› Encouraging an “us-them” culture

› “The customer is always right.”

The Role of ManagementLeadership matters

Pitfall 7: Letting complexity continue


Tackling the risk of 3rd party software including open source

› Use new components

› Use components that do not have any reported CVEs

› Create component library

› Reduce number of versions of a single component

› Don’t forget middleware, OS, network, database, and performance management tools

› Use continuous delivery pipeline tools to catalog which 3rd party software is used and where it’s located

And when a vulnerability is identified, use the continuous delivery pipeline to find all affected applications, quickly generate a fix and deploy


Recommendations for DevOps Success

1. Start small and grow

2. Create a high trust environment

3. Measure based on customer experience

4. Create meaningful, common dashboards

5. Left shift control points

6. Include expertise from key groups such as QA, EA, Security

7. Standardize

Thank you

forrester.com

Amy [email protected]

@AmyDeMartine

Kurt [email protected]

@ksbittner

INSERT JASON CLOSING SLIDES

#DevOpsRoadTrip

#DevOpsRoadTrip4.21 – 4.22Devopsdaysrox.org

#DevOpsRoadTrip

devops roadtrip - denver

Technology