The four pillars of DevOps at Hiscox

Jonathan Fletcher

Enterprise Architect, Technology and Platform Lead

Me

• Jonathan Fletcher

• Architect in Hiscox Group IT since 2012

• Ex Dev

• Ex Ops

• http://enterprisedevops.blogspot.com

• http://www.devops.com

• @FletcherJofanon

2

Why DevOps ?

• From our IT Strategy – – “Be nimble in responding to market opportunities”

– “Flexible technology at the heart of the business”

Common complaints

• “Throw it over the wall” behaviour - it’s not my problem

• Lack of holistic understanding of the software delivery lifecycle

• Slow pace of change

• Expensive cost of change

• Late discovery of issues in the project lifecycle

• Unaligned goals and incentives – pulling in different directions

4

DevOps friction

More process re

view

s M

ore change control review

s M

ore deploym

ent freezes M

ore standards

control boards

Mor

e fr

eque

nt c

hang

esLo

wer

tol

eran

ce f

or

outa

geM

ore

com

ple

x ap

plic

atio

nsM

ore

com

ple

x de

plo

ymen

ts

Do more! Do less!

RFC’sCABDeployment guideRollback guideDaily status callsStaff availabilityIssue trackingEnvironment bookingEscalation processesEmergency processesSmall change processesetc etcMr. Dev Mr. Ops

IT today

6

I’m a business analyst

I’m a DBA

I’m a developer

I work in support

I’m an infrastructure

engineer

I’m a business stakeholder

I’m a release manager

I’m a security consultant

Is DevOps the right name?

• Why do we think the issue of working well together and aligning goals is limited to Development and Operations?

• Shouldn’t everyone involved in the change process should work together to accomplish shared goals?

• DevTestBizThingyOps should be the real name © J.Fletcher

DevOps – so what is it?

8

• “Bad behaviour arises when you abstract people away from the consequences of their actions” – Jez Humble

• DevOps is a culture of empathy, shared goals and incentives

Disclaimer

!9

The four pillars

10

Cu

ltu

re

Pro

cess

Peo

ple

Tech

no

log

y

Culture – the things people do when there is no one around to tell them what to do

Process – you need to have processes that support different silo's working together to achieve a fast pace of change

People - if you don't have the right people then it doesn't matter how great your technology is

Technology - what are the building blocks you need to be a mature DevOps enterprise?

11

People

People• Restlessnes (relentlessly looking to improve themselves, others,

processes etc)

• Good technical ability across a broad skill set (understand as much of other people's jobs as possible)

• Everyone can code and use version control

• Everyone understands the test triangle

• Organised in small product focused teams rather than technology silo's (align teams to the business not the technology)

• Common incentive schemes

• Favour automation and repeatability above anything else

• CIO/CTO are DevOps biggest guardians & SME's and seek to destroy anything that affects that culture

• Natural face-to-face influencers rather than endless emailers

• Natural sharers of information

• Take an interest in their specialism outside of work (i.e. go to conferences and take part in the wider community)

12

BermudaUS Europe London MarketsUK

Hiscox yesterday (ish!)IT

cap

abili

ty

Groupdevelopment

Group supportGroup

infrastructureGroup testing Group DBA

Group release and

deployment

Group architecture

Hiscox tomorrow (ish!)

Europe

Dev

Support

Testing

DBA

Release and deployment

Architecture

UK

Dev

Support

Testing

DBA

Release and deployment

Architecture

London market

Dev

Support

Testing

DBA

Release and deployment

Architecture

USA

Dev

Support

Testing

DBA

Release and deployment

Architecture

Bermuda

Dev

Support

Testing

DBA

Release and deployment

Architecture

Hiscox Model

• Federated

• Cross skilled teams

• Cradle to grave responsibilities

• Shared goals and incentives

• Underpinned by the Platform Services Group

16

Culture

Culture• Measure everything, always

• Individuals have empathy for the rest of the team (i.e. they don't pass the buck)

• Shared goals and incentives

• Don't reward the fire fighter, reward the fire preventer

• Reward innovation and challenging the status quo

• Don't punish people when they try something new but fail

• There is no IT and "business". IT as much "the business" as the sales people.

• Seeking to break down silo's

• "It's not my job" doesn't exist

• "Its my server/code/network/database" doesn't exist

• Individuals are empowered to make decisions

• Top-up management rather than top-down

17

This says everything

18

Shared goals, incentives, empathy & transparency

19

20

Process

Process• Agile

• Continuous Integration

• Continuous Delivery

• Lean

• Fail-early, fail often

• Release management team are facilitators of change not guardians of change (i.e. they try and aid change rather than stopping/slowing it)

• All change (I mean all) goes through the pipeline from left to right (dev, test, acceptance, production)

• Knowledge sharing and "just enough" documentation is part of the process

• Measuring success and failure is part of the process

• Retrospectives are part of the process

21

22

Technology

Technology

• TDD/BDD everything (including Puppet etc)

• Everything is in version control (code, automated tests, server config etc)

• Release automation tooling

• Convergent desired state tooling

• Public Cloud

• The same trending, monitoring & alerting solutions available through nonprod & prod

• Application Performance Management

• Service Virtualisation

• Continuous Integration

• Continuous Unit tests

• Continuous Service level tests

• Continuous GUI tests

• Performance testing

23

Platform Services

• Growth of the business is challenging IT to find new and better ways to do things

– Means working smarter not harder. Doesn’t mean an ever increasing head count

• Platform Services helps break down silo’s between teams by providing a change platform that is re-usable between multiple teams

• Help others use the platform (they don’t implement themselves!)

Core platform capabilities

• Source code management

• Artefact management

• Automated application deployment

• Automated server configuration

• Load performance test

• Automated functional test

• Continuous Integration and automated code build

• Application Performance Management

• Agile planning

• Defect management

• More...

Be careful...

You don’t solve a silo issue by creating another silo! BAD

Having a team that evangelises DevOps ideas, concepts and tooling is GOOD

27

IS IT WORTH IT?

28

Just some of the benefits…

• 150 deployments in the last 3 days in one application alone

• The week before go-live on our biggest ever change program we reduced 17.5 man days of effort to about 10 minutes

• Help enable changing a 10 week change cycle down to 2 weeks

• We went from 1 person knowing how to do to do a release to thousands (kind of!)

DevOps has visibility at the highest levels

29

Project Sponsor

MyBosses boss

DevOps – how – top 5 hints?

30

1. Employ the right people in the right team structure

2. Empower the team – let them make the right decisions

3. There are processes and tools that help align working practises to achieve empathy and shared goals (such as increasing the pace of change)

4. Commonly large amounts of automation is prevalent in a DevOps environment to create metrics, reduce manual wasted effort and increase the pace of change

5. Keep those 4 pillars in mind. DevOps isn’t just a technical challenge

Thanks and any questions?

31