go faster - remove inhibitors to rapid innovation

Go Faster!Tech, Process and

Organization

Fred [email protected]

@fgeorge52Copyright © 2015 by Fred George. All rights reserved. 1

3

Fred George

Copyright © 2015 by Fred George. All rights reserved.

Consulting Roles• Change Agent• Disruptor• “Hand grenade I am

throwing into development”• CTO describing Fred to his

Vice President

4

Industry “Hot” Topics


MicroServices

Cloud

Docker

Dev/Ops Full-stack developer Agile

Lean startupCassandra

Event busMVP

(minimum viable

product

Programmer anarchy

Go Faster

No-SQL

One Hacker Way

5

Why: New Problem Domains: The Cynefin Framework


Simple

ComplicatedComplex

ChaoticCause a Effect

Cause aaa Effect

Effect? Cause?

Cause ? Effect

Disorder

6

Why: Competition is coming


• Enablement through technologies• Cloud computing• Ubiquitous, high bandwidth• Languages (and supporting frameworks)

• Recognition of business opportunities• Silicon Valley innovators as role models• Accelerating business needs• Few inhibitors for global competitors• Reduction of entry barriers for niche competitors• Micro-marketing to specific customer niches

7

How Fast Can You Go?


8

Iteration Length in Agile Age


1 mo.

20152000 2005 2010

3 wks.

2 wks.

1 wk.

1 day

XPXP

XP

XP

SCRUM**2-3 months

9

Project Delivery Cycles


5 yrs.

1980 1990 2000

1 mon.1

week

1 day

2010

1 year

Proj

ect D

urat

ion

(log) Waterfall

Waterfall

AgileAgile

A2

A2

Waterfallw OO

10

How Fast Can You Go?


Deployment to Production Every 3.5 minutes

11

Inhibitors


12

•Cloud exploitation•Specialized databases•New programming languages•Open source frameworks (users and creators)•Continuous releases

“Valley Tech”


13

Hardware Lead Times


6 mo.

1990 1995 2005

30 min.

1 week1 day

2015

1 mo.

2000 2010

5 min.

5 sec.Orde

r lea

d tim

es (l

og) Data

Center

VirtualMachines

Docker

Commercial

CloudCapacity planning?Dedicated ops team?

14Copyright © 2015 by Fred George. All rights reserved.

10

1

1000

100

100,0001 10 100 1000 10,000

Size (loc)

Num

ber o

f Se

rvice

s

MicroServices

Rails / Java Zone

SOA

Workshop

Forward

Netflix

Services Impact

0MQ0MQ 0MQ

0MQ 0MQ 0MQ

Copyright © 2015 by Fred George. All rights reserved. 15

Kafka

Service

Emerging: Event Bus

New Architecture Patterns

Copyright © 2015 by Fred George. All rights reserved. 16

Bus

ServiceService

Service

NeedNeed

NeedSol’n’s Sol’n

•Variants easy•Graceful degradation

ChooseSol’n

17

Incremental Applications


Event Bus

Rental offers

Car rent

Legacy

server

Brand offersLocation

offers

Segmentation

Membership

CarRent.com Login

Sally

A B

18

Databases:Holy Grail or Ball-and-Chain?

•Fortune 100 view:• Entity-oriented; consistency essential• As few as possible

•MicroService view:• Shift to Event Bus, plus• DB per MicroService (if persistence needed)• Poly-glot (various NoSQL, SQL) + event buses• Few (10%) writable; even fewer transactional



Rental offers

Car rent

Legacy

server


offers

Segmentation

Membership

CarRent.com Login

Sally

A B Rental offers

Event publishing

Solution collecting

300msresponder

Redis


Rental offers

Car rent

Legacy

server


offers

Segmentation

Membership

CarRent.com Login

Sally

A B

Membership

enrichmentETL Key-

value

21

Open Source: Netflix


22

Open Source: Docker


23

Disruptive Technology: Functional Languages


•Middleware exploiting:• Kafka written in Scala• RabbitMQ written in Erlang

•Case study: MailOnline page rendering• Legacy system: 130K loc Java/JSP’s• Replacement: 4,000 loc Clojure

24

Mitigate Process Inhibitors


25

Understand Your Problem


Complex

Simple

Complicated

ChaoticCause a Effect

Cause aaa Effect

Effect? Cause?

Cause ? Effect

Disorder


Requirements

Trying ideas

“Experimentation drives Innovation”

New Vision of “What” to Build


Task

Story

Feature

ProjectInitiative

Co-Credit: Greg Reiser, ThoughtWorks

Re-think Interactions

28

• Bad metrics:

• Lines of code

• Development stages and milestones met

•Business success metrics:

• Sales, clicks, registrations, customer retention,

etc.

Measure What Matters


29

Mitigate Organization Inhibitors


30

Over-Specialization


•Theory: • Specialist are more productive

•Practice: •Overhead of communication is under-estimated•Unbalanced workload creates delays

31

Specialization Institutionalized with Titles


Case Study:•50 IT professionals•25+ titles•0 people understanding projects

32

Solution: Fix the Titles


Step 1: Define competence in key technologies

Master

Journeyman

Apprentice

Strategic Technologies• Ruby• Java• iOS• Android• DB (SQL and non-SQL)• Testing• OO Design• Node.js• HTML/CSS/JavaScript• …

33

New Titles for New Projects


Graduate Developer

Not competent in a key technology

Developer Competent in a key technology

Senior Developer

Expert in a key technology

System Developer

Competent in 5-7 key technologies

Master Developer

Expert in 3-5 key technologies

35

Non-dedicated Leadership


36

Bring Work to the Team…


Not musical chairs for each small project


MicroServices

Cloud

Docker


Lean startup Cassandra

Event busMVP

(minimum viable

product

Programmer anarchy

Go Faster

No-SQL

Managerless process

39

Project Delivery Cycles


NRK

NRK

5 yrs.

1980 1990 2000

1 mon.1

week

1 day

2010

1 year

Proj

ect D

urat

ion

(log) Waterfall

Waterfall

AgileAgile

Waterfallw OO

40

Acknowledgements


•Neil Hutchinson – Founder of Forward• Adopting a “Fail Fast” culture

•Carl Gaywood• Former programmer who wanted to Go Faster

•Paul Ingles and Mike Jones• Implementers of “Experimentation drives innovation”

41

More Information…•Google:•MicroService Architecture for videos•MicroXchg 2015, 2016 conferences•MicroService track in most conferences• Programmer Anarchy for managerless teams•Docker for latest container deployment


go faster - remove inhibitors to rapid innovation

Software