realworld cd pipelines - qcon london 2020 · 2016-04-05 · realworld cd pipelines it’s not...

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d©

20

16

Clo

ud

Bee

s, In

c. A

ll R

igh

ts R

eser

ved

Realworld CD PipelinesIt’s not always a linear stream

Nigel Harniman

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About me

2

Nigel Harniman

@harniman

Build Engineer, Devops and Architect

DevOps, Infra as Code,Continuous Delivery

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About CloudBees

WE ARE

The Enterprise

Jenkins Company

OUR GOAL

To be the

automation

leader helping

organizations

develop and

deliver better

software faster.

WE OFFER

Continuous

delivery solutions

powered by

Jenkins at

enterprise scale

with enterprise

support.

OUR

BUSINESS VALUE

We enable IT to

deliver better

software faster by

optimizing their

use of Jenkins on

premise or in the

cloud.

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

4

Jenkins? CI? CD? Code Jenkins plugin?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

5

Developer?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

6

Operations?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

7

Tester?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

8

Build Engineer?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

9

Project Manager?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

10

Doing CI?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

11

Automated Deployment?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

12

Wants to automatedeployment?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

13

Who has an E2E Pipeline?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

14

Single component?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

15

Complex suite of components?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

16

Whose is perfect?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

About you

17

Who uses Jenkins?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

18

“Software is eating the world.”

“Every business is in the

software business.”

“It’s an

application

economy.”

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Software is Everywhere

19Photo courtesy of Bill Abbott via Flickr

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

20Photo courtesy of Naddsy via Flickr

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

21

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

This wasn’t a one off….

2222

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

23

How Do You Deliver Better Software Faster?

ProdDev

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

We need

24

a process that ensures quality

executes reliably and quickly

delivers value early

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

We need

25

to automate

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

We need

26

an automated pipeline

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Typical Linear Pipeline

27

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

28

Jenkins Pipeline Job

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Pipeline Job Type

29

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

30

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Showing progress and history

31

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

32

We can optimise this

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Optimised Pipeline

33

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

34

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Pipeline Stage View

35

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

36

We’re done ?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

37

commit change

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

38

trigger the process

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

39

verify quality

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

40

and release!!

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

41

But wait….

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

42

single component

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

43

or microservice

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Only true if

44

tests are thorough

independently releasable

no breaking changes for consumers

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

45

a system is comprised ofmultiple apps /components

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

46

have dependencies

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

47

new feature

provided by new/updated interface

require min version

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

48

system level concerns:

performance test

functional test

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

49

embedded systems : firmware image

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

50

???

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

51

assembly build

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

52

release pipeline

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Release Pipeline

53

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

54

Release Pipeline

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

55

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

56

another manual step?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

57

which versions?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

58

wrong build = regression

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

59

automated trigger from upstream?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

60

use build step

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

61

build job: 'release-build', parameters: [

[$class: 'StringParameterValue', name:

'component1', value: comp1ver],


'component2', value: comp2ver],


'component3', value: comp3ver]

], wait: false

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

62

need build numbers for each component

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

63

null = lastSuccessfulBuild?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

64

inspect deployed system?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

65

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Manifest

66

n. cargo-list for use of Customs officers; list of passengers in aircraft or of trucks etc. in goods train

Oxford English Dictionary

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

67

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

68

what’s in the manifest?list of components and versions

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

69

#Tue Mar 01 23:11:41 GMT 2016

mobile-deposit-api=1.0.0-21

mobile-deposit-ui=1.0.0-4

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

70

read/write manifest

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

71

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

72

where to store the manifest?

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

73

jenkins build workspacescm

Jenkins build artifactexternal file systemartifact repository

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

76

cost of environmentsparallel or single build?

sequencing

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

78

number of upstreamsupstream build freq

release-build trigger raterelease-build duration

wait time

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

81

jenkins queues buildspipeline stage concurrency

concurrency rule skips builds

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

82

works with scmusing Jenkins artifacts needs

additional step

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

92

re-deploy allcompare with runningcompare with manifest

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

94

re-deploy allcompare with runningcompare with manifest

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

99

pipelines ≠ linearassembly:multi-component

build fan-intrigger downstreamtrack with manifests

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

How to get started

• Install pipeline plugin

– https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin

• Read the docs

– https://documentation.cloudbees.com/docs/cookbook/_conti

nuous_delivery_with_jenkins_pipeline.html

• Dzone ref card

– https://dzone.com/refcardz/continuous-delivery-with-jenkins-

workflow

• Download the mobile-api projects (qcon branches)

– https://github.com/harniman/mobile-deposit-api/tree/qcon

– https://github.com/harniman/mobile-deposit-api/tree/qcon

– https://github.com/harniman/mobile-deposit-

release/tree/qcon

102

https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin

https://documentation.cloudbees.com/docs/cookbook/_continuous_delivery_with_jenkins_pipeline.html

https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow

https://github.com/harniman/mobile-deposit-api/tree/qcon

https://github.com/harniman/mobile-deposit-api/tree/qcon

https://github.com/harniman/mobile-deposit-release/tree/qcon

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Get Certified!

103

Be recognized for your Jenkins skills

Differentiate yourself from your peers

Increase your value to your employer

Hiring Managers can more easily identify qualified candidates

Learn more, register and get the Study Guides:https://www.cloudbees.com/jenkins-certification

https://www.cloudbees.com/jenkins-certification

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d©

201

5 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d

Thank you

and questions!!

www.cloudbees.com

[email protected]

@harniman

http://www.cloudbees.com

© 2

01

6 C

lou

dB

ees,

Inc.

All

Rig

hts

Res

erve

d©

20

15

Clo

ud

Bee

s, In

c. A

ll R

igh

ts R

eser

ved

realworld cd pipelines - qcon london 2020 · 2016-04-05 · realworld cd pipelines it’s not...

Documents