scaling software builds with jenkins

Scaling Software Buildswith Jenkins

01

About me

Name: Andrey Adamovich

Bio: Developer, coach, speaker, author

Company: Aestas/IT

E-mail: andrey@aestasit.com

Linkedin: http://www.linkedin.com/in/andreyadamovich

•••••

02

Software development process

03

Why scale?04

Feedback loop

05

Reason 106

Monolithicbuild

07

Monolithic build

Time-consuming

Resource-consuming

Frustration-accumulating

•••

08

Before youcan scale...09

SPLIT!!!10

Reason 211

Too manythings to

build12

Too many things to build

Too many components

Too many branches

Too many unit/integration tests

Too many validation rules

••••

13

Reason 314

Too manythings to test

15

Too many test variations

Different operating systems

Different target platforms

Different markets

Different customers

••••

16

Reason 417

Lack ofresources

18

Reason 519

Lack of time!20

Solutions?21

Add morehardware!

22

or...23

Go to theCloud!

24

Who is Jenkins?

25

Architecture

26

UI

27

Master/Slave

28

Labelling

29

Slaves vs.Executors

30

Executor contention

31

Other types of contention

CPU contention

I/O contention

••

32

Cloud slaves

33

Executor scaling

34

Amazon Web Services

35

Elastic Compute Cloud

instances

storage

images

•••

36

EC2 Console

37

ALERT!!!38

It's not free!39

But...40

There is aFree Tier!

41

Cost calculations

Name Compute Units Memory Hourly Cost

t1.micro 2 0.60 GB $0.02

m1.small 1 1.70 GB $0.06

c1.medium 5 1.70 GB $0.14

m1.medium 2 3.75 GB $0.12

m1.large 4 7.50 GB $0.24

c1.xlarge 20 7.00 GB $0.58

m3.xlarge 13 15.00 GB $0.5042

Labor cost

43

Public access44

Public Clouds

VMWare vCloud

Ubuntu Enterprise Cloud

Microsoft Azur

Google Cloud

Digital Ocean

Cloudbees Cloud

••••••

45

Private Clouds

OpenStack

VMWare vSphere

Amazon VPC

•••

46

Jenkins plugins

47

Jenkins EC2 plugin

Allow Jenkins to start slaves on EC2 or Ubuntu Enterprise Cloud

(Eucalyptus) on demand, and kill them as they get unused. With this

plugin, if Jenkins notices that your build cluster is overloaded, it'll start

instances using the EC2 API and automatically connect them as

Jenkins slaves.

“

48

Preparation49

1. Configure access/secret key

50

2. Configure key pair

51

3. Configure security group

52

4. Select machine image

Amazon base images

AWS market place

Create your own

•••

53

5.1. Configure EC2 plugin

54

5.2. Configure EC2 plugin

55

5.3. Configure EC2 plugin

56

Demo57

Cloudbees

Company behind Jenkins

Kohsuke Kawaguchi (author of Hudson/Jenkins) works there

Jenkins Enterprise Edition

Jenkins Cloud = DEV@Cloud

••••

58

DEV@Cloud

API & SDK

Pricing based on size (# masters/executors/slaves)

Slave OS variety

iOS coming soon

Windows Phones/Tablets coming soon

Manual or auto-scaling

Minute-based pricing

•••

••

••

59

Free-tier

300 free build minutes a month

1 build executor

FOSS programme for open-source projects

•••

60

Services Partner

61

Questions?62

GitHub projects

Jenkins setup:

https://github.com/aestasit/talks2013-devconfu-scaling-builds-setup

Slides:

https://github.com/aestasit/talks2013-devconfu-scaling-builds-slides

63

Scaling Software Builds with Jenkins

Author: Andrey Adamovich

Company: Aestas/IT

E-mail: andrey@aestasit.com

Linkedin: http://www.linkedin.com/in/andreyadamovich

••••

64

Feedback

65