scaling software builds with jenkins
DESCRIPTION
This talk revolves around the scalability strategies for Jenkins, the most used Continuous Integration server. In particular, the talk focuses on how to scale Jenkins on the cloud (Amazon EC2) and create a build matrix so that the code can be built on different OSes. Furthermore, we will also look at the Cloudbees offering.TRANSCRIPT
Scaling Software Buildswith Jenkins
01
About me
Name: Andrey Adamovich
Bio: Developer, coach, speaker, author
Company: Aestas/IT
E-mail: [email protected]
Linkedin: http://www.linkedin.com/in/andreyadamovich
•••••
02
Software development process
03
Why scale?04
Feedback loop
05
Reason 106
Monolithicbuild
07
Monolithic build
Time-consuming
Resource-consuming
Frustration-accumulating
•••
08
Before youcan scale...09
SPLIT!!!10
Reason 211
Too manythings to
build12
Too many things to build
Too many components
Too many branches
Too many unit/integration tests
Too many validation rules
••••
13
Reason 314
Too manythings to test
15
Too many test variations
Different operating systems
Different target platforms
Different markets
Different customers
••••
16
Reason 417
Lack ofresources
18
Reason 519
Lack of time!20
Solutions?21
Add morehardware!
22
or...23
Go to theCloud!
24
Who is Jenkins?
25
Architecture
26
UI
27
Master/Slave
28
Labelling
29
Slaves vs.Executors
30
Executor contention
31
Other types of contention
CPU contention
I/O contention
••
32
Cloud slaves
33
Executor scaling
34
Amazon Web Services
35
Elastic Compute Cloud
instances
storage
images
•••
36
EC2 Console
37
ALERT!!!38
It's not free!39
But...40
There is aFree Tier!
41
Cost calculations
Name Compute Units Memory Hourly Cost
t1.micro 2 0.60 GB $0.02
m1.small 1 1.70 GB $0.06
c1.medium 5 1.70 GB $0.14
m1.medium 2 3.75 GB $0.12
m1.large 4 7.50 GB $0.24
c1.xlarge 20 7.00 GB $0.58
m3.xlarge 13 15.00 GB $0.5042
Labor cost
43
Public access44
Public Clouds
VMWare vCloud
Ubuntu Enterprise Cloud
Microsoft Azur
Google Cloud
Digital Ocean
Cloudbees Cloud
••••••
45
Private Clouds
OpenStack
VMWare vSphere
Amazon VPC
•••
46
Jenkins plugins
47
Jenkins EC2 plugin
Allow Jenkins to start slaves on EC2 or Ubuntu Enterprise Cloud
(Eucalyptus) on demand, and kill them as they get unused. With this
plugin, if Jenkins notices that your build cluster is overloaded, it'll start
instances using the EC2 API and automatically connect them as
Jenkins slaves.
“
48
Preparation49
1. Configure access/secret key
50
2. Configure key pair
51
3. Configure security group
52
4. Select machine image
Amazon base images
AWS market place
Create your own
•••
53
5.1. Configure EC2 plugin
54
5.2. Configure EC2 plugin
55
5.3. Configure EC2 plugin
56
Demo57
Cloudbees
Company behind Jenkins
Kohsuke Kawaguchi (author of Hudson/Jenkins) works there
Jenkins Enterprise Edition
Jenkins Cloud = DEV@Cloud
••••
58
DEV@Cloud
API & SDK
Pricing based on size (# masters/executors/slaves)
Slave OS variety
iOS coming soon
Windows Phones/Tablets coming soon
Manual or auto-scaling
Minute-based pricing
•••
••
••
59
Free-tier
300 free build minutes a month
1 build executor
FOSS programme for open-source projects
•••
60
Services Partner
61
Questions?62
GitHub projects
Jenkins setup:
https://github.com/aestasit/talks2013-devconfu-scaling-builds-setup
Slides:
https://github.com/aestasit/talks2013-devconfu-scaling-builds-slides
63
Scaling Software Builds with Jenkins
Author: Andrey Adamovich
Company: Aestas/IT
E-mail: [email protected]
Linkedin: http://www.linkedin.com/in/andreyadamovich
••••
64
Feedback
65