the immutable journey

The Immutable Journey

Tom Tsai @ Startup

hello!

I am Tom TsaiI am here because I want to give life to the servers

� Startup (DevOps)� Trend Micro (QA, DevOps)� DevOps Lecturer

“Why Come To This Section ?

Outline

� What is Immutable Infrastructure?

� Configuration Management == Die

� Container Orchestration - Kubernetes

� Image CD Pipeline Build Up

� Q & A

1.

Immutable Infrastructure

Build

A stack that you build once (be it a virtual machine image, container image, or something else), run one

or many instances of, and never change again. Ref

V0.0.1

Deploy

The deployment model is to terminate the instance/container and start over from a new one Ref

V0.0.2 V0.0.1

What is the Benefit

� Simplifying operations

� Continuous deployments, fewer failures

� Reduces errors and threats

Ref

Yes ! Docker is Immutable Infrastructure

Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries

Golden Image

Is there a image to execute all kind of application ?

Ruby Python

C#

PHPJavascript

Perl

RJava

Groovy

Clojure

GoF# Pig

Image Layer

How to prepare the image for all kind of application in alpha, beta, production phase using?

OS

Environment

Application

Change Seldomly

Change Frequently

Dockerfile

Is Dockerfile the only solution?

2.

Configuration Management

“Which Configuration

Management Do You Use?

If the server is Immutable,

Can We Throw away CM?

DockerfileEasy to use, hard to maintain, extend

configuration management should Be Die !?

Configuration ManagementSteep Learning Curve, easy to maintain, extend

Using Cm to provision Image

Packer is a tool for creating machine and container images for multiple platforms from a single source configuration.

Packer Template

Builders: Base image

Provisioners: Select

prepare tool (e.g. Chef)

Post-Processors:

store image (e.g.

dockerhub, aws ecr)

Image Layer

Separate Image to two part, OS+Environment & Application

OS

Environment

Application

Change Seldomly

Change Frequently

Chef X Docker X Packer

1. Push tag to

trigger webhook

2. Execute Packer

to provision image

3. Get Provision

script from Chef

4. Push Image to

dockerhub, image

tag equal to git

tag

Ref

“Different phase Images ?

V0.0.1 alpha V0.0.1 beta V0.0.1 prod

All PhaSe Image

Get $PHASE from environment variable. Get different phase configuration in one image.

app/start.sh /binary /conf/alpha.yml /beta.yml /prod.yml

Build Structure

ln -s /app/conf/$PHASE.yml /app/conf/current.yml

binary -conf /app/conf/current.yml

start.sh

3.

Kubernetes

Without K8S ...

With K8S ...

K8S Infra

K8S Terminology

PodA group of one or more containers

Replica SetEnsures that a specified number of pod “replicas” are running

DeploymentProvides declarative updates for Pods and Replica Sets

Servicea logical set of Pods and a policy by which to access them

K8S Deployment

� Create a Deployment to bring up a Replica Set and Pods.

� Update Deployment (e.g. rolling update Pods)

� Rollback to an earlier Deployment revision

� Pause and resume a Deployment.

COntainer Deployment

K8S API

1. Create Deployment

2. Update Image ver.

3. Create Service

4. Of course, Testing

Setup K8S is so

Hard

Use it in google cloud engine will be easy

But I still Try in AWS By

� AWS CloudFormation

� Chef

Don’t forgot to add tag KubernetesCluster in all aws resource

AWS

4.

Image CD Pipeline

“Have you ever heard Jenkins?

How dare

you ...

Not to mention

DeployBuild Test

CM X Docker X Packer Kubernetes Depends ...

Place your screenshot here

Jenkins 2.0 Pipeline is Good

� Leverage Groovy DSL to achieve pipeline as code

� Loading Script Text from Version Control

� Retaining Global Libraries

Everything is done, take a break ?!

More and More Job

Using Jenkins Pipeline Plugin after one month, I feel there is still room for improvement...

insufficient

Pipeline Job: Still need to

maintain Manually

Develop: Inside Jenkins

website

Groovy: Cannot include

third-party library

Jenkins Job DSL

Trigger Seed Job just like compile your General Jenkins job

Seed Job

Job A

DSL

Job B

DSL

Job A Pipeline

Job B Pipeline

After Using ...

Pipeline Job: Generate

automatically

Develop: everywhere

Groovy: embed other

language

demo

Jenkins Deployable Now !

/CM Script/ # do anything you want

/dsl-repo/dsl/ # job dsl for generate jenkins job /pipeline/ # pipeline script for job dsl including

/jobs/ # Jenkins Job generating by job dsl

/workflow-libs/ #Pipeline Global Library

Jenkins Build Structure (/var/lib/jenkins/...)

Script Security

Script security plugin block groovy script, after enabling global security… (workaround)

Test Your Jenkins Job?

thanks!

Any questions?

smalltown20110306

smalltown0110

smalltown0110