workshop: from zero to cluster in the cloud with terraform
TRANSCRIPT
01
02
DevFest03
DevTernity04
Let's start!05
What is Terraform?Terraform is a tool for building, changing, and versioning
infrastructure safely and efficiently. Terraform can manage
existing and popular service providers as well as custom
in-house solutions.“07
TerraformMulti-cloud provisioning tool (AWS, Google Cloud, Azure)
Management API aggregator (Kubernetes, Chef, DNSimple)
Declarative language (HCL, JSON)
•••
08
Our goal
09
Go! Go! Go!10
InstallTerraform
11
Terraform project*.tf - infrastructre definitions
*.tfvars - external variables
*.tfstate[.backup] - project state
.terraform/ - downloaded provider plugins
••••
12
Configure providerCreate your first Terraform file - 10_providers.tf :
provider "aws" {
version = "1.0"
}
provider "template" {
version = "1.0"
}
01.
02.
03.
04.
05.
06.
13
Init provider(s)Download provider plugins by executing:
terraform init01.
14
Planning/applying changesRemember, this command does not create anything:
terraform plan
But this does:
terraform apply
01.
01.
15
Query dataCreate the 15_data.tf file:
data "aws_ami" "workshop_ubuntu_trusty" {
most_recent = true
filter {
name = "name"
values = [ "devops-ubuntu-14-04-x64*" ]
}
owners = [ "self" ]
}
01.
02.
03.
04.
05.
06.
07.
08. 16
Query datadata "aws_subnet" "workshop_subnet_primary" {
cidr_block = "10.1.1.0/24"
}
data "aws_subnet" "workshop_subnet_secondary" {
cidr_block = "10.1.2.0/24"
}
01.
02.
03.
04.
05.
06.
17
Query datadata "aws_security_group" "workshop_security_group" {
name = "workshop_security"
}
01.
02.
03.
18
terraformapply
19
OutputsCreate the 99_outputs.tf file:
output "ami_id" {
value =
"${data.aws_ami.workshop_ubuntu_trusty.image_id}"
}
01.
02.
03.
04.
20
Outputsoutput "subnet_id" {
value =
"${data.aws_subnet.workshop_subnet_primary.id}"
}
01.
02.
03.
04.
21
Outputsoutput "security_id" {
value =
"${data.aws_security_group.workshop_security_group.id}"
}
01.
02.
03.
04.
22
terraformrefresh
23
terraformoutput
24
terraformoutput ami_id
25
Create SSH keyssh-keygen -t rsa -f student.key01.
26
VariablesCreate the 00_variables.tf file:
variable "project_name" {
default = "my-cluster"
}
01.
02.
03.
27
Create key pairCreate the 11_key_pair.tf file:
resource "aws_key_pair" "student_key" {
key_name = "${var.project_name}-student-key"
public_key = "${file("student.pub")}"
}
01.
02.
03.
04.
28
Alternative: use randomprovider "random" {
version = "1.0"
}
01.
02.
03.
29
terraform init30
Alternative: use randomresource "random_id" "project_id" {
byte_length = 8
}
resource "random_pet" "project_name" {
length = 2
}
01.
02.
03.
04.
05.
06.
31
Alternative: use random${random_id.project_id.b64_std}
${random_id.project_id.hex}
${random_id.project_id.dec}
${random_id.project_name.id}
01.
02.
03.
04.
32
Create serverCreate the 25_servers.tf file:
resource "aws_instance" "cluster_node" {
ami = "${data.aws_ami.workshop_ubuntu_trusty.id}"
instance_type = "t2.small"
tags {
Name = "${var.project_name}-node"
}
...
01.
02.
03.
04.
05.
06.
07.
33
Create server...
key_name = "${aws_key_pair.student_key.key_name}"
subnet_id = "${data.aws_subnet.workshop_subnet_primary.id}"
vpc_security_group_ids =
[ "${data.aws_security_group.workshop_security_group.id}" ]
}
01.
02.
03.
04.
05.
06.
34
Provision serverAdd the following snippet within aws_instance resource:
connection {
user = "ubuntu"
private_key = "${file("student.key")}"
}
01.
02.
03.
04.
35
Provision serverprovisioner "remote-exec" {
inline = [
"wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch
"sudo apt-get update",
"sudo apt-get install apt-transport-https",
"echo 'deb https://artifacts.elastic.co/packages/5.x/apt stabl
...
01.
02.
03.
04.
05.
06.
07.
36
Provision server ...
"sudo apt-get update && sudo apt-get install elasticsearch",
"sudo /usr/share/elasticsearch/bin/elasticsearch-plugin instal
"sudo service elasticsearch start"
]
}
01.
02.
03.
04.
05.
06.
37
terraformapply?
38
Didn't work?39
Warning!
40
terraformtaint
41
terraformapply
42
Add configurationprovider "template" {
version = "1.0"
}
01.
02.
03.
43
terraform init44
Add configurationdata "template_file" "es_config" {
template = "${file("elasticsearch.yml.tpl")}"
vars {
cluster_name = "${var.project_name}"
}
}
01.
02.
03.
04.
05.
06.
45
Add configurationdata "template_file" "jvm_opts" {
template = "${file("jvm.options.tpl")}"
}
01.
02.
03.
46
Add configurationprovisioner "file" {
content = "${data.template_file.es_config.rendered}"
destination = "/tmp/elasticsearch.yml"
}
provisioner "file" {
content = "${data.template_file.jvm_opts.rendered}"
destination = "/tmp/jvm.options"
}
01.
02.
03.
04.
05.
06.
07.
08.
47
Provision serverprovisioner "remote-exec" {
inline = [
...,
"sudo cp /tmp/elasticsearch.yml /etc/elasticsearch/elasticsear
"sudo cp /tmp/jvm.options /etc/elasticsearch/jvm.options",
"sudo service elasticsearch start"
]
}
01.
02.
03.
04.
05.
06.
07.
08.
48
terraformtaint
49
terraformapply
50
Use countvariable "node_count" {
default = "2"
}
01.
02.
03.
51
Use countresource "aws_instance" "cluster_node" {
count = "${var.node_count}"
tags {
Name =
"${var.project_name}-node-${format("%02d", count.index + 1)}
}
...
01.
02.
03.
04.
05.
06.
07.
52
terraformapply
53
terraformshow
54
terraformconsole
55
Add outputsoutput "cluster_ips" {
value = "${aws_instance.cluster_node.*.public_ip}"
}
01.
02.
03.
56
terraformrefresh
57
Add load balancerresource "aws_alb" "cluster_lb" {
name = "${var.project_name}-lb"
internal = false
security_groups = [
"${data.aws_security_group.workshop_security_group.id}"
]
subnets = [
"${data.aws_subnet.workshop_subnet_primary.id}",
"${data.aws_subnet.workshop_subnet_secondary.id}"
]
}
01.
02.
03.
04.
05.
06.
07.
08.
09.
10.
11.
58
Add load balancerresource "aws_alb_target_group" "cluster_target_group" {
name = "elasticsearch"
port = 9200
protocol = "HTTP"
vpc_id = "${data.aws_subnet.workshop_subnet_primary.vpc_id}"
}
01.
02.
03.
04.
05.
06.
59
Add load balancerresource "aws_alb_target_group_attachment" "cluster_target" {
count = "${var.node_count}"
target_group_arn =
"${aws_alb_target_group.cluster_target_group.arn}"
target_id =
"${aws_instance.cluster_node.*.id[count.index]}"
port = 9200
}
01.
02.
03.
04.
05.
06.
07.
08.
60
Add load balancerresource "aws_alb_listener" "cluster_front_end" {
load_balancer_arn = "${aws_alb.cluster_lb.arn}"
port = "9200"
protocol = "HTTP"
default_action {
target_group_arn =
"${aws_alb_target_group.cluster_target_group.arn}"
type = "forward"
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09.
10.61
Configure Kibanaresource "aws_instance" "kibana" {
ami = "${data.aws_ami.workshop_ubuntu_trusty.id}"
instance_type = "t2.small"
...
01.
02.
03.
04.
62
Copy cluster configurationresource "null_resource" "copy_cluster_config" {
count = "${var.node_count}"
provisioner "local-exec" {
command = "scp ..."
}
depends_on = [ "aws_instance.cluster_node" ]
}
01.
02.
03.
04.
05.
06.
07.
63
Final task!64
terraformdestory
65
Infrastructure as Code
67
Kief Morris
68
DevTernity69
That's all!70
Thank you!71
72