WordPress MeetupKarlsruhe

AutoScaling WordPress with Docker & AWS

22. September 2016 Jan Löffler, CTO Plesk

2011 2012 2013 2014 2015 20160%

5%

10%

15%

20%

25%

30%

WordPress related to all websites worldwide

http://w3techs.com/technologies/history_overview/content_management/all/y

# WEBSITES WORLDWIDE

1.058.544.813http://www.internetlivestats.com/total-number-of-websites/

38% of E-Shops

26,6%2,7%2,2%

High Traffic Sites???

NEEDFOR

SPEED

my-wordpress-site.com

my-wordpress-site.com

MYSQL ServerLoad-Balancer

my-wordpress-site.com

MYSQL ServerLoad-Balancer

File Storage

my-wordpress-site.com

MYSQL Server

Load-Balancer

File Storage

CDN

MYSQL Server

Kommt ihr noch mit?

DEPLOYMENT

2015 20160%

5%

10%

15%

20%

25%

30%

Nutzung von Docker

verdoppelt auf 30%http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2016-state-cloud-survey

my-wordpress-site

Moderne Web-

Entwicklung2016

IMMUTABLE INFRASTRUCTURE

IMMUTABLE STACKS

ELB myapp-v1

app.example.org

EC2+ Docker

EC2+ Docker

EC2+ Docker

100%

IMMUTABLE STACKS

ELB myapp-v1

EC2+ Docker

EC2+ Docker

EC2+ Docker

ELB myapp-v2

EC2+ Docker

EC2+ Docker

app.example.org

90%

10%

IMMUTABLE STACKS

ELB myapp-v1

EC2+ Docker

EC2+ Docker

EC2+ Docker

ELB myapp-v2

EC2+ Docker

EC2+ Docker

app.example.org

0%

100%

APP INSTANCE

VM (EC2)

Basis Image (AMI)

Docker Container

Application

✓ VPC✓ Security

Groups✓ AutoScaling✓ CloudWatch✓ CloudFront✓ Route53

✓ Docker Runtime✓ Logging✓ Monitoring✓ Security config

✓ Debian✓ Nginx✓ php7-fpm✓ NewRelic

AWS

API

Docker Registry

docker pull

docker push

AMI

DER WUNSCH

DIE REALITÄT

$ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface

$ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$

$ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$

$ describe-reserved-instances$ $ describe-reserved-instances-listings$ $ describe-reserved-instances-modifications$ $ describe-reserved-instances-offerings$ $ describe-route-tables$ $ describe-scheduled-instance-availability$

$ describe-scheduled-instances$ $ describe-security-group-references$ $ describe-security-groups$ $ describe-snapshot-attribute$ $ describe-snapshots$ $ describe-spot-datafeed-subscription$ $ describe-spot-fleet-instances$ $ describe-spot-fleet-request-history$ $ describe-spot-fleet-requests$ $ describe-spot-instance-requests$ $ describe-spot-price-history$ $ describe-stale-security-groups$ $ describe-subnets$ $ describe-tags$ $ describe-volume-attribute$ $ describe-volume-status$ $ describe-volumes$ $ describe-vpc-attribute$ $ describe-vpc-classic-link$ $ describe-vpc-classic-link-dns-support$ $ describe-vpc-endpoint-services$ $ describe-vpc-endpoints$ $ describe-vpc-peering-connections$ $ describe-vpcs$ $ describe-vpn-connections$ $ describe-vpn-gateways$ $ detach-classic-link-vpc$ $ detach-internet-gateway$ $ detach-network-interface$ $ detach-volume$ $ detach-vpn-gateway$ $ disable-vgw-route-propagation$ $ disable-vpc-classic-link$ $ disable-vpc-classic-link-dns-support$ $ disassociate-address$ $ disassociate-route-table$ $ enable-vgw-route-propagation$ $ enable-volume-io$ $ enable-vpc-classic-link$ $ enable-vpc-classic-link-dns-support$ $ get-console-output$ $ get-console-screenshot$ $ get-host-reservation-purchase-preview$ $ get-password-data$ $ help$ $ import-image$ $ import-key-pair$ $ import-snapshot$ $ modify-hosts$ $ modify-id-format$ $ modify-identity-id-format$ $ modify-image-attribute$ $ modify-instance-attribute$ $ modify-instance-placement$ $ modify-network-interface-attribute$ $ modify-reserved-instances$ $ modify-snapshot-attribute$ $ modify-spot-fleet-request$ $ modify-subnet-attribute$ $ modify-volume-attribute$ $ modify-vpc-attribute$ $ modify-vpc-endpoint$ $ modify-vpc-peering-connection-options$ $ monitor-instances$ $ move-address-to-vpc$ $ purchase-host-reservation$ $ purchase-reserved-instances-offering$ $ purchase-scheduled-instances$ $ reboot-instances$ $ register-image$ $ reject-vpc-peering-connection$ $ release-address$ $ release-hosts$ $ replace-network-acl-association$ $ replace-network-acl-entry$ $ replace-route$ $ replace-route-table-association$ $ report-instance-status$ $ request-spot-fleet$ $ request-spot-instances$ $ reset-image-attribute$ $ reset-instance-attribute$ $ reset-network-interface-attribute$ $ reset-snapshot-attribute$ $ restore-address-to-classic$ $ revoke-security-group-egress$ $ revoke-security-group-ingress$ $ run-instances$ $ run-scheduled-instances$ $ start-instances$ $ stop-instances$ $ terminate-instances$ $ unassign-private-ip-addresses$ $ unmonitor-instances$ $ wait

210 calls for EC2 only

$ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface

$ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$

$ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$

$ describe-reserved-instances$ $ describe-reserved-instances-listings$ $ describe-reserved-instances-modifications$ $ describe-reserved-instances-offerings$ $ describe-route-tables$ $ describe-scheduled-instance-availability$

$ describe-scheduled-instances$ $ describe-security-group-references$ $ describe-security-groups$ $ describe-snapshot-attribute$ $ describe-snapshots$ $ describe-spot-datafeed-subscription$ $ describe-spot-fleet-instances$ $ describe-spot-fleet-request-history$ $ describe-spot-fleet-requests$ $ describe-spot-instance-requests$ $ describe-spot-price-history$ $ describe-stale-security-groups$ $ describe-subnets$ $ describe-tags$ $ describe-volume-attribute$ $ describe-volume-status$ $ describe-volumes$ $ describe-vpc-attribute$ $ describe-vpc-classic-link$ $ describe-vpc-classic-link-dns-support$ $ describe-vpc-endpoint-services$ $ describe-vpc-endpoints$ $ describe-vpc-peering-connections$ $ describe-vpcs$ $ describe-vpn-connections$ $ describe-vpn-gateways$ $ detach-classic-link-vpc$ $ detach-internet-gateway$ $ detach-network-interface$ $ detach-volume$ $ detach-vpn-gateway$ $ disable-vgw-route-propagation$ $ disable-vpc-classic-link$ $ disable-vpc-classic-link-dns-support$ $ disassociate-address$ $ disassociate-route-table$ $ enable-vgw-route-propagation$ $ enable-volume-io$ $ enable-vpc-classic-link$ $ enable-vpc-classic-link-dns-support$ $ get-console-output$ $ get-console-screenshot$ $ get-host-reservation-purchase-preview$ $ get-password-data$ $ help$ $ import-image$ $ import-key-pair$ $ import-snapshot$ $ modify-hosts$ $ modify-id-format$ $ modify-identity-id-format$ $ modify-image-attribute$ $ modify-instance-attribute$ $ modify-instance-placement$ $ modify-network-interface-attribute$ $ modify-reserved-instances$ $ modify-snapshot-attribute$ $ modify-spot-fleet-request$ $ modify-subnet-attribute$ $ modify-volume-attribute$ $ modify-vpc-attribute$ $ modify-vpc-endpoint$ $ modify-vpc-peering-connection-options$ $ monitor-instances$ $ move-address-to-vpc$ $ purchase-host-reservation$ $ purchase-reserved-instances-offering$ $ purchase-scheduled-instances$ $ reboot-instances$ $ register-image$ $ reject-vpc-peering-connection$ $ release-address$ $ release-hosts$ $ replace-network-acl-association$ $ replace-network-acl-entry$ $ replace-route$ $ replace-route-table-association$ $ report-instance-status$ $ request-spot-fleet$ $ request-spot-instances$ $ reset-image-attribute$ $ reset-instance-attribute$ $ reset-network-interface-attribute$ $ reset-snapshot-attribute$ $ restore-address-to-classic$ $ revoke-security-group-egress$ $ revoke-security-group-ingress$ $ run-instances$ $ run-scheduled-instances$ $ start-instances$ $ stop-instances$ $ terminate-instances$ $ unassign-private-ip-addresses$ $ unmonitor-instances$ $ wait

210 calls for EC2 only

S3ELB RDS

CloudFront

CloudWatchAutoScali

ng

Route53 Event

s

IAM

S3api

SNS

PLESK WORDPRESS AWS SCALER

https://github.com/plesk/wordpress-aws-scaler

AWS

manage-wordpress.sh

Docker Registry

docker pull

docker push

AMI

PLESK WORDPRESS AWS SCALER

manage-wordpress.sh create

Too quick? How does i t work internal ly?

Install AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/installing.html)

$ sudo pip install awscli$ aws configure$ aws --version

Clone WordPress AWS Scaler git repo (https://github.com/plesk/wordpress-aws-scaler)

$ git clone https://github.com/plesk/wordpress-aws-scaler.git$ cd wordpress-aws-scaler

Build Docker Image and Upload to Docker Hub (https://hub.docker.com/explore/)

$ docker build -t janloeffler/wordpress-aws-scaler:latest -t janloeffler/wordpress-aws-scaler:0.2 .$ docker push janloeffler/wordpress-aws-scaler:latest

List exist ing AWS resources

List existing AWS Resources

$ aws ec2 describe-vps $ aws ec2 describe-security-groups $ aws ec2 describe-instances$ aws rds describe-db-instances[...]

Do this for all AWS components

$ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events

Create new resources

Create new AWS Resources

$ aws ec2 create-vpc$ aws ec2 create-security-group[...]

Do this for all AWS components

$ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events

Create new Launch Configurat ion

Generate EC2-User-Data Script to run directly after booting EC2 VM

$ cat >ec2-user-data.sh <<EOL#!/bin/bashdocker pull janloeffler/wordpress-aws-scaler:latestdocker run -d -p 80:80 -p 443:443 -e WORDPRESS_DB_HOST='${DB}' -e WORDPRESS_DB_USER='${DB_USERNAME}' -e WORDPRESS_DB_PASSWORD='${DB_PASSWORD}' -e WORDPRESS_DB_NAME='${DB_NAME}' -e WORDPRESS_DB_PREFIX='${WORDPRESS_DB_PREFIX}' -e WORDPRESS_URL='http://${ELB}' -e WORDPRESS_TITLE='${WORDPRESS_TITLE}' -e WORDPRESS_USER_EMAIL='${WORDPRESS_USER_EMAIL}' -e NEWRELIC_KEY='${NEWRELIC_KEY}' -e NEWRELIC_NAME='${NEWRELIC_NAME}' -e S3_KEY='${IAM_USER_KEY}' -e S3_SECRET='${IAM_USER_SECRET}' -e S3_BUCKET='${S3_BUCKET_NAME}' -e S3_BUCKET_URL='${S3_URL}' -it janloeffler/wordpress-aws-scaler:latestEOL

Create new Launch Configuration for AutoScaling

$ aws autoscaling create-launch-configuration --launch-configuration-name $LC_NAME --image-id $AMI --instance-type $INSTANCE_TYPE --key-name $KEYNAME --security-groups $SEC_GROUP_ID --user-data file://ec2-user-data.sh

You must love JSON !!! ; -){ "Reservations": [ { "OwnerId": "699328319947", "ReservationId": "r-041ac13209e0d4eef", "Groups": [], "RequesterId": "226008221399", "Instances": [ { "Monitoring": { "State": "enabled" }, "PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com", "State": { "Code": 16, "Name": "running" }, "EbsOptimized": false, "LaunchTime": "2016-09-05T12:19:54.000Z", "PublicIpAddress": "52.210.200.157", "PrivateIpAddress": "172.31.25.83", "ProductCodes": [ { "ProductCodeId": "4jvb72q6a56js2x7jzd24jar5", "ProductCodeType": "marketplace" } ], "VpcId": "vpc-fffbe19a", "StateTransitionReason": "", "InstanceId": "i-0ea3384a24a3c6aba", "ImageId": "ami-64385917", "PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal", "KeyName": "Marko Heijnen", "SecurityGroups": [ { "GroupName": "WordPress-Docker-Test", "GroupId": "sg-1b8dde7c" } ],

"ClientToken": "70aa81e1-10a3-4b83-acc7-62702015d06e_subnet-cff90cb9_1", "SubnetId": "subnet-cff90cb9", "InstanceType": "m3.medium", "NetworkInterfaces": [ { "Status": "in-use", "MacAddress": "06:c0:22:65:b4:ed", "SourceDestCheck": true, "VpcId": "vpc-fffbe19a", "Description": "", "Association": { "PublicIp": "52.210.200.157", "PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com", "IpOwnerId": "amazon" }, "NetworkInterfaceId": "eni-2f4efa63", "PrivateIpAddresses": [ { "PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal", "Association": { "PublicIp": "52.210.200.157", "PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com", "IpOwnerId": "amazon" }, "Primary": true, "PrivateIpAddress": "172.31.25.83" } ], "PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal", "Attachment": { "Status": "attached", "DeviceIndex": 0, "DeleteOnTermination": true, "AttachmentId": "eni-attach-7f9031a0", "AttachTime": "2016-09-05T12:19:54.000Z" },

[…]

Plesk Mission

“We simplify the lives of web professionals.”

Jan LöfflerCTO Plesk● @jlsoft2● jan@plesk.com● http://www.slideshare.net/jlsoft/● https://about.me/jan.loeffler

code.talks 2016, Hamburg

Keynote:

“High traffic sites with WordPress, Docker &

AWS”

How to auto-scale WordPress on AWS and make it fly!

Stay tuned!Jan Löffler Marko

Heijnen

BACKUP

47

We b D e v e l o p m e n t Ke y To o l s – w h a t w e b d e v e l o p e r s n e e d a n d u s e

26,6% of all websites worldwide

2,7% 2,2%(high traffic

sites)

70% of developers use git as primary source management solution

Increasing usage by web agencies

38% of all online stores worldwide

runs on top of WordPress as plugindeveloped by automattic

used by 30% of IT companiesincreased from 13% to 30% in 12 monthsfurther 35% plan introduction

Sources: w3techs.com, Rightscale, StackOverflow Survey 2015, 2016, Netcraft

311% growth

17.2% usage26.8% growth

17.9% usage14.9% growth

16% of all webservers22% of all active sites

continuously growing, while Apache is shrinking

WordPress Management

Free SSL everywhere

http2 50+ extensions

CLI

Automatic UpdatesSite Migration Free support

30+ languages nginxServer Security DNS

✔

Plesk Onyx: Continuous Delivery Deployment Pipeline (sample)

49

Jenkins runs as Docker container managed by Plesk

Plesk Multi Server Extension installs three environments

Apps managed by Plesk via Docker