altitude sf 2017: fastly gslb: scaling your microservice and multi-cloud environments

59
Fastly GSLB: Scaling your microservice and multi-cloud environments Chris Buckley Sales Engineer | Fastly

Upload: fastly

Post on 22-Jan-2018

388 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Fastly GSLB: Scaling your microservice and multi-cloud environments

Chris BuckleySales Engineer | Fastly

Page 2: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Sales Engineer @ Fastly, Media & Publishing

• Previously Director of DevOps @ Business Insider

A little about me...

Page 3: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Introduction to Load Balancing

• Workshop 1: Multi-Cloud Load Balancing

• Workshop 2: SOA/Microservice Routing

• Workshop 3: Geographic Load Balancing

Load Balancing Workshop Program

Page 4: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Introduction to Load Balancing

Page 5: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

As the name implies, load balancing is designed to distribute traffic between

multiple resources.

From Wikipedia:

“Load balancing aims to optimize resource use, maximize throughput,

minimize response time, and avoid overload of any single resource.”

Load Balancers, How Do They Work?

Page 6: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Global Server Load Balancing (GSLB) is a method of

directing traffic across multiple networks, be it on the

same site, or on the other side of the world.

• GSLB can be used to load balance across multiple

networks, and primarily uses DNS.

Traditional GSLBs (Global Server Load Balancers)

Page 7: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Fastly GSLBs

● Fastly allows customers to make unlimited number of custom rules

● Fastly software load balances between nodes in POP

● Fastly uses DNS to load balance between POPs

BrowserRecursive Resolver

Authoritative Name Server

Reverse HTTP Proxy

Application

DN

SL4

L7

Fastly executes rules

TCP Handshake

Page 8: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Multi-Cloud/Multi-Datacenter Load Balancing

88

Infrastructure agnostic global andlocal server load balancing

Unified approach to microservice architecture

Page 9: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Infrastructure-Agnostic Distribution

● Manage traffic across multiple IaaS, datacenters, and hybrid-clouds

● Use as GSLB and/or LSLB● Instant convergence and failover

Content-Aware Routing

● Load balance requests to servers using an any number of custom rules

● Support microservices architecture● Choose from various distribution algorithms

to manage load across servers

Immediate Control

● Programmatically add/delete/modify your servers without having to version your VCL

● Automatic server health checks● Changes to custom rules update routing

configuration globally within seconds

Traffic Scalability

● Instantly scale to multiple terabits per second (Tbps) in a manner that is both cost-effective and transparent

● Mitigates thundering herd problem

Why Fastly Load Balancing?

Page 10: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Before we begin...

Page 11: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• To ease usage of the API, set environment variables for API key and service

ID.

• Copy API key from the Github README.md, and your service ID from the

sheet given to you at the beginning of this workshop

• Assuming a Bash shell, run the following:

Initial Setup

export API_KEY=<api_key>

export SERVICE_ID=<service_id>

Page 12: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• We will be working with an API that returns JSON as its response.

• Make sure you are taking note of the responses!

• JQ can be used for making responses readable (link in README.md).

• If at any time you get version drift in this workshop, you can find out your

current active version and work from that using the following command:

General Workshop Tips

curl sv -H "Fastly-Key: ${API_KEY}" \https://api.fastly.com/service/${SERVICE_ID}/details | jq

Page 13: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

You will receive a JSON response listing all versions. You will want to find the one

where "active": true:

General Workshop Tips cont...

{"testing": false,"locked": true,"number": 10,"active": true,"service_id": "3SewbytL2TOibn8tO3OFrM","staging": false,"created_at": "2017-06-16T01:07:30+00:00","deleted_at": null,"comment": "","updated_at": "2017-06-16T01:14:14+00:00","deployed": false

},

You can then clone from your current active version and work from there.

Page 14: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Workshop 1: Multi-cloud Load Balancing

Page 15: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Get familiar with the Fastly API

• Learn about Dynamic Server Pools

• Use a pool to add servers dynamically

• Create a multi-cloud load balancing configuration

Goals of Workshop 1: Multi-cloud Load Balancing

Page 16: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Goals of Workshop 1: Multi-cloud Load Balancing

Page 17: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• In order to begin adding Dynamic Server Pools, we will first need to clone

your service and create a new pool.

• In this case we will be cloning version 1 of your service to version 2:

Step 1: Cloning a new version

curl -sv -H "Fastly-Key: ${API_KEY}" \https://api.fastly.com/service/${SERVICE_ID}/version/1/clone \| jq

Page 18: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

JSON response:

Step 1: Cloning a new version cont..

{"testing": false,"locked": false,"number": 2,"active": false,"service_id": "1OPpKYvOlWVx37twfGVsq0","staging": false,"created_at": "2017-06-16T21:10:09+00:00","deleted_at": null,"comment": "","updated_at": "2017-06-16T21:10:11+00:00","deployed": false

}

Page 19: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

In the repo there is included a main.vcl to upload to our service. Two things to

note:

1. There is a clearly defined space in vcl_recv where we will be doing our

custom VCL for this workshop.

2. Below you will see return(pass). For this workshop we are passing all traffic

to the backends so that we can see immediate responses (and not cached

responses).

Step 2: Upload boilerplate VCL for service

Page 20: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

To upload VCL we must URL encode the file so that we can send it via Curl:

Step 2: Upload boilerplate VCL for service cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST -H \"Content-Type: application/x-www-form-urlencoded" \--data "name=main&main=true" \--data-urlencode "[email protected]" \https://api.fastly.com/service/${SERVICE_ID}/version/2/vcl | jq

Page 21: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

JSON response:

Step 2: Upload boilerplate VCL for service cont...

{"name": "main","main": true,"content": "<lots of VCL>,"service_id": "1OPpKYvOlWVx37twfGVsq0","version": 2,"deleted_at": null,"created_at": "2017-06-20T20:23:52+00:00","updated_at": "2017-06-20T20:23:52+00:00"

}

Page 22: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Next, we will need to create a Dynamic Server Pool to add our servers to (note we are now working with version 2):

Step 3: Create Dynamic Server Pool

curl -sv -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/version/2/pool -d \'name=cloudpool&comment=cloudpool' | jq

Page 23: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

JSON response:

Step 3: Create Dynamic Server Pool cont...

{"name": "cloudpool","comment": "cloudpool","service_id": "5wFmyFopB74kQoYtrxY3tw","version": "2","max_tls_version": null,"shield": null,"tls_ca_cert": null,"request_condition": null,"first_byte_timeout": "15000","tls_ciphers": null,"tls_sni_hostname": null,"updated_at": "2017-06-21T20:37:51+00:00","id": "3qoucl7vkyLGf2KXmJ1XIK",…

}

Page 24: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Let’s add the pool ID as an environment variable...

Step 3: Create Dynamic Server Pool cont...

export POOL_ID_1=3qoucl7vkyLGf2KXmJ1XIK

Page 25: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

We can now begin to start adding servers to the pool (use the IPs listed above to add the servers)

You will run this command twice with the different IP addresses:

Step 4: Add servers to the pool

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_1}/server -d \'address=X.X.X.X' | jq

Page 26: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

JSON response:

Step 4: Add servers to the pool cont...

{"address": "13.58.97.100","service_id": "1OPpKYvOlWVx37twfGVsq0","pool_id": "49Y2yUtPUukwoGu9KDxFM","deleted_at": null,"port": "80","max_conn": 0,"created_at": "2017-06-20T20:55:33+00:00","comment": "","weight": "100","updated_at": "2017-06-20T20:55:33+00:00","id": "6G190tLMQZ9QDE2jRcLkwA"

}

Page 27: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Our last configuration step. Now we have added our pool, activate the version.

Once activated, dynamic servers are not tied to a version and can be added and removed dynamically:

Step 5: Activate new version

curl -vs -H "Fastly-Key: ${API_KEY}" -X PUT \https://api.fastly.com/service/${SERVICE_ID}/version/2/activate |

jq

Page 28: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

JSON response:

Step 5: Activate new version cont...

{"testing": false,"locked": true,"number": 2,"active": true,"service_id": "1OPpKYvOlWVx37twfGVsq0","staging": false,"created_at": "2017-06-20T20:09:40+00:00","deleted_at": null,"comment": "","updated_at": "2017-06-20T21:03:44+00:00","deployed": false,"msg": "WARNING: Unused backend 'dummy'"

}

You can safely ignore the dummy WARNING, it is an unused pool to activate your initial service for this workshop.

Page 29: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Open your browser and navigate to your supplied domain:

<num>.lbworkshop.tech

You should now be able to see something like this:

Step 6: Browse the new load balanced pool

GCE serving the request EC2 serving the request

Page 30: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

10 minute break...

Page 31: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Workshop 2: SOA / Microservice Routing

Page 32: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Get familiar with Dynamic Server Pools & TLS

• Define a routing condition for sending to your

microservice Server Pool.

• Create two different Server Pools for routing to

different independently scalable load balancers.

Goals of Workshop 2: SOA/Microservice Routing

Page 33: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Goals of Workshop 2: SOA/Microservice Routing

Page 34: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Clone the active service to a new development version (this time cloning

version 2 to version 3):

Step 1: Create Dynamic Server Pool

curl -sv -H "Fastly-Key: ${API_KEY}" -X PUT \https://api.fastly.com/service/${SERVICE_ID}/version/2/clone | jq

{"testing": false,"locked": false,"number": 3,....

}

● JSON response:

Page 35: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Add an environment variable for the blog hostname (for TLS)::

Step 1: Create Dynamic Server Pool cont...

export BLOG_HOST=blog.lbworkshop.tech

● Create our Server Pool (with TLS data):

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/version/3/pool -d \"name=blog&comment=blog&use_tls=1&tls_cert_hostname=${BLOG_HOST}|

jq

Page 36: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• JSON response:

Step 1: Create Dynamic Server Pool cont...

{"name": "wordpress","comment": "wordpress","use_tls": 1,"tls_cert_hostname": "blog.lbworkshop.tech","version": "3","id": "5sufVWoTHlOAOYVRm5jg2G",…

}

● Export the new pool ID (for use later):

export POOL_ID_BLOG=5sufVWoTHlOAOYVRm5jg2G

Page 37: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• In your repo you will find a file wordpress.vcl.

• This contains VCL to send any requests on the blog route (in this case /blog) to the new created created Wordpress load balancer.

• Let's add this to main.vcl underneath our workshop 1 set req.backend = cloudpool;. It should look something more like this, ready to upload:

Step 2: Add conditional routing for blog pool

main.vcl before:

############################################ LB WORKSHOP - DO STUFF HERE###########################################

# Workshop 1 default backend.set req.backend = cloudpool;

############################################ END LB WORKSHOP###########################################

Page 38: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

main.vcl after:

Step 2: Add conditional routing for blog pool cont...

# Workshop 1 default backend.set req.backend = cloudpool;

# Adding conditional routing to our Wordpress serviceif (req.url == "/blog") {

set req.backend = wordpress;# We will also need to change the host header and modify the request url# so that we hit the right endpoint.set req.http.Host = "blog.lbworkshop.tech";# Remove the /blog from the request URL so that we hit the root of the service# using regular expressionsset req.url = regsub(req.url, "^/blog", "/");

}

Page 39: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Upload this as main-blog so to use the new configuration and backend for the blog:

Step 2: Add conditional routing for blog pool cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST -H "Content-Type: \application/x-www-form-urlencoded" --data \"name=main-blog&main=true" --data-urlencode "[email protected]" \https://api.fastly.com/service/${SERVICE_ID}/version/3/vcl | jq

Page 40: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• JSON response:

Step 2: Add conditional routing for blog pool cont...

{"name": "main-blog","main": true,"content": <lots of vcl>,"service_id": "1OPpKYvOlWVx37twfGVsq0","version": 3,"deleted_at": null,"created_at": "2017-06-21T05:04:54+00:00","updated_at": "2017-06-21T05:04:54+00:00"

}

Page 41: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Add the Wordpress TLS endpoint to the blog pool (using the previously added POOL_ID_BLOG environment variable):

Step 3: Add TLS backend to Server Pool

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_BLOG}/server -d \"address=${BLOG_HOST}&comment=wp&port=443" | jq

Page 42: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• JSON response:

Step 3: Add TLS backend to Server Pool cont...

{"address": "blog.lbworkshop.tech","comment": "wp","service_id": "1OPpKYvOlWVx37twfGVsq0","pool_id": "5sufVWoTHlOAOYVRm5jg2G","deleted_at": null,"port": "443","max_conn": 0,"created_at": "2017-06-21T05:11:18+00:00","weight": "100","updated_at": "2017-06-21T05:11:18+00:00","id": "RzWDZMrMIV1FiHYUY4k5k"

}

Page 43: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Activate new version (ensuring we’re working with version 3):

Step 3: Add TLS backend to Server Pool cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X PUT \https://api.fastly.com/service/${SERVICE_ID}/version/3/activate | jq

Page 44: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Navigate to the new endpoint and see the blog in all its glory (replace the boilerplate URL with your custom service URL):

http://<num>.lbworkshop.tech/blog

Step 4: Check out your new blog!

Page 45: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Workshop 3: Geographic Dynamic Server Pools

Page 46: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Use geographic conditions to route to the closest origin

load balancing pool.

• Learn about failover logic in the case of origins being

unavailable.

• Define a fault tolerant configuration built for speed and

redundancy.

Goals of Workshop 3: Geographic Dynamic Server Pools

Page 47: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• As we will be introducing code to fail over to a secondary origin (in the case of the primary origin failing or being unavailable), we will need to add health checks to our service.

• Set a simple check to test the index of the site, with default health check settings.

• First, clone current version (this time to version 4):

Step 1: Create origin health check

curl -sv -H "Fastly-Key: ${API_KEY}" -X PUT \https://api.fastly.com/service/${SERVICE_ID}/version/3/clone | jq

Page 48: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Add health check to the new version. As the health check will be done over HTTP/1.1 we will also add a host header in the check:

Step 1: Create origin health check cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/version/4/healthcheck -d \"name=geo-healthcheck&host=lbworkshop.tech&path=/" | jq

Page 49: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

JSON response:

Step 1: Create origin health check cont...

{"name": "geo-healthcheck","path": "/","service_id": "1OPpKYvOlWVx37twfGVsq0","version": 4,"threshold": 3,"window": 5,"http_version": "1.1","timeout": 500,"method": "HEAD","updated_at": "2017-06-16T19:05:02+00:00","expected_response": 200,"deleted_at": null,"host": "lbworkshop.tech",…

}

Page 50: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Create 2 server pools. One called "west" and one "east".

• Attach the health check created previously to each pool.

• East first:

Step 2: Create Geographic Dynamic Server Pools

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/version/4/pool -d \'name=east&comment=east&healthcheck=geo-healthcheck' | jq

● Add the pool ID as an environment variable:

export POOL_ID_EAST=<pool_id>

Page 51: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• And then west:

Step 2: Create Geographic Dynamic Server Pools cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/version/4/pool -d \'name=west&comment=west&healthcheck=geo-healthcheck' | jq

● Add environment variable:

export POOL_ID_WEST=<pool_id>

Page 52: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Add two instances from workshop 1 into separate pools; the GCE instance into west, and the EC2 instance into east (run twice, with the different pool IDs and the instance for each pool):

Step 2: Create Geographic Dynamic Server Pools cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_EAST}/server -d \'address=13.58.97.100&comment=ec2' | jq`

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST \https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_WEST}/server -d \'address=104.196.253.201&comment=gcs' | jq

Page 53: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• In your repo you will find a file geo.vcl. This contains the logic for backend selection based on geography, with failover to the secondary origin in case the original pool is unavailable (based on the health checks).

• Edit the main.vcl file, and replace the following code:

Step 3: Adding VCL for backend selection

# Workshop 1 default backend.set req.backend = cloudpool;

Page 54: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Replace it with the new geo.vcl code:

Step 3: Adding VCL for backend selection cont...

# APAC, Asia, and US West all go to US West backend.if (server.region ~ "^(US-West|APAC|Asia)") {

set req.backend = west;# All other regions default to US East} else {

set req.backend = east;}

# West failover to East# from unhealthy backend or from restart because of backend status codeif(req.backend == west && (!req.backend.healthy || req.restarts > 0)) {

set req.backend = east;# East failover to West} else if(req.backend == east && (!req.backend.healthy || req.restarts > 0)) {

set req.backend = west;}

Page 55: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Upload main.vcl as an update to development version (we have given the new VCL a new name to distinguish from the old, so that we can set this as the new "main":

Step 3: Adding VCL for backend selection cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X POST -H \"Content-Type: application/x-www-form-urlencoded" \https://api.fastly.com//service/${SERVICE_ID}/version/4/vcl --data \"name=main-blog-geo&main=true" --data-urlencode "[email protected]"

Page 56: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Activate our new version and see the new Geo Load Balancing in effect:

Step 3: Adding VCL for backend selection cont...

curl -vs -H "Fastly-Key: ${API_KEY}" -X PUT \https://api.fastly.com/service/${SERVICE_ID}/version/4/activate

● As this workshop is being held in San Francisco, you should be see the GCE instance. After Apache is shut down on the GCE us-west1 instance, you should see failover to our EC2 instance in us-east2.

Page 57: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Thank You

Page 58: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

• Workshop GitHub Repository: http://bit.ly/lbworkshop

• API documentation: https://docs.fastly.com/api/

• Dynamic servers documentation: https://docs.fastly.com/guides/dynamic-servers/

• Working with services: https://docs.fastly.com/api/config#service

• Working with service versions: https://docs.fastly.com/api/config#version

• Conditions documentation: https://docs.fastly.com/guides/conditions/

• Geo related features: https://docs.fastly.com/guides/vcl/geoip-related-vcl-features

• Creating health checks via the API: https://docs.fastly.com/api/config#healthcheck

Further Reading

Page 59: Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Personal Contact Information

• Name: Chris Buckley

• Email: [email protected]

• Twitter: @ChrisBuckleySA

Contact Information

Fastly Contact Information

• Email: [email protected]

• IRC: #fastly on Freenode

• Community: https://community.fastly.com/