The Developer’s GuideTO AMAZON WEB SERVICES

1Guide to Amazon Web Services

1 What is Infrastructure as a Service?

2 How to Determine if IaaS is right for you

2 IaaS may not be right for you if...

3 The Major IaaS Players

4 Pro Tips for IaaS Evaluation

5 5 Best Practices for Deploying an Application on AWS

1Guide to Amazon Web Services

1 What is Infrastructure as a Service?

2 How to Determine if IaaS is right for you

2 How to determine if IaaS may not be right for you

3 The Major IaaS Players

4 Pro Tips for IaaS Evaluation

5 5 Best Practices for Deploying an Application on AWS

4AWS Outage Survival Guide

6 Surviving AWS Failures with a Node.js & MongoDB Stack

10 Migrating from GoDaddy DNS to Amazon Route 53

11 Recovering from a DNS service outage in AWS using Monit

Table of Contents

i

Chapter 1Guide to Amazon Web Services

1

Welcome to the latest Kinvey eBook. We

aim to help our community of develop-

ers keep pace with the latest trends in

app development, tools and marketing.

Typically we share our perspective on

our blog, but when our audience is

interested in a topic that can’t be done

justice in 500 words, we publish an

eBook instead.

This eBook curates some of the best

thinking from Kinvey and outside

experts on how developers should

approach Infrastructure as a Service

(IaaS). The eBook emphasizes Amazon

Web Services (AWS) because it’s the best

known vendor in the space, though we

don’t recommend one provider over

another. The eBook highlights the

benefits of IaaS, helps in the IaaS vendor

selection process, shares best practices

for hosting an app on AWS, and provides

tips for what to do in case of a service

outage.

What is Infrastructure as a Service?There are a handful of “__ as a service”

(*aaS) providers comprising the mobile

and cloud computing ecosystems today.

Although individually each company may

be unique, collectively they share a

common goal: accelerating the rate of

innovation by removing costs and

barriers to technology deployment. IaaS

2

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

is perhaps the most fundamental of the

categories. Before diving into our guide,

let’s first review IaaS basics:

According to TechTarget, Infrastructure

as a Service is a cloud-based provision

model that services cloud storage, virtual

servers and networking components to

application owners for a usage-based

cost. Its goal is to is to become a

foundation for Platform as a Service

(PaaS) and Software as a Service (SaaS)

providers by providing a flexible

operating environment. In the IaaS

environment, the service provider owns,

runs and maintains the infrastructure

equipment, while the consumer takes

responsibility for configuration and

operations of the guest operating

system, software and database.

A variety of technologies can benefit

from IaaS: cloud-based CRM systems,

web, media and mobile applications, big

data systems, and much more. But, IaaS

is not right for everyone. There are some

important factors to consider before

determining if your application would

benefit from IaaS.

Chapter 2How to Determine if IaaS is right for you

3

+–

There are pros and cons associated with IaaS

that must be assessed...

“

”

Choosing an IaaS provider is a big

decision. You want to trust the provider

with your data and, essentially, your

entire application infrastructure. As with

any service, there are pros and cons

associated with IaaS that must be

assessed before deciding whether or not

to use it for your application.

You could benefit from IaaS if...• there is the potential for spikes in users

or usage. Do you have upcoming press

coverage that may cause spikes in

downloads? Is your application

seasonal?

• you plan to expand your feature set.

With an increase in features comes an

increased demand on infrastructure.

“Scalability” doesn’t only apply to the

number of users, sometimes features

too need to scale.

• you’re an individual developer, small dev

shop or startup with no existing data

center infrastructure or you’re an

established company taking on a large

project that would require significant

additional data center infrastructure or

staff.

• you have a low server-to-admin ratio

and are looking to cut costs.

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

IaaS may not be right for you if...• usage is minimal or flat, and you have

no plans to drive significant growth.

Maybe your product is built specifically

for only a small subset of users (e.g., an

internal application for a small company).

• you don’t have a clear sense of your

application’s storage and networking

needs. Successful IaaS deployments

benefit from clear user up-front

requirements.

• you are an enterprise concerned about

rogue users. (One risk of IaaS is rogue or

unwarranted commandeering of

services. Because IaaS requires gover-

nance and usage monitoring, Vordel's

Mark O'Neill recommends that

enterprises establish cloud service

governance frameworks that help

prevent employees from accessing

information or services they are not

permitted to use.)

4

5

The Major IaaS Players: A ComparisonOnce you’ve decided to use an IaaS

solution, it’s time to pick a vendor. While

AWS may be the most prevalent player

in the space - it is estimated to own

roughly 70 percent of the IaaS market -

there are several other reputable

vendors to consider, and many factors to

evaluate. Rest assured, there are plenty

of resources out there to help you

narrow down the options.

This chart from TechRepublic evaluates

the best-known IaaS providers against

how they compare to common “cloud

promises.” The comparison takes into

account a range of factors, including

pricing (variety, average, data transfer

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

and storage costs), scalability (scaling up

and down, monitoring, and APIs), and

choice / flexibility (number of data

center locations, number of instance

types, and supported operating

systems).

Another way to size-up vendors is by

common “user concerns,” which is the

thrust of this second TechRepublic chart.

Specifically, the table assesses vendors

against security features (certifications

and protection), ease of migration (open

standards and VM upload), and

reliability [service age, Service Level

Agreement (SLA), and support].

Pro Tips for IaaS Evaluation Regardless of whether you select an IaaS

vendor based on its “cloud promises” or

“user concerns,” Kinvey’s lead architect,

Shubhang Mani, advises you consider

the following:

Is IaaS really the solution you need?

Depending on the complexity of your

infrastructure needs, you might be

better off opting for a higher layer in the

stack such as a Platform as a Service

(PaaS), or even Backend as a Service

Beyond data, developers want to tap into many tools and services that other clever minds have

created...

“

”

+–

(BaaS). The advantages of choosing

these alternatives are reduced opera-

tional complexity and decreased time to

market. The disadvantages are reduced

control and a narrower set of choices as

to the underlying infrastructure

components. PaaS or BaaS might be a

good starting point, allowing you to

focus on building the application /

system until you deem it necessary to

exert greater control on the choice of

infrastructure components.

IaaS is not a silver bullet

Some key benefits of IaaS include ease in

deployment, redundancy, and the ability

to scale much faster than conventional

means. Deploying a distributed system,

especially across geographic regions, can

also be achieved in more easily.

However, it is important to note that

while IaaS gives you the means to

provision, deploy and scale infrastruc-

ture, it is up to you to configure, monitor

and maintain said infrastructure and use

it in a manner that makes the most

sense to your system. You might be able

to build multiple redundancies into a

complex system sitting across geograph-

ic regions in a matter of hours, but if

your firewall rules are improperly

configured, you’re still vulnerable to

attack.

6

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

Plan for the worst case

No system is infallible. Outages can

occur, even at your cloud provider of

choice. It is important that you have a

plan in place to address this were it to

occur. For example, you should consider

offsite backups for your critical data and

alternate deployments from your main

location.

Understand your support requirements

Not all IaaS providers provide the same

levels of support. In some cases, support

is priced independently from the actual

product and may end up being quite

expensive. Most providers offer a free

support level via community forums.

This may or may not be sufficient

depending on your needs.

With so many factors to consider,

selecting an IaaS vendor for your app is

arguably the most difficult part of the

process. The next step is actually

deploying your app on the chosen IaaS

provider. Because AWS is the most

popular vendor it will be the main focus

moving forward.

5 best practices for deploying an application on AWS Amazon Web Services is a major

Infrastructure as a Service provider that

provides elastic capacity, quick deploy-

ment and automation for applications

7

without using capital expenditure.

Within AWS are several infrastructure

building blocks, including EC2, S3, and

RDS, to name a few. Click here for a full

list of AWS products for mobile applica-

tion hosting.

AWS may be a compelling choice for

your app’s infrastructure needs, but

remember, if you want more than just

hosting, there are other categories of

*aaS vendors, such as Platform as a

Service, Software as a Service, and

Backend as a Service, that address

different application needs. You may

want to consider vendors in “adjacent”

service categories as well. That said,

below is a list of best practices for

hosting an app on AWS.

Use the right tool for the job. Alex Handy, Senior editor of SD Times,

advises: “Know what your project/appli-

cation is and the problem it solves

before you dig in. Let AWS manage the

infrastructure so you can focus on the

business you do best.” As previously

mentioned, there are several services

within the AWS offering. Alex suggests

combining multiple: “For example, try

Amazon Relational Database Service for

your database, AWS Elastic Beanstalk for

your development environment, or

Amazon Elastic Map Reduce for your

Hadoop cluster and Big Data needs.”

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

Familiarize yourself with these tools

before diving in.

Start small Start by moving a small project to AWS

before your full project is underway.

This way you can fully test and learn

about the various components that

you’ll be using without worrying about

managing an entire project.

Start free.

Consider starting with Amazon’s free tier

to test and become familiar with the

Know what your project/appliccation is

and the problem it solves before you dig in.

“

”platform before jumping into a full

development effort. You get 5GB of

storage free for a year on S3, so you can

easily back something up for free to see

if AWS is the way to go.

Leverage multiple availability zones.If you want your app to be fault-tolerant,

mirroring across availability zones is key

8

for high availability and disaster

recovery. Ensure your design anticipates

and manages component failure to

significantly reduce the chances of it

failing.

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

...always have a backup plan in the event of an

outage

“

”

Design for failure and nothing will failUnderstand Amazon’s disaster recovery

principles, and always have a backup plan

in the event of an outage. Our developers

have compiled multiple guides on recover-

ing from and preparing for an Amazon

outage - take a look at the next segment,

and learn how to be proactive for the sake

of your application.

AWS outage survival guide An outage on your cloud service provider

likely means an outage for your app and

thus a delay in the experience on the

user-end. This can be extremely detrimen-

tal to your app’s ratings and overall usage.

The unfortunate truth is that everyone has

unforeseen outages due to factors out of

our control. If it wasn’t AWS, it would be

the other cloud provider you chose. But

there is still hope: when AWS is down,

you can still be up if you take the proper

measures to prepare for possible

outages in advance.

4 things you didn’t know about AWS

outages

The eastern region of the US is the

most likely area to suffer outages

because ...… it has the oldest data centers.

… it is the largest in terms of data center

footprint.

… it is the default region for most

customers, who don’t bother changing it

because it’s cheaper and/or they aren’t

aware they can change the region.

Availability zones are guaranteed to

be distinct per customer only. However, there are no guarantees as to

the composition of the availability zone

across customers. To illustrate, if you

launch an instance in availability zone 1a

and a different customer launches an

instance in their availability zone 1a, the

two instances are not guaranteed to run

on the same subset of physical

infrastructure. This is why Amazon does

not specify or call out names of availabil-

ity zones in their outage status updates,

because 1a for one person could be 1d

for another.

9

Some outages are worse than others. This is because outages that affect core

infrastructure components such as

Amazon EBS (Elastic Block Store) have a

ripple effect on other AWS products that

utilize these components. For example,

it is possible to provision an EC2

instance using ephemeral storage that is

bound to the instance. This instance

does not utilize EBS and should

theoretically be impacted to a lesser

degree. Let’s say you have an application

that runs on this instance. This applica-

tion also uses a MySQL database and

you’ve opted to use Amazon’s Relational

Database Service (RDS). Now you’re back

to (potentially) being affected by an EBS

outage since RDS uses EBS for storage.

Amazon provides a rich set of API’s

that allow users to access the control

plane for various infrastructure

components.

In fact, a judicious use of these API’s via

client libraries and scripts is what allows

one to be able to do things like launch

new instances based on traffic volume

and / or provision new infrastructure as

necessary. When an outage occurs,

affected customers attempt to remedi-

ate their situation by trying to provision

new servers in other availability zones

and / or regions in order to redirect web

traffic and/or proceed with other tasks.

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

control planes unusable, especially if the

outage affects components that are

used to service the control planes

themselves. This results in customers

being unable to minimize downtime

faced by their application / systems. A

potential solution to this would be to

have a hot / cold standby set of compo-

nents ready in a different region. When

the outage occurs, it would be a matter

of bringing these components online

with the understanding that outages

that span regions are far less likely than

those within a single region.

This means that there’s a fairly large

spike in traffic hitting the control plane

which invariably results in slowdowns

and timeouts. Another side effect of the

outage could also be to render the

SURVIVAL KITAWS

Chapter 3Surviving AWS Failures with a node.js & mongoDB Stack

10

11

In this segment, we’ll explain how to fully

prepare for an AWS EC2 outage with a

node.js and MongoDB stack. Node+Mon-

go on EC2 is a very popular software

stack among web services developers.

There are many user guides on how to

design this system with built-in redun-

dancy so that even coordinated failures

don’t bring down the service. The

absolute minimum for a resilient service

requires a MongoDB replica set behind a

load-balanced node farm.

You are not ready for an EC2 outage until

you have deliberately shut down

components in your system and verified

the expected behavior. As you periodical-

ly do this, you might discover that there

are gaps you did not account for. Take

the following steps to be as prepared as

possible:

1. A Node.js single event will by default

crash on an unhandled exception. Use

upstart or forever to restart the process.

2. Use Monit, an external process on your

server that makes liveness checks and

potentially restarts your service. Monit

will also email you if and when it had to

restart. While upstart ensures that your

process is up, monit ensures that it is

responsive.

3. Your application instances and your

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

MongoDB instances should each be load

balanced across multiple Availability

Zones. The more the better.

4. Place the node servers and the mongo

servers all in a security group, which

allows only the Mongo ports internally

and your application ports externally.

This is trivial to set up and protects your

database from external requests.

5. MongoDB’s authentication provides

additional protection. Mongo’s security

model has limited robustness, but

having authentication in your MongoDB

store is still useful even if the application

and the database are inside an EC2

security group. For your data to get

exposed, you will have to make multiple

mistakes at the same time, which

happens, but the chances are greatly

reduced.

6. Ensure that failover happens smoothly.

Shutdown the primary Mongo instance

and see what happens as requests keep

coming in. The replicas notice the down

primary and one of them takes over, but

upon an incoming request you see this

error message: “unauthorized db:mydb

lock type:-1 client:127.0.0.1”

7. What this error message means is that

the failover happened, but your

application’s request is not authenticat-

12

ed. This is an example of an esoteric bug

that may not show up until you do a full

end to end test. The bug is now in a pull

request. Since pull requests don’t get

released quickly, use npm git dependen-

cies to install your app from your forked

repo.

8. While you have a down MongoDB

instance, it may take a long time for your

application to restart. The default in

node-mongodb-native is “no timeout”,

which means leaving it down to the OS.

To avoid yet another timeout cycle, use

Mongo’s connectTimeoutMS setting.

9. This ensures that your restarts will take

a little over 500ms, if you have a down

instance. However, don’t assume that

your Mongo SDK supports it –

node-mongodb-native doesn’t, unless

you use this github patch.

10. You are now prepared for an AWS

outage. Bring it on.

Migrating from GoDaddy DNS to Amazon Route 53This next guide walks through the

process of migrating from GoDaddy

hosting to Amazon’s Route 53. This only

applies to domain names that are

controlled by another registrar, where

GoDaddy is used just for hosting. If the

domain name itself is controlled through

...know how your appli-cation fails when a net-work service like DNS is

unavailable

“

”

GoDaddy, unfortunately this guide cannot

help - you’ll just have to wait until

GoDaddy is back online.

1. To start, log into the Route 53 dashboard

in the AWS Console.

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

2. As the landing page describes, your first

step is to create a “hosted zone”. A

hosted zone is a concept created by

Amazon to describe a collection of DNS

records (i.e. A records, CNAME records,

MX records) that are managed together

under a single parent domain name. You

can use the Route 53 dashboard to

manage these hosted zones. For a good

explanation of the different types of DNS

records, check out Google’s guide.

3. Click “Create Hosted Zone,” and fill in

your domain name in the form that

appears on the right.

4. When you created the hosted zone for

your domain, Route 53 filled in some

basic DNS records. To see them, select

13

your domain name from the list, and

click “Go to Record Sets” in the top right.

You’ll see that Route 53 populated your

domain with a set of NS records and

SOA records.

5. Your current site may have extra DNS

records needed. For example, if you host

a blog on Tumblr, you probably have a

CNAME record setup to create that link.

Also, if you receive email at your domain,

you probably have an MX record set for

that. Make a list of all these extra

records you’ll need, and add them now

(use the “Create Record Set” button).

6. Finally, it’s time to make the switch.

Head over to your current registar, and

change the nameservers from the

GoDaddy addresses to the ones

provided in the NS record set on your

Route 53 dashboard. Note: because DNS

servers around the world cache this

value, it may take some time to see the

change work while the update propa-

gates through the DNS system.

Recovering from a DNS service outage in AWS using MonitOur final AWS outage survival guide uses

a tool called Monit to recover from a

DNS service outage. A DNS failure isn’t

something you see everyday at AWS, but

when it happens it can cause problems

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

with your application. If your app relies

on DNS to connect to another server (i.e.

database), it may stop trying after a few

failed lookups. At this point things

probably require manual intervention,

even if the DNS service recovers. The key

takeaway from this is that you should

know how your application fails when a

network service like DNS is unavailable.

And you should know whether or not it

will require manual intervention to

recover.

To prevent manual intervention, you can

use a tool called Monit. Monit is a

daemon that is responsible for monitor-

ing your server and application health. It

can also be configured to check the

health of external services such as DNS.

Let’s take a look at a basic config to start:

This config monitors an HTTP application

that listens on 127.0.0.1 on port 9009.

When there is a failure, Monit alerts you

and attempts to restart the process.

We don’t want to constantly restart the

service if it is unhealthy, so if the app

fails 10 times within 10 cycles, then

Monit will leave the app off and stop

14

monitoring it.

When a network service fails like DNS,

your application becomes unhealthy,

tries to restart 10 times, and then

becomes unmonitored. Now you’re at a

point where things require manual

intervention to recover, even if DNS

becomes available. Let’s configure Monit

to take care of everything for us.

Building on our previous configuration:

- We’re still monitoring our HTTP app on

port 9009

- When the application fails now, it kicks

off the aws-dns-healthcheck monitor.

- The aws-dns-healthcheck monitor

checks to see if it can resolve DNS on

172.16.0.23 (the default AWS DNS

server).

- When DNS reports healthy for 3 cycles,

monit execs a script to try to recover the

app.

The example recovery script is simple:

It calls /usr/bin/monit start appsrv1

which will attempt to start the app and

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

re-enables monitoring. It also disables

our aws-dns-healthcheck since we know

DNS is healthy.

You should now be able to recover your

app during a DNS failure. To be 100%

sure this works as intended, we can

simulate another DNS outage using

iptables.

Run this command: sudo iptables -A

OUTPUT -p udp –dport 53 -j DROP

This command will prevent any DNS

queries from completing. If your app

relies on DNS you should see it fail once

its DNS cache has expired. This should

trigger the DNS check to be enabled via

Monit. You can check the status of Monit

monitoring using the command: sudo

monit status

To re-enable DNS run this:

sudo iptables -F

Your application should now try to

recover after the DNS check returns to a

healthy state.

The examples above focus on how to

approach a failure in the DNS service. In

reality there could be a myriad of

external services that may affect the

health of your app. You can use the

same approach as described above and

expand the aws-dns-healthcheck into a

generic health check for your applica-

15

tion. This could include testing network

connectivity, connectivity to external

services (e.g: a database), and any other

processes that your app depends on.

You can find examples for monitoring all

kinds of services on the Monit website

here.

Again, the takeaway from this is to know

how your application behaves under

different failure scenarios. Testing

connectivity loss, loss of network

services, and other failures are very

important when building a high

availability application.

DEVELOPER’S GUIDE TO AMAZON WEB SERVICES

Written by

Kelly Rice and Shubhang Mani

Designed by

Jake McKibben

Survival Guide Authors

Ivan Stoyanov, Dave Wasmer

and Joey Imbasciano

Written by

Kelly Rice and Shubhang Mani

Designed by

Jake McKibben

Survival Guide Authors

Ivan Stoyanov, Dave Wasmer

and Joey Imbasciano