VPC - Flying Blind on a Rocket Cycle

Matthew Boeckman - VP of DevOps at Craftsy.com

@matthewboeckman

http://enginerds.craftsy.com

Who is Craftsy

● Instructor led training videos for passionate hobbyists

● #19 on Forbes’ Most Promising Companies 2014

Three things

Keep it simple

Get there now

Be flexible

*disclaimer

VPC - Why

VPC is mature network

topology for AWS

VPC - Why

Network ACL’s allow for true

edge blocking

VPC - Why

Instances can be members of

multiple Security Groups

SG membership can change

post-instance launch

VPN connectivity enables

extension of your corporate or

facilities network to AWS

VPC - Why

Our stack in ec2-classic

What we hate about ec2-classic

● inflexible security groups

● per-IP maintenance of SG’s across regions

● ALLOW TCP 22 FROM 0.0.0.0/0

● no edge

● no edge

● no edge

● no edge

Our stack in VPC

VPC - flying blind

netcat, tcpdump and patience

be the packet

host a

host b

SG

SG

ACL

ACL

outout,in

out,in

out,in

in

out

out,in

out,in

out,inin

flexible security groups!

Office tomcat SG - applies to all tomcat instances

environment specific tomcat SG - only applied for a

specific environment

LIMITS

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Appendix_Limits.html

LIMITS

100 SG’s per VPC

50 Rules per SG (including inherited rules by

referencing other SG’s)

5 SG’s per network interface (VPC hosts can have

multiple network interfaces)

HARD LIMIT 250 rules per interface

40 ACL’s per subnet (in + out <= 40)

routing

Private subnets can only route traffic destined for the internet to a

NAT instance (eni-0…). Public subnets route to the IGW. Routes

can be automatically propagated from VPN connections.

NAT instances

HOW BIG?!

● we chose m1-medium… because…. it seems big enough?

sure.

● we were wrong

● upgrading NAT instances - easy, anytime, no interruption

(m3.xl)

Connecting to your HQ with a VPN

● AWS docs on this are perfect - check if your firewall is on the supported

list. If so, one click configuration for your firewall

● A VPN connection - includes two tunnels, connected to two different IP’s

at VPC. THESE UNDERGO MAINTENANCE - PRACTICE FAILOVER

Cross region VPN

http://aws.amazon.com/articles/5472675506466066

http://fortycloud.com/interconnecting-two-aws-vpc-regions/

AWS has no product offering here. You can easily VPN two VPC’s in

the same region but not, you know, in different regions.

Tunnel MTU size is a thing

iptables -t mangle -A FORWARD -o eth0 -p tcp -m tcp --tcp-flags

SYN,RST SYN -m tcpmss --mss 1361:1536 -j TCPMSS --set-mss

1360

seriously?

ACL’s ARE NOT STATEFUL

ALLOW tcp 80 src 10.85.0.0/16

ALLOW tcp 443 src 10.85.1.0/24

ALLOW tcp established any

DENY ALL

SNS, Redshift, Route53, RDS

SNS - has no legs in VPC. Systems subscribing to SNS topics from private

subnets need an HTTP proxy in a public subnet for SNS to reach them.

Redshift/RDS- has legs in VPC - migrate your redshift or rds instances to

VPC (yay!)

Route53 - no support for “views” in VPC.

reservations!

Instance reservations purchased in EC2 classic DO NOT

MAGICALLY MOVE TO VPC

Do. Not. Forget. This. Step.

migration time best time

- use AWS support or account teams

- start with subnets and basic nat, vpn

- dev environments, soak

- preprod, soak

cloned production

shut it down

thank you

QUESTIONS!

Matthew Boeckman

@matthewboeckman

thank you

QUESTIONS!

Matthew Boeckman

@matthewboeckman

http://enginerds.craftsy.com

(deck will be there)

thank you

QUESTIONS!

Matthew Boeckman

@matthewboeckman

http://enginerds.craftsy.com

(deck will be there)

public subnet routing

In VPC you define a route table, and associate it with subnet(s). This

example sends all traffic for the defined CIDR block to local, and all

other traffic (0.0.0.0/0) to the internet gateway (igw). Traffic can

also be routed to Elastic Network Interfaces (eni) or VPN gateways

(vgw)

Lasciate ogne speranza, voi ch'intrate

tcp4 0 0 192.168.1.24.59021 www.pandora.com.http

ESTABLISHED

TCP connections originate on the sending side from ports 1025-

65535, they arrive at the port you specified (http 80, https 443, dns

53, etc). Return to your request comes back from that socket with

some exceptions (ftp).

ACL’s inbound/outbound are the inverse of your thinking. OUT from

your ec2 instance is IN to the ACL, then OUT to the destination host.

Destination hosts’ reply is IN to the ACL (yep!) and OUT to your ec2

instance.

My advice? Punt. Leave ACL’s as ALLOW ALL, and add DENY

statements against specific CIDR blocks or IP addys. Security Groups

are stateful. Use them.