aws re:invent 2016: workshop: aws professional services effective architecting workshop (arc320)
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
November 29, 2016
ARC320
Effective Architecting WorkshopAWS Professional Services
Mark Statham
What to Expect from the Session
• Practical architecture, design and planning session
• Test your skills as an AWS architect, to migrate and transform
• Deliver business outcomes, under time constraints
• Learn new approaches from your peers
• Meet new people and have fun!
Session Timeline
10 mins – The scenario and key considerations
75 mins – Get architecting!
50 mins – Architecture presentations & discussion
10 mins – Our approach
bit.ly/aws-arc320
The Scenario – “ACMEdigi”
• Provider of offline scanned mail in digital format – ”ACMEbox”
• Originally started as an experiment and now increasing in popularity
• Business is under cost pressures – underestimated TCO
• Major changes required within 18 months for service to continue
• CEO wants to explore how AWS can help and requests
• Target architecture, consider cost, availability, and scalability
• Migration plan detailing how to move the service to AWS
• Application optimization roadmap
• DR options available and considerations
IDS
Load
Balancer
App
Cluster #2
App
Cluster #1
App
Cluster #2
App
Cluster #1
Web Server
DNS
Servers
Web Server
DB Cluster
Member #1
DB Cluster
Member #2
HSMNAS
Storage
Virtual IP
Co-location Data Center
DNS HTTPS SFTP
HTTP
NFS DB HTTPS
ALL TRAFFIC
HTTPS
Digitizers
Digitizers
HTTPS
SFTP
Server
SFTP
DR Data Center
App
Cluster #2
App
Cluster #1
Web
Server
Web
Server
Load
Balancer
Firewall
NAS
Storage
HSMDB Cluster
Member #3
Replication
Log Shipping
External
Third PartyUsers
TCP
Firewall
NFS DB HTTPS
Mail Ingestion Center
Scanning devices
bit.ly/aws-arc320
Recommended approach bit.ly/aws-arc320
Determine priority based on the compelling events
Contract expiry
Resource and performance constraints
Cost optimization
Application transformation
Look for quick wins
Leverage services where possible
Rearchitect self-contained application modules
Ask questions—your AWS consultant can help you
Meeting the deadline bit.ly/aws-arc320
Read the case study (if you haven’t already)
Quickly determine what skills you have on your team
Divide and conquer some tasks, form the following sub-teams
Listen to new ideas and approaches
Get something on paper quickly and iterate, iterate, iterate
Focus on action over planning
Save a few minutes to compose your “final solution”
Have fun and make new friends!
Migration Architecture
Migration Planning
Future State Architecture
Project Management
Architecture Concepts bit.ly/aws-arc320
Design for failure and nothing fails
Loose coupling sets you free
Implement elasticity
Build security in every layer
Think parallel
Don’t fear constraints
Leverage different storage options
TIME REMAINING
HOURS MINUTES
Meeting the deadline bit.ly/aws-arc320
Read the case study (if you haven’t already)
Quickly determine what skills you have on your team
Divide and conquer some tasks, consider the following sub-teams
Listen to new ideas and approaches
Get something on paper quickly and iterate, iterate, iterate
Save a few minutes to compose your “final answer”
Have fun and make new friends!
Migration Architecture
Migration Planning
Future State Architecture
Project Management
TIME’S
UP!
VPC Architecture
Single region. Production, minimum two
Availability Zones for high availability.
DR, instantiated via AWS CloudFormation.
Subnet structure
Public: Only subnets routed to the Internet. Contains
Internet-facing resources (ELB, WAF, Proxy…).
Application and Data: Contains application (Apache,
JBOSS) and database, AWS CloudHSM components,
VPC Endpoint to S3. Internal routing only.
Ingestion: Only subnet routed to the internal network.
Contains bastion hosts and digitizers.
public
route table
internal
route table
Ingestion
route table
AWS
Availability Zone 1
Public subnet
Application subnet
Ingestion subnet
Datasubnet
Availability Zone 2
Public subnet
Application subnet
Ingestion subnet
Datasubnet
Virtual Private
Gateway
Internet Gateway
VPC
Production VPC
time
Network – Step 2
VPN replaced by
AWS Direct Connect
for a more consistent
network experience
Two Direct Connect
locations for resiliency
Network – Step 1
VPC architecture
implemented
Connected with
ingestion network
using VPN
T0App – Step 1
“Lift and shift” with some
re-platforming quick-wins
(Amazon RDS, Amazon
Route 53…)
Data Access Service module
redeveloped to replace NFS by
Amazon S3
Web static content served by
Amazon CloudFront/S3
IDS replaced by a WAF solution
available in AWS Marketplace
T0 + 7 months
Users
DNS HTTPS
HTTPS
SFTP
server
External
Third Party
DB
EFS file
system
CloudFront
distributionStatic content
S3 bucket
Uploads and
SFTP config
Application
Load Balancer
Web Application
Firewall instances
HTTP
App Cluster
1 & 2 instances
Web Server instances
EC2 instance with
EC2 Auto Recovery
Auto Scaling group
with scaling policies
Route 53
hosted zone
AWS
Registration
Login
Payment
Doc Manager
Presentation
Core
Batch Processing
Encryption
Administration
Digitizer
Data Access Service
Modules
redeveloped
Step 1
Amazon S3 +
Lifecycle
Configuration
Digital
documents
Oracle DB
instance
Standby
instance
CloudHSM 1 CloudHSM 2
HTTPS
Bastion host
Administrators
Scanning devices
Digitizers
TCP
SFTP
Egress proxy
instances
Internet
Migration highlights
On-premise
Step 1
Servers: AWS Server Migration Service (SMS) if the source is
VMware. Otherwise 3rd-party solution (Racemi, CloudEndure…)
Databases: Migration to RDS Oracle EE with AWS Database
Migration Service (DMS)
Digital documents: Initial upload using Snowball appliance twice.
Delta sync with 3rd-party solution (ExpeDat, Atunity, Aspera…)
Encryption keys: SafeNet key replication to CloudHSM
DR Consideration: AWS service features meet customer
requirements and risk profile, e.g. multi-AZ design vs. physical DC
DR Recovery: infrastructure automation to recover within defined
RTO, use RDS point in time recovery, EC2 snapshots to meet RPO
time
Network – Step 2
VPN replaced by
Direct Connect for a
more consistent
network experience
Two Direct Connect
locations for resiliency
Network – Step 1
VPC architecture
implemented
Connected with
ingestion network
using VPN
T0App – Step 2
Auto Scaling for automated resiliency and scalability, caching for session handling
Encryption module redeveloped to use AWS KMS instead of CloudHSM for data and file encryption
Micro-services developed for registration and login
Use of serverless services (Amazon API Gateway, AWS Lambda, Amazon DynamoDB…)
Oracle database migrated to Amazon Aurora
T0 + 12 monthsApp – Step 1
“Lift and shift” with some
re-platforming quick-wins (RDS,
Route 53…)
Data Access Service module
redeveloped to replace NFS by
S3
Web static content served by
CloudFront/S3
IDS replaced by a WAF solution
available in AWS Marketplace
Users
DNS HTTPS
HTTPS
SFTP
server
External
Third Party
DB
EFS file
system
CloudFront
distributionStatic content
S3 bucket
Uploads and
SFTP config
Application
Load Balancer
Web Application
Firewall instances
HTTP
App Cluster
1 & 2 instances
Web Server instances
EC2 instance with
EC2 Auto Recovery
Auto Scaling group
with scaling policies
Route 53
hosted zone
AWS
Registration
Login
Payment
Doc Manager
Presentation
Core
Batch Processing
Encryption
Administration
Digitizer
Data Access Service
Modules
redeveloped
Step 1
Amazon S3 +
Lifecycle
Configuration
Digital
documents
Oracle DB
instance
Standby
instance
CloudHSM 1 CloudHSM 2
HTTPS
Bastion host
Administrators
Scanning devices
Digitizers
TCP
SFTP
Egress proxy
instances
Internet
App Cluster 1
instances
App Cluster 2
instances
HTTPS
Digitizers
Scanning devicesTCP
Lambda
function
KMS
ElastiCache
Redis Multi-AZ
Path-based routing
HTTP
Web Application
Firewall instances
AWS
EC2 instance with
EC2 Auto Recovery
Auto Scaling group
with scaling policies
Auto Scaling group
with fixed capacity
Registration
Login
Payment
Doc Manager
Presentation
Core
Batch Processing
Encryption
Administration
Digitizer
Data Access Service
Modules
redeveloped
Step 2
CloudHSM 1 CloudHSM 2
Amazon S3 +
Lifecycle
Configuration
KMS
Digital
documents
Registration
service
Table
Users
Login
service
Table
Sessions
External
Third Party
Administrators
Bastion host
Aurora DB
instance
Standby
instance
DB
S3 pre-signed
upload URL
Static content
Egress proxy
instances
InternetUsers
DNS HTTPS
Route 53
hosted zone
CloudFront
distribution
Application
Load Balancer
S3 bucket
time
Network – Step 2
VPN replaced by
Direct Connect for a
more consistent
network experience
Two Direct Connect
locations for resiliencyT0 + 17 months
Network – Step 1
VPC architecture
implemented
Connected with
ingestion network
using VPN
T0App – Step 2
Auto Scaling for automated
resiliency and scalability, caching
for session handling
Encryption module redeveloped
to use KMS instead of CloudHSM
for data and file encryption
Oracle database migrated to
AWS Aurora
Micro-services developed for
registration and login
Use of serverless services
(API Gateway, Lambda,
DynamoDB…)
App – Step 3
AWS API Gateway and AWS
WAF implemented
Micro-services developed for
payment and doc manager
Administration, presentation, and
core modules migrated to use
AWS Elastic Beanstalk web
environment
Batch processing module
redeveloped for a more real-time
delivery using Lambda and
Elastic Beanstalk worker
environment
App – Step 1
“Lift and shift” with some
re-platforming quick-wins (RDS,
Route 53…)
Data Access Service module
redeveloped to replace NFS by
S3
Web static content served by
CloudFront / S3
IDS replaced by a WAF solution
available in AWS Marketplace
App Cluster 1
instances
App Cluster 2
instances
Digitizers
Scanning devicesTCP
Lambda
function
KMS
ElastiCache
Redis Multi-AZ
Path-based routing
HTTP
Web Application
Firewall instances
AWS
EC2 instance with
EC2 Auto Recovery
Auto Scaling group
with scaling policies
Auto Scaling group
with fixed capacity
Registration
Login
Payment
Doc Manager
Presentation
Core
Batch Processing
Encryption
Administration
Digitizer
Data Access Service
Modules
redeveloped
Step 2
Amazon S3 +
Lifecycle
Configuration
KMS
Digital
documents
Registration
service
Table
Users
Login
service
Table
Sessions
External
Third Party
Administrators
Bastion host
Aurora DB
instance
Standby
instance
DB
S3 pre-signed
upload URL
Static content
Egress proxy
instances
InternetUsers
DNS HTTPS
Route 53
hosted zone
CloudFront
distribution
Application
Load Balancer
S3 bucket
Doc Manager
service
Table
Document
Administration
service
Web Environment
Doc Processing
service
Worker Environment
Presentation & Core
service
Web Environment
AWS WAF
Step 3
Private service (API
authentication required)Public service
Elastic Beanstalk
container
Host-based
IPS solution
CloudFront
distribution
Payment
service
Table
Subscription
API gateway
Egress proxy
instances
Internet
Registration
Login
Payment
Doc Manager
Presentation
Core
Batch Processing
Encryption
Administration
Digitizer
Data Access Service
Modules
redeveloped
Amazon S3 +
Lifecycle
Configuration
KMS
Digital
documents
ElastiCache
Redis Multi-AZ
Aurora DB
instance
Standby
instance
Registration
service
Table
Users
Login
service
Table
Sessions
Lambda
function
External
Third Party
S3 pre-signed
upload URL
Digitizers
Scanning devicesTCP
Administrators
Bastion host
Users
DNS HTTPS
Route 53
hosted zone
S3 bucket
AWS