cloud first: new architecture for new infrastructure
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
KJ Wu (吳貴融), Solutions Architect, AWS
Leo Wang, Staff Engineer, Trend Micro
2016/05/20
Cloud First: New Architecture for
New InfrastructureCustomer Sharing: Migration Story of Trend Micro File
Reputation Service
Agenda
• What is your Cloud Journey?
• Cloud well-architected patterns
• Customer Sharing
• Migration Story of Trend Micro File Reputation Service
The New Normal
Cloud First
Object Oriented
Programming
Client Server
Architecture
Service-oriented
Architecture
(SOA)
The journey we’re seeing with AWS customers
Dev & Test True Production Mission Critical All-in
Build production apps
Migrate production apps
Marketing
Build mission-critical apps
Migrate mission-critical apps
Development and
test environments
Corporate standard
This is not your focus
And focus on your core mission
Lower the time spent
on infrastructure
Dedicate more
resources to
innovation
Concentrate on
new business
initiatives
“Our goal is to move at the speed of business. Our customers’ needs change
constantly, and we need to be able to adapt to that.”
Keith Homewood – Cloud Product Owner, Nordstrom
Pillars of Well-Architected Framework in Cloud
Security Reliability Performance
Efficiency
Cost
Optimization
Security
The ability to protect information, systems, and assets
while delivering business value through risk assessments
and mitigation strategies.
• Data protection
• Privilege management
• Infrastructure protection
• Detective controls
Well-Architected Cloud Patterns
Build Security into every layer
• Encrypt data in transit and
rest between application tiers
• Enforce principle of least
privilege across every service
• Protect your critical resources
from application-layer and
network attacks
Amazon
VPC
AWS Direct
Connect
AWS IAM AWS KMSAWS WAF
AWS
CloudTrail
Reliability
The ability of a system to recover from infrastructure or
service failures, dynamically acquire computing resources
to meet demand, and mitigate disruptions such as
misconfigurations or transient network issues.
• Foundations
• Change management
• Failure management
Well-Architected Cloud Patterns
Design for failure and nothing will fail
App
Server
Database
Server
• Applications should continue to
function even if the underlying
application component fails,
communication is lost or physical
hardware fails, is
removed/replaced.
• Auto-healing & Monitoring
It also means High Availability
Amazon
RDSRDS DB
instance
read
replica
Amazon
CloudWatch
Multi-AZ
Well-Architected Cloud Patterns
Leverage global infrastructure
(Multi-AZ and Multi-Region)
Web Instance
RDS DB Instance
Active (Multi-AZ)
Availability Zone #1 Availability Zone #2
Web Instance
RDS DB Instance
Standby (Multi-AZ)
ELB
Balancer
User
Amazon
Route 53
Oregon Region
Tokyo Region
Well-Architected Cloud Patterns
Loose coupling sets you free
• Design architectures with
independent components
• Design every component as a
black box
• Load balance clusters
Amazon
SQSElastic Load
Balancing
Well-Architected Cloud Patterns
Become Stateless
• Don’t store state in server
• Leverage services to hold
state information
• Application functions
regardless of which
application node processes
the request
Amazon
DynamoDB
Amazon
ElastiCache
Memcached
Redis
Performance Efficiency
The ability to use computing resources efficiently to meet
system requirements, and to maintain that efficiency as
demand changes and technologies evolve.
• Compute
• Storage
• Network
Well-Architected Cloud Patterns
Embrace Elasticity & Automate
• Auto Scaling sets you free on
guessing your application
capacity need
• Automate installation and
configuration of environment
• Automate with CI/CD toolsAuto Scaling
AWS
CodeCommit
AWS
CodeDeploy
AWS
CodePipelineAWS
OpsWorks
Well-Architected Cloud Patterns
Leverage different storage options
• RDBMS only? Do you need
data warehouse?
• Key-value data only?
• Need a scalable object
storage?
• Read/write ratio of your data
request?
Amazon
Glacier
Amazon
S3
Amazon
DynamoDB
Amazon
ElastiCacheAmazon
RDS
Well-Architected Cloud Patterns
Think Parallel
Hour 1 Hour 2 Hour 3 Hour 4
• One Server working for Four
hours costs the same as Four
servers working for One hour
• Combine with elasticity to
increase capacity when you
need it most
• MicroServices architecture
Amazon ECS Amazon ECR
Cost Optimization
The ability to avoid or eliminate unneeded cost or
suboptimal resources.
• Matched supply and demand
• Cost-effective resources
• Expenditure awareness
• Optimizing over time
Cost Optimization – Common Practices on AWS
Monitoring to
collect and
track metrics
Consolidated
Billing &
Detailed
Billing Reports
Tag your
resources
Design
Architecture
with the right
services
On-Demand,
Reserved and
Spot Instances
Cloud Architecture KEEPS Innovating….
Serverless Architecture
No servers by managed services
SOA design & stateless function
Event-driven
Integration with the third
party services
Shift focus to Richer user experience
The real example of Serverless architecture
My Customer Service
Motivation for listening to
social media
Twitter API Amazon
Kinesis
AWS
Lambda
Amazon
Machine Learning
Amazon
SNS
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Leo Wang – FRS Infra.
Staff Engineer
5/26/2016
Trend Micro FRS Infra.
Migration Experience Sharing
Agenda
• Introcution
• Why AWS?
• Migration Experience
• Data Migration Experience
• Application Migration Experience
Enabling a Smart Protection
Strategy
Inspired to Stay a Step Ahead
Global Threat Intelligence from
the Smart Protection Network
Backed by Global Research
and Support
Fast Facts
Founded: 1988, United States
Headquarters: Tokyo, Japan
Number of Employees: 5,258
File Reputation
Files
Process
Solution
Personal Profile
• Leo Wang
• Trend Micro File Reputation Service Team
• Staff Engineer
WHY AWS?
Data Grows and Grows
0
200
400
600
800
1000
1200
1400
Total Storage Needed
2017 Q1
2019 Q4
HA?
DR?
Stable?
Easy Extend?
Data Temperature?
200%
Growth
We Need Scaling
0
500000
1000000
1500000
2000000
2500000
Daily Sourcing Samples
Max
Capacity
We Need Flexibility
Request / Time Effort On-Premise AWS
A Machine 10~30 Days 5 Minutes
PoC Environment 1~3 Months 10 Minutes
DR Site 6~13 Months 30 Minutes
DATA
MIGRATION
EXPERIENCE
AWS cloudcorporate data center
File
Metadata
AWS Data Migration Solution
DO BY OUR OWN WAY
Amazon
S3
Amazon
Glacier
AWS Import/
Export Snowball
import/
export
AWS Storage
Gateway
Amazon
RDSAWS Database
Migration Service
Service to Service
corporate data center
File
Metadata
Storage Service on top of AWS
S3
Glacier
Dynamo
Hot File
Storage
Cold File
Storage
Metadata
Database
Common
Storage
Webservice
Availability Zone A
Availability Zone B
Internet
Gateway
ELB
Service to Service
Common
Storage
Webservicecorporate data center
corporate data center
APP / Service
On AWS
File
Metadata
APPLICATION
MIGRATION
EXPERIENCE
Fully Leverage Cloud
Highly Scalability
Highly Flexibility
Re-Design
And
Re-Write
EVERYTHING!
On-Premise Design
Analytic
Service
corporate data center
Worker
Worker
Worker
Limited
Queue
Size
Limited
Number
of
Machines
Throttling
to Prevent
Overloaded
Limited
Machine
Spec
Flexible and Scalable
Analytic
Service
corporate data center
Worker
Worker
Worker
Throttling
to Prevent
Exceed
Budget
Plan
Dynamic
Machine
Spec
Unlimited
Queue
Size
Auto-
Scaling
RD’s Choice
Analytic
Service
corporate data center
Worker
Worker
Worker
Machine Spec
# of
Machines
# of Clusters
Buffer
Size
Size of Historical
Data
HOW TO MAKE DECISION?
COST CONSCIOUS DESIGN
On-Premise
Server $500 / monthRack+Power $130 / monthNetwork $100 / month * Mbps
AWS
All Prod STG,DEV, DR EC2 RDS DataTransfer S3 ElastiCache Glacier
System A6,000 5,000 1,000 3,000 2500 100 - 400 -
System B7,000 6,500 500 2,000 - 500 1500 - 3,000
System C1,000 400 600 900 50 10 - 40 -
View of Cost
Analytic
Service
corporate data center
Worker
Worker
Worker
$3000
$30
$10
$600
2M
Requests
Change of Mindset
RD’s View:
• Every resource’s cost is clear
• Cost defines design
• Manager’s View:
– Every system’s/request’s cost is clear
– Cost defines business scope
Experiences from
• Enterprise Session
• 1:00 – 1:40pm
FRS Migration Experience
• Deep Drive on AWS session
• 3:30 – 4:10pm
DevOps at FRS TrendMicro
• IoT & Big Data session
• 3:30 – 4:10pm
Analytic Engine - A common Big Data computation service on the AWS
Thank You