aws re:invent recap 2016 taiwan part 2
TRANSCRIPT
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Olivier Klein 奧樂凱, Emerging Technologies Solutions Architect, Asia-Pacific
Activate your Cloud Superpowers!
We can now Architect to be Serverless
Fully Managed
• No provisioning
• Zero Administration
• High-Availability
Developer Productivity
• Focus on the code
• Innovate rapidly
• Reduce time to market
Scalability
• Automatically
• Scale up and down
Many Serverless Options on AWS
Storage
Database
Compute
Messaging and Queues
Gateways
User Management
Internet of Things
Machine Learning
Streaming Analytics
Search
AWS Lambda
Run your code without thinking about
servers. Pay only for compute time
you consume.
Triggered through API calls or state
changes in your AWS environment
Scales automatically to match the
incoming event rate
Charged per 100ms execution time
AWS Lambda
Amazon S3 Amazon DynamoDB
Amazon Kinesis
AWS CloudFormation
AWS CloudTrail
Amazon CloudWatch
Amazon SNSAmazonSES
AmazonAPI Gateway
Amazon Cognito
AWSIoT
AmazonAlexa
Cron events
DATA STORES ENDPOINTS
REPOSITORIES EVENT/MESSAGE SERVICES
Event Sources that integrate with AWS Lambda
… and the list continues to grow.
Amazon Config
AWS Serverless Application Model (“SAM”)
Common language to describe
contents of a serverless app via YAML
Uses CloudFormation to provision
resources and deploy the app
Supports anything CloudFormation
supports
Open specification (Apache 2.0)
SAM Template
Resources: GetHighscoresFunction:Type: AWS::Serverless::FunctionProperties:CodeUri: s3://olivierk-deliverables/aws-jumpy-fish.zipHandler: index.handlerRuntime: nodejs4.3Policies: AmazonDynamoDBReadOnlyAccessEvents:GetHighscores:Type: ApiProperties:Path: /getHighscoresMethod: ANY
HighscoreTable:Type: AWS::Serverless::SimpleTable
Serverless Game Demo
Amazon
LambdaAmazon API
Gateway
Amazon S3Amazon
CloudFront
Browser
Amazon
DynamoDB
Development Is Changing
Reduce Risk Smaller,
targeted
applications
Deliver faster Reactive to
customer
needs
More
experimental
S o u r c e
AWS CodeCommit
B u i l d
?
S t a g i n gP r e - p r o d u c t i o n
P r o d u c t i o n
AWS CodeDeploy
A W S C o d e P i p e l i n e
Advantages of CI/CD
Smaller changes,
less risk and costAutomated execution,
increased reliability and
scalability
Less features, more
reliable updatesFewer lines of code,
improved security
AWS CodeBuildB u i l d s e r v i c e f o r c o m p i l i n g s o u r c e c o d e
a n d r u n u n i t t e s t s
AWS OpsWorks for Chef Automate
Fully managed Chef Server
Amazon EC2 Systems Manager
Collection of AWS tools for package installation,
patching, resource configuration and task automation
Other Services to help with automation
M e t h o d C a l l F u n c t i o n C h a i n i n g D a t a b a s e Q u e u e s
Different Ways to Coordinate Functions
AWS Step FunctionsC o o r d i n a t e t h e c o m p o n e n t s o f d i s t r i b u t e d
a p p l i c a t i o n s u s i n g v i s u a l w o r k f l o w s
Round-trip latency
Intermittent connectivity
Expensive bandwidth
Programming and updating embedded software needs specialized skills
Limited to what is on the device unless you rewrite or program the device
Challenges of Devices Living on the Edge
Built into devices
at manufacture
AWS GreengrassEmbedded Lambda compute,
messaging & data caching in
connected devices
Install the
Greengrass runtimeLambda functions on
AWS & Devices
Manage from
AWS console
Same programming
model
Local communication
and orchestration
Amazon QuicksightFast, cloud-powered BI service that makes
it easy to build visualizations, perform ad-
hoc analysis, and get insights from data.
Get started within
minutes through your
browser
Fast results with in-
memory calculation
engine (SPICE)
1/10th of the cost of
traditional BI tools –
pay as you go per user
Decouple Storage and Compute
Traditionally analytical workloads
required large databases or data
warehouses, with storage and
compute close to each other
Big Data benefits from decoupling
storage and compute
Amazon S3 offers virtually unlimited
storage at a per GB/month rate
No need to
move data
Query S3 directly
& right away
No infrastructure to
setup & manage
Fast results
within seconds
Pay for just the
queries you run
Amazon AthenaInteractive query service that makes it
easy to analyze data in Amazon S3
using standard SQL
Athena & Quicksight Demo
Amazon
S3
Amazon
Athena
Amazon
Quicksight
Analyze past flight performance data stored in S3
Bureau of Transportation Flight Data Statistics
www.transtats.bts.gov
Create visualizations from S3 with Athena & Quicksight
S o u r c e s M o d e l s A p p l i c a t i o n s Q u e r i e sU s e r s P r o c e s s i n g
The Modern Data Architecture Is Agile
2 . S o u r c e D a t a
S 3 U p l o a d
K i n e s i s F i r e h o s e
D y n a m o D B S t r e a m s
S n o w b a l l
S n o w b a l l E d g e
S n o w m o b i l e
3 . L i f e c y c l e
m a n a g e m e n t
a n d c o l d s t o r a g e
5 . D a t a
g o v e r n a n c e ,
s e c u r i t y ,
p r i v a c y
Anal yt ics
D a t a b a s e
M i g r a t i o n
S e r v i c e
1 . I n g e s t i o n
D a t a s t o r e t a r g e t
4 .
M e t a d a t a
c a p t u r e
6 . S e l f - s e r v i c e
d i s c o v e r y , s e a r c h ,
a c c e s s
7 .
M a n a g i n g
d a t a
q u a l i t y
S 3
E F S
D y n a m o D B
R D S
E B S
8 . P r e p a r i n g f o r
A n a l y t i c s
9 .
O r c h e s t r a t i o n
a n d j o b
s c h e d u l i n g
1 0 .
C a p t u r i n g
d a t a
c h a n g e s
G l a c i e r E M R
A t h e n a
E M R
E l a s t i c S e a r c h
R e d s h i f t
A I
M a c h i n e L e a r n i n g
Q u i c k s i g h t
?
The Modern Data Architecture on AWS
2 . S o u r c e D a t a
S 3 U p l o a d
K i n e s i s F i r e h o s e
D y n a m o D B S t r e a m s
S n o w b a l l
S n o w b a l l E d g e
S n o w m o b i l e
3 . L i f e c y c l e
m a n a g e m e n t
a n d c o l d s t o r a g e
5 . D a t a
g o v e r n a n c e ,
s e c u r i t y ,
p r i v a c y
Anal yt ics
D a t a b a s e
M i g r a t i o n
S e r v i c e
1 . I n g e s t i o n
D a t a s t o r e t a r g e t
4 .
M e t a d a t a
c a p t u r e
6 . S e l f - s e r v i c e
d i s c o v e r y , s e a r c h ,
a c c e s s
7 .
M a n a g i n g
d a t a
q u a l i t y
A W
S
G l u
e
S 3
E F S
D y n a m o D B
R D S
E B S
8 . P r e p a r i n g f o r
A n a l y t i c s
9 .
O r c h e s t r a t i o n
a n d j o b
s c h e d u l i n g
1 0 .
C a p t u r i n g
d a t a
c h a n g e s
G l a c i e r E M R
A t h e n a
E M R
E l a s t i c S e a r c h
R e d s h i f t
A I
M a c h i n e L e a r n i n g
Q u i c k s i g h t
The Modern Data Architecture on AWS
AWS GlueEasily understand your data sources,
prepare the data, and load it reliably to
data stores and your analytics pipeline
Integrated with:
S3, RDS, Redshift & any JDBC-
compliant data store
1GiB
GPU Memory
2 GiB
4 GiB
8 GiB
Current
Generation
EC2
Instance
Elastic GPUs: GPU Acceleration on-demand
The Power of Speech: Alexa
Alexa, the voice service that powers
Echo, provides capabilities, or skills,
that enable customers to interact with
devices using voice
Alexa Skills Kit (ASK) allows everyone
to build and publish their own skills
Skills can be powered by AWS
Lambda
Unlimited
Replays
Returns an MP3
or audio stream
Lightning Fast
Response
Fully Managed and
Low Cost
Amazon PollyTurn text into lifelike speech using deep
learning technologies to synthesize
speech that sounds like a human voice
Amazon Polly
“The temperature
in WA is 75°F”
“The temperature
in Washington is 75 degrees
Fahrenheit”
Amazon Polly: Text In, Life-like Speech Out
Amazon LexConversational interfaces for your
applications, powered by the same
Natural Language Understanding
(NLU) & Automatic Speech Recognition
(ASR) models as Alexa
Integrated
development in
AWS console
Trigger AWS
Lambda
functions
Multi-step
conversations
Continually improving
ASR & NLU models
Enterprise
connectorsFully Managed
Intents
A particular goal that the
user wants to achieve
Utterances
Spoken or typed phrases
that invoke your intent
Slots
Data the user must provide to fulfill the
intent
Prompts
Questions that ask the user to input
data
Fulfillment
The business logic required to fulfill the
user’s intent
BookHotel
Amazon RekognitionImage Recognitions and Analysis
powered by Deep Learning which
allows to search, verify and organize
millions of images
Easy to use Batch Analysis Real-time
Analysis
Continually Improving Low Cost
Demographic Data
Facial Landmarks
Sentiment Expressed
Image Quality
Brightness: 25.84
Sharpness: 160
General Attributes
Serverless Rekognition Demo
Serverless website that uses Rekognition to identify
faces and classify pictures
Amazon S3
AWS Lambda
Amazon API
Gateway
Amazon
DynamoDB
Amazon
Rekognition
Mobile
CodeFor.Cloud/image
AWS ArtifactP o r t a l f o r o n - d e m a n d a c c e s s t o
AW S c o m p l i a n c e r e p o r t s
BROADEST SERVICES TO SECURE APPLICATIONS
NETWORKING
VIRTUAL
PRIVATE
CLOUD
WEB
APPLICATION
FIREWALL
IDENTITY
IAM ACTIVE
DIRECTORY
INTEGRATION
SAML
FEDERATION
BROADEST SERVICES TO SECURE APPLICATIONS
NETWORKING
VIRTUAL
PRIVATE
CLOUD
WEB
APPLICATION
FIREWALL
ENCRYPTION
KEY
MANAGE-
MENT
SERVICE
CLOUDHSM SERVER-
SIDE
ENCRYPTION
ENCRYPTION
SDK
IDENTITY
IAM ACTIVE
DIRECTORY
INTEGRATION
SAML
FEDERATION
BROADEST SERVICES TO SECURE APPLICATIONS
COMPLIANCE
CONFIGCLOUD
TRAIL
SERVICE
CATALOG
CONFIG
RULESINSPECTOR
NETWORKING
VIRTUAL
PRIVATE
CLOUD
WEB
APPLICATION
FIREWALL
ENCRYPTION
KEY
MANAGE-
MENT
SERVICE
CLOUDHSM SERVER-
SIDE
ENCRYPTION
ENCRYPTION
SDK
IDENTITY
IAM ACTIVE
DIRECTORY
INTEGRATION
SAML
FEDERATION
AWS Shield for EveryoneWeb applications running on AWS are
already protected by Shield Standard -
no action is required
Protection from volumetric and state
exhaustion attacks
Enabled by default for everyone!
AWS Shield AdvancedFor additional protection against very
large and sophisticated attacks
Advanced
notifications via
Cloudwatch
Cost protection on
ELB, CloudFront,
Route 53
24/7 DDoS
response team
and support
WAF included at
no additional
cost
AWS X-RayAnalyze and debug distributed
applications in production
fanout-00002
hello-1.mbfzqxzcpe.us-
east-..
hello-2.mbfzqxzcpe.us-
east-..fanout-00005
fanout-00003
throttleDynamoDB
indexDynamoDB
fanout-00004
400 traces
0.30 ms
600 traces
0.19 ms
1000 traces
0.13 ms400 traces
0.30 ms
1000 traces
0.28 ms
400 traces
0.30 ms
850 traces
0.16 ms
850 traces
0.17 ms
C l i e n t
C l i e n t
800 traces
0.19 ms
fanout-00001
560 traces
0.19 ms
fanout-00006
A W S O p s W o r k s
F o r
C h e f A u t o m a t e
A m a z o n E C 2 S y s t e m s
M a n a g e r
A W S
C o d e B u i l d
A W S X -
R a y
A W S P e r s o n a l
H e a l t h
D a s h b o a r d
A W S
S h i e l d
A m a z o n
P i n p o i n t
A W S
G l u e
A W S
B a t c h
C # I n A W S
L a m b d a
A W S
L a m b d a @ E d g e
A W S S t e p
F u n c t i o n s
E l a s t i c G P U s F o r
E C 2
A m a z o n
L i g h t s a i l
F 1 I n s t a n c e s
A m a z o n A t h e n a
A m a z o n A I
A m a z o n
R e k o g n i t i o n
A m a z o n
P o l l y
A m a z o n
L e x
P o s t g r e S Q L F o r
A u r o r a
A W S
G r e e n g r a s s
A W S S n o w b a l l
E d g e
A W S
S n o w m o b i l e