(arc342) designing & building an end-to-end email solution using aws
TRANSCRIPT
![Page 1: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/1.jpg)
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Mahae Koh, Amazon SES
Morgan Thomas, Amazon SES
October 2015
Closing the LoopBuilding Email Infrastructure and Applications with AWS
ARC342
![Page 2: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/2.jpg)
The State and Scale of Email
- Email is not dead
- The Internet sends and receives 2 million emails per
second
![Page 3: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/3.jpg)
The State and Scale of Email
- Classic email architectures require
- Postmasters
- Deliverability managers
- System administrators
- Engineers
A typical 1000-user organization will spend upwards of
$1.8 million a year just on managing spam
Source: Radicati Group
![Page 4: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/4.jpg)
Postmasters
- Responsible for keeping inboxes clear of spam
- Configure DNS-based sender reputation systems
- Process complaints to identify false negatives
- Analyze senders to identify spammers and botnets
- Maintain sender reputation metrics
![Page 5: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/5.jpg)
Postmasters
- Responsible for keeping inboxes clear of spam
- Configure DNS-based sender reputation systems
- Process complaints to identify false negatives
- Analyze senders to identify spammers and botnets
- Maintain sender reputation metrics
![Page 6: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/6.jpg)
Deliverability Managers
- Ensure that marketing campaigns reach widest audience
possible
- Target campaigns based on user preferences
- Maintain high sender reputation
- Enforce compliance with email security standards
- Sender Policy Framework (SPF)
- DomainKeys Identified Mail (DKIM)
![Page 7: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/7.jpg)
Deliverability Managers
- Ensure that marketing campaigns reach widest audience
possible
- Target campaigns based on user preferences
- Maintain high sender reputation
- Enforce compliance with email security standards
- Sender Policy Framework (SPF)
- DomainKeys Identified Mail (DKIM)
![Page 8: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/8.jpg)
System Administrators and Developers
- Ensure that your email infrastructure runs smoothly
- Email firewall devices
- Security appliances
- Mailbox servers
- Build and manage applications
- Campaign management
- Customer support
- Archival
![Page 9: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/9.jpg)
The State and Scale of Email
- Every business must navigate the email landscape to
communicate with customers
- Maintaining your position in the ecosystem as a sender
and receiver is critical
![Page 10: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/10.jpg)
Email Architecture with AWS
![Page 11: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/11.jpg)
Classic Email Architecture
Internet FirewallAppliances
Content Scanners
MailboxServers
Consumer Devices
Email Applications
![Page 12: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/12.jpg)
Classic Email Architecture
Internet Firewall Appliances
Content Scanners
MailboxServers
Consumer Devices
Email Applications
![Page 13: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/13.jpg)
Classic Email Architecture
Internet Firewall Appliances
Content Scanners
Mailbox Servers
Consumer Devices
Email Applications
![Page 14: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/14.jpg)
Internet
Firewall Appliances
Content Scanners
Mailbox Servers
Consumer Devices
Amazon SESEmail
Applications
Sending Mail with Amazon SES
![Page 15: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/15.jpg)
Internet
Firewall Appliances
Content Scanners
Mailbox Servers
Consumer Devices
Amazon SESEmail
Applications
Receiving Mail
![Page 16: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/16.jpg)
Receiving Mail with Amazon SES
- Launched September 2015
- Postmaster as a service
- Easy programmatic access to receiving email
- Integrates with Amazon S3, AWS KMS, Amazon SNS,
Amazon WorkMail, and AWS Lambda
![Page 17: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/17.jpg)
Internet
Firewall Appliances
Content Scanners
Mailbox Servers
Consumer Devices
Amazon SESEmail
Applications
Closing the Loop
![Page 18: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/18.jpg)
Internet Amazon SES
MailboxServers
Consumer Devices
Email Applications
Closing the Loop
![Page 19: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/19.jpg)
Internet Amazon SES
Amazon S3
AWS KMS
Email Architecture with AWS
![Page 20: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/20.jpg)
Internet Amazon SES Amazon SNS Amazon SQS
Email Architecture with AWS
![Page 21: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/21.jpg)
Internet Amazon SES AWS Lambda
Email Architecture with AWS
![Page 22: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/22.jpg)
Internet Amazon SES AmazonWorkMail
Email Architecture with AWS
![Page 23: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/23.jpg)
Amazon SES Receipt Rules
Internet Amazon SES
*@amzn.co
AmazonWorkMail
Customer Support
![Page 24: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/24.jpg)
Amazon SES Receipt Filters
Internet Amazon SES
54.240.12.34
Inbox
173.194.33.150
Reject
✗
![Page 25: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/25.jpg)
Amazon SES
- Works with postmasters and deliverability managers of
major email service providers
- Provides powerful tools for postmasters, deliverability
managers, systems administrators, and developers
- Makes email accessible for businesses
![Page 26: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/26.jpg)
![Page 27: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/27.jpg)
Building Email Applications
![Page 28: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/28.jpg)
Building Email Applications
Use AWS to solve email-related business problems
Leverage the following services:
AWS LambdaAmazon SES Amazon
DynamoDB
and more…
![Page 29: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/29.jpg)
Example Solutions Leveraging SES & AWS
• Automatically create support tickets from customer
email.
• Implement an email auto-responder.
• Process email bounces and complaints.
• Create an email archival solution.
• Receive files from customers via email.
![Page 30: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/30.jpg)
Example Solution 1:Anonymous Email Communication
![Page 31: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/31.jpg)
Anonymous Email
Seller BuyerProduct sale
![Page 32: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/32.jpg)
Anonymous Email
Seller Buyer
Product sale
Email communication
![Page 33: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/33.jpg)
Anonymous Email
Seller Buyer
Product sale
Email communication
Proxy
![Page 34: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/34.jpg)
Anonymous Email
![Page 35: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/35.jpg)
Anonymous Email — Requirements
• Protect customer information
• Utilize email standards
• Secure and resilient to abuse
• Fault tolerant
• Provide auditing and history
![Page 36: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/36.jpg)
Anonymous Email — Traditional Architecture
Receiving email
servers
Spam & virus
scanning
Routing and
application logic
Database
Sending email
servers
Seller Buyer
![Page 37: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/37.jpg)
Amazon DynamoDB or Amazon RDS
Anonymous Email — With AWS
Receiving email
servers
Spam & virus
scanning
Routing and
application logic
Database
Sending email
servers
Seller Buyer
Amazon SESAmazon EC2
Amazon SES
Amazon Lambda
![Page 38: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/38.jpg)
Anonymous Email — SES Receipt Rule
![Page 39: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/39.jpg)
Anonymous Email — SES Receipt Rule
Amazon S3
bucket
Amazon SES
AWS Lambda
function
SES receipt
rule
![Page 40: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/40.jpg)
Anonymous Email — AWS Lambda Application
AWS Lambda
Filter abuseRead
message
Amazon S3
bucket
Determine recipient
DynamoDB
table
Forward message
Amazon SES
Amazon SES
From seller
To buyer
![Page 41: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/41.jpg)
Anonymous Email — AWS Lambda Application
![Page 42: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/42.jpg)
Anonymous Email — Improvements
• Disable communication after elapsed time
• Store transaction logs
• Emit metrics
• Time until response
• Unanswered emails
![Page 43: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/43.jpg)
Anonymous Email — Savings
Customers using SES for this solution can expect to gain:
• Over 50% cost savings compared to EC2-based solution
• Reduction in operations and maintenance
• Effortless scaling from 100/day to 100MM/day
$150-$600/month
for multi-AZ EC2
instances & load balancer
$50/month
for 500,000 messages
with SES
![Page 44: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/44.jpg)
Example Solution 2:List Unsubscribe Support
![Page 45: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/45.jpg)
List Unsubscribe — The Problem
![Page 46: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/46.jpg)
• Header in email messages, usually marketing
• Supported by email clients & providers (gmail, hotmail, etc.)
From: [email protected]
Subject: Monthly newsletter
Date: August 31, 2015 3:13:02 PM PDT
List-Unsubscribe: <mailto:[email protected]>
List Unsubscribe — The Solution
![Page 47: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/47.jpg)
List Unsubscribe — Requirements
• Add List-Unsubscribe header to outgoing emails
• Accept email unsubscribe requests
• Manage subscriber preferences
• Secure and resilient to abuse
• Fault tolerant
![Page 48: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/48.jpg)
• Unique identifier encapsulates customer ID or email
address
• Could optionally contain mailing or campaign ID for
detailed unsubscribe metrics
• HTTP URLs are also welcome
List Unsubscribe — Add header
From: [email protected]
Subject: Monthly newsletter
Date: August 31, 2015 3:13:02 PM PDT
List-Unsubscribe: <mailto:[email protected]>
![Page 49: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/49.jpg)
List Unsubscribe — SES Receipt Rule
AWS Lambda
Amazon SESUnsubscribe
request
Lambda SES event object contains
information about the message received
SES receipt
rule
![Page 50: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/50.jpg)
List Unsubscribe — Example SES Event"ses": {
"mail": {
"commonHeaders": {
"from": ["Morgan Thomas <[email protected]>"],
"to": ["Bobby <[email protected]>"],
"returnPath": "[email protected]",
"messageId": "j4ghdrd4gmvo00",
"date": "Thu, 10 Sep 2015 17:25:37 +0000",
"subject": "Monthly newsletter"
},
"headers": […],
"messageId": "f18rgld2sbme6gjpuhncr2gmlo00"
},
"receipt": {
"spamVerdict": { "status": ”FAIL” },
"virusVerdict": { "status": ”PASS” },
"dkimVerdict": { "status": "PASS” },
"spfVerdict": { "status": "PASS” },
"processingTimeMillis": 409
}
}
Commonly used headers
Complete list of headers
SES message ID
Information provided by
SES about the email
![Page 51: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/51.jpg)
List Unsubscribe — AWS Lambda Application
AWS Lambda
Amazon SES
Parse recipient
Look up email address
Unsubscribe
DynamoDB
table
DynamoDB
table
Filter abuse
Unsubscribe request
![Page 52: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/52.jpg)
Example Solution 3:Centralized Abuse Filtering
![Page 53: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/53.jpg)
What’s in Common?
Filter abuse
Read Look up Forward
Parse Look upUnsub-scribe
Filter abuse
![Page 54: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/54.jpg)
What’s in Common? — Abuse Filtering
• You don’t want spam
• You don’t want viruses
Optionally:
• You only want authenticated mail
• (valid SPF & DKIM)
• You only want mail for valid recipients
• (existing users in directory)
![Page 55: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/55.jpg)
Active Rule Set
S3 action Lambda action
Lambda action
Amazon SES
List unsubscribe
Anonymous email
Abuse filter
mail.ses-example.com
unsubscribe.ses-example.com
1
2
3
(applies to all recipients)
![Page 56: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/56.jpg)
Active Rule Set
S3 action Lambda action
Lambda action
Amazon SES
List unsubscribe
Anonymous email
Abuse filter
mail.ses-example.com
unsubscribe.ses-example.com
1
2
3
Lambda action Bounce action Stop action
Determine if mail is desired(is it spam?)
If desiredreturn
‘stop_rule’
If not desiredreturn
‘continue’
RequestResponse (synchronous)
![Page 57: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/57.jpg)
Active Rule Set
S3 action Lambda action
Lambda action
Amazon SES
List unsubscribe
Anonymous email mail.ses-example.com
unsubscribe.ses-example.com
1
2
3
Abuse filter
Lambda action Bounce action Stop actionS3 action
![Page 58: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/58.jpg)
Active Rule Set
S3 action Lambda action
Lambda action
Amazon SES
List unsubscribe
Anonymous email
mail.ses-example.com
unsubscribe.ses-example.com
1
2
3
Abuse filter
Lambda action Bounce action Stop actionS3 action
![Page 59: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/59.jpg)
Your Future Rule Set
Amazon SES
????
• Share ideas and feature requests on the AWS forums
• forums.aws.amazon.com
• Follow announcements and email tips on the SES blog
• sesblog.amazon.com
![Page 60: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/60.jpg)
Thank you!
![Page 61: (ARC342) Designing & Building An End-To-End Email Solution Using AWS](https://reader033.vdocuments.net/reader033/viewer/2022042706/58a207651a28ab40098b5123/html5/thumbnails/61.jpg)
Remember to complete
your evaluations!