introducing amazon rds for postgresql (dat210) | aws re:invent 2013
DESCRIPTION
AWS customers have been asking us for Amazon RDS for PostgreSQL, and we’re excited to announce its immediate availability. Learn how you can offload the management of your PostgreSQL database instances to Amazon RDS using automated backups and point-in-time recovery, Multi-AZ deployments for high availability, and provisioned IOPS for fast and predictable performance. Also learn how to take advantage of familiar PostgreSQL features such as PostGIS with Amazon RDS for PostgreSQL.TRANSCRIPT
![Page 1: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/1.jpg)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
DAT210 – Introducing Amazon RDS for PostgreSQL
Srikanth Deshpande - Senior Product Manager, AWS
Nick Hertl – Software Development Manager, AWS
Gabe Arnett – Senior Director, Moody’s Analytics
November 14, 2013
![Page 2: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/2.jpg)
Amazon Relational Database Service RDS is a managed relational database service that is simple to deploy,
easy to scale, reliable, and cost-effective
Managed Service
Easy to Scale and Operate
Choice of Database Engines
High Availability
High Performance
Amazon Relational Database Service (RDS)
![Page 3: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/3.jpg)
Backups and Disaster Recovery
![Page 4: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/4.jpg)
Push-Button Scaling
![Page 5: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/5.jpg)
Multi-AZ Deployments
![Page 6: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/6.jpg)
Security Internet
IAM
VPC
![Page 7: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/7.jpg)
DB Parameter Groups
{DBInstanceClassMemory/12582880} Filter=“connection”
![Page 8: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/8.jpg)
Amazon RDS for PostgreSQL • Database version: PostgreSQL 9.3.1
• Includes valuable Amazon RDS functionality – Fast deployment
– Backups and point-in-time recovery
– Snapshots and restore
– Compute and storage scaling
– Multi-AZ
– Provisioned IOPS
8
![Page 9: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/9.jpg)
Launching a Postgres DB Instance
![Page 10: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/10.jpg)
Select Production Use (or not)
![Page 11: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/11.jpg)
Instance Details
![Page 12: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/12.jpg)
Additional Configuration
![Page 13: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/13.jpg)
Management Options
![Page 14: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/14.jpg)
Running Instance
![Page 15: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/15.jpg)
Connecting
![Page 16: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/16.jpg)
Permissions superuser role (Postgres)
rds_superuser role (RDS provided)
Load and use extensions
View and kill sessions
Create tablespace
Assign replication role
…
![Page 17: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/17.jpg)
Extensions
• PostGIS available
• rds.extensions parameter: – btree_gin
– btree_gist
– chkpass
– citext
– cube
– dblink
– dict_int
– dict_xsyn
– earthdistance
– fuzzystrmatch
– hstore
– intagg
– intarray
– isn
– ltree
– pgcrypto
– pgrowlocks
– pg_trgm
– plperl
– plpgsql
– pltcl
– postgis
– postgis_tiger_geocoder
– postgis_topology
– sslinfo
– tablefunc
– tsearch2
– unaccent
– uuid-ossp
![Page 18: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/18.jpg)
High Performance
16,500+ Read and 8,500+ Write = 25,000+ IOPS
![Page 19: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/19.jpg)
Getting Started
• Launch an instance from AWS Management
Console
• Configure network
• Load extensions
• Export from existing database using pg_dump
• Import to RDS using pg_restore
![Page 20: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/20.jpg)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
RDS PostgreSQL
Gabe Arnett, Senior Director, Moody’s Analytics
November 14, 2013
![Page 21: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/21.jpg)
• Moody’s Analytics offers unique tools and best practices for measuring and
managing risk through expertise and experience in credit analysis,
economic research, and financial risk management.
• Product offerings include leading-edge software, advisory services, and
credit and economic research.
• A subsidiary of Moody's Corporation (NYSE: MCO), which reported revenue
of $2.7 billion in 2012, employs approximately 7,200 people worldwide and
maintains a presence in 29 countries.
![Page 22: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/22.jpg)
Legacy Platform
Calculation Engine
![Page 23: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/23.jpg)
DataPortal
Source 2
Sybase 1
Sybase 2
Sybase 3
Oracle
Source 3
Source 7
Source 4
Source 8
Source 5
Source 6
Source 1
Ftp server
Routine 2
Routine 3
excl EJV
Legacy App Db
4x daily
EDF
Data
Transfer
App1x daily
1x daily
4x daily
1x daily
Front End Db’s
(11 servers)
MS SQL Server
Routine 4
Standalone
C++ App
Vendor
Data
Engine /
Calculator
1x monthly
Routine 11x daily
App Server 2
Job 2
(4x daily)
Job 3
(1x daily)
App Server 3
Job 4
(1x daily)
App Server 1
Job 1
(4x daily)
Pasta,
anyone?
![Page 24: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/24.jpg)
Overhaul
![Page 25: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/25.jpg)
Write Master (AZ 1)
Read Replica 1
(Warm Standby AZ 2)
Read Replica 2 (AZ 1)
WAL
Cascading Replication
ETL Cluster
Calculation Engine
(reads from RR2 and writes results to WM)
External Data Sources
Amazon Simple Storage
Service
Summer Fun
![Page 26: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/26.jpg)
Pros
• We learned a tremendous amount and could probably write a solid
blog post or whitepaper
• No cost other than infrastructure
• Support/maintenance tasks now very reasonable and can be done
with existing resources without incurring additional costs
![Page 27: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/27.jpg)
Cons
• Lots of time spent finding the write configurations, trial and error,
testing and more testing
• I have to convince really talented Java, Python, and .NET
developers that they have to be PostgreSQL system admins
• We are an enterprise, and as such I have to have an enterprise level
of support
![Page 28: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/28.jpg)
Write Master (Multi-AZ)
Read Replica
ETL Cluster
Calculation Engine
(Reads from RR2 and writes results to WM)
External Data Sources
Amazon Simple Storage
RDS PostgreSQL
Warm Standby
Region 2
Snapshot Copy
futu
re
![Page 29: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/29.jpg)
Write Master (Multi-AZ)
Read Replica
ETL Cluster
Calculation Engine
(Reads from RR and writes results to WM)
External Data Sources
Amazon Simple Storage
RDS PostgreSQL
Warm Standby
Region 2
Snapshot Copy
futu
re
![Page 30: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/30.jpg)
• Achieve the same performance as existing setup on Amazon EC2, if
not better, in a matter of minutes
• We get built-in backup/recovery/replication/fault tolerance/multi-AZ
• More robust operational support built in, and my developers can get
back to the business of development
Why Amazon RDS PostgreSQL?
![Page 31: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/31.jpg)
Up and Running
Self-Managed(hours)
• Launch Amazon EC2 w/EBS
• Mount and Raid0 Amazon EBS
• Install PostgreSQL
• Move data and logs
• Edit .conf files
• Create users
• Load/Use DB
• Create snapshot, and then…
RDS PostgreSQL(minutes)
• Add/Edit CIDR/IP block to security
group (pg_hba.conf)
• Edit DB parameter group to apply
configuration settings
(postgresql.conf)
• Launch RDS instance
• Load/Use DB
• Sit back and monitor or let
Amazon CloudWatch do it for us…
![Page 32: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/32.jpg)
Backup/Retention
• Single-click backup policy upon creation
• No schedule to implement or forget
• Snapshots are easy to find
– All easily found in the AWS Management Console and searchable
• One-click restore to point in time = AWESOME!!!
![Page 33: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/33.jpg)
Monitoring
• Amazon CloudWatch metrics alongside instance details
– A challenge to find and consolidate all the EBS volumes + EC2 instances
• Logs are in the console
– Not fun to dig through the logs, assuming we actually had that kind of time
• Event subscriptions for faults
– Extra pro-active protection
![Page 34: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/34.jpg)
Scale and Redundancy
• At launch, RDS PostgreSQL is multi-AZ enabled with a click
– We had to spin up a second instance and then configure WAL and hope and
pray
– Bit of configuration and tuning to get the correct performance for this without
impacting write performance and ensuring near real-time reads
– Lossless factor is a risk if the write master fails
![Page 35: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/35.jpg)
Next
• Additional legacy data platforms
• Extending PostgreSQL
– Developing key/value store for near real-time data ingestion
– Integrating with Solr
– Front end datamart
• Redshift for BI use cases
![Page 36: Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013](https://reader034.vdocuments.net/reader034/viewer/2022052315/554d8959b4c90567188b4fd9/html5/thumbnails/36.jpg)
Please give us your feedback on this
presentation
As a thank you, we will select prize
winners daily for completed surveys!
DAT210