amazon rds postgresql: enabling innovation with cloud managed databases
TRANSCRIPT
![Page 1: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/1.jpg)
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Grant McAlister – Senior Principal Engineer - RDS
March 2017
Amazon RDS for PostgreSQLEnabling Innovation with Cloud Managed Databases
![Page 2: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/2.jpg)
Development to Production
DB1
Test1
DB2
Test2
Big Test
DB1 DB2 DB3
DB4 DB5 DB6
Production
DB1DB2
QA
DB1 DB2
Patching
HA & DRBackup & Recovery
ScaleSecurity
![Page 3: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/3.jpg)
Amazon Relational Database Service
RDS is a managed Relational database service that is simple to deploy, easy to scale, reliable and cost-effective
Managed Service
Easy to Scale and Operate
Choice of Database Engines
High Availability
High Performance
Amazon Relational Database Service (RDS)
![Page 4: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/4.jpg)
RDS Version Updates
Now Supporting 9.6
Minor Releases• 9.6.1• 9.5.4 • 9.4.9 • 9.3.14
![Page 5: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/5.jpg)
Extension Support Additions
9.6 bloom & pg_visibility
9.3 Original - 32
9.3 Current - 35
9.4 Current - 399.5 Current - 44
Future - ???9.6 Current - 46
![Page 6: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/6.jpg)
Availability
![Page 7: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/7.jpg)
Backups and Disaster RecoveryDB Snapshots
• User-driven snapshots of database• Kept until explicitly deleted
Automated Backups• Nightly system snapshots + transaction backup• Enables point-in-time restore to any point in retention period, up to
the last 5 minutes• Max retention period = 35 days
Cross region copy• Change regions• Disaster recovery
![Page 8: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/8.jpg)
Availability – Read and Write – Multi-AZ
Secondary
Physical Synchronous
Replication
Primary
AZ1 AZ2
Application Application
![Page 9: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/9.jpg)
Availability – Read and Write – Multi-AZ
SecondarySecondary Primary
Physical Synchronous
Replication
AZ1 AZ2
Application Application
![Page 10: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/10.jpg)
Availability – Read and Write – Multi-AZ
SecondarySecondary Primary
Physical Synchronous
Replication
AZ1 AZ2
DNS
cname update
Application Application
Primary Update
![Page 11: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/11.jpg)
Availability – Read and Write – Multi-AZ
Secondary Primary
Physical Synchronous
Replication
AZ1 AZ2
DNS
cname update
Application Application
![Page 12: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/12.jpg)
Read Replicas = Availability
Secondary
Application
Read Replica
Read Replica
SyncReplication
Multi-AZ
Primary
Async Replication
![Page 13: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/13.jpg)
Read Replicas = Availability
Secondary
Application
Read Replica
Read Replica
SyncReplication
Multi-AZ
Primary
Eventually Consistent Reads
Writes & Consistent
Reads
Async Replication
![Page 14: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/14.jpg)
Read Replicas = Availability
Application
Read Replica
Read ReplicaSecondary Primary
Eventually Consistent Reads
Writes & Consistent
Reads
Async Replication
![Page 15: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/15.jpg)
Read Replicas = Availability
Application
Read Replica
Read ReplicaSecondary Primary
Eventually Consistent Reads
Writes & Consistent
Reads
Async Replication
![Page 16: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/16.jpg)
Read Replicas = Availability
Application
Read Replica
Read ReplicaSecondary Primary
Eventually Consistent Reads
Writes & Consistent
Reads
Async Replication
Upgrade
![Page 17: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/17.jpg)
Read Replicas = Availability
Application
Read Replica
Read ReplicaSecondary Primary
Eventually Consistent Reads
Writes & Consistent
Reads
Async Replication
Modify
![Page 18: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/18.jpg)
Cross Region Replicas – DR & Moves
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
Read Replica
AZ1Async Replication
US-EAST-1 EU-WEST-1
![Page 19: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/19.jpg)
Cross Region Replicas – DR & Moves
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
Read Replica
AZ1Async Replication
US-EAST-1 EU-WEST-1
![Page 20: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/20.jpg)
Cross Region Replicas – DR & Moves
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
Read Replica
AZ1
New Primary
Async Replication
US-EAST-1 EU-WEST-1
![Page 21: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/21.jpg)
Cross Region Replicas – DR & Moves
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
Read Replica
Application
AZ1
New Primary
Async Replication
US-EAST-1 EU-WEST-1
![Page 22: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/22.jpg)
Cross Region Replicas – DR & Moves
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
Read Replica
Application
AZ1
New Primary
Async Replication
US-EAST-1 EU-WEST-1
Application
SecondarySync
AZ2
![Page 23: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/23.jpg)
Cross Region Replicas – DR & Moves
Read Replica
Application
AZ1
New Primary
US-EAST-1 EU-WEST-1
Application
SecondarySync
AZ2
![Page 24: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/24.jpg)
Cross Region Replicas – Reduce Latency
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
US-EAST-1
![Page 25: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/25.jpg)
Cross Region Replicas – Reduce Latency
SecondarySync
Application
Read ReplicaPrimary
AZ1 AZ2
Application
Read Replica
Application
AZ1Async Replication
US-EAST-1 EU-WEST-1
![Page 26: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/26.jpg)
AuroraR/W
Application Application
Aurora R
Application ROApplication
SyncReplication
AZ-1 AZ-2 AZ-3
Storage Storage Storage Storage Storage Storage
Aurora Storage
4/6 sync writes
Aurora PostgreSQL – In Preview
![Page 27: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/27.jpg)
AuroraR/W
Application Application
Aurora R
Application ROApplication
SyncReplication
AZ-1 AZ-2 AZ-3
Storage Storage Storage Storage Storage Storage
Aurora Storage
4/6 sync writes
Aurora PostgreSQL – In Preview
Async Invalidation
![Page 28: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/28.jpg)
AuroraR/W
Application Application
Aurora R
Application ROApplication
SyncReplication
AZ-1 AZ-2 AZ-3
Storage Storage Storage Storage Storage Storage
Aurora Storage
4/6 sync writes
AuroraR/W
Aurora R
Aurora PostgreSQL – In Preview
Async Invalidation
FAILOVER
![Page 29: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/29.jpg)
Amazon Aurora with PostgreSQL Compatibility
Performance By The NumbersMeasurement Result
PgBench >= 2x faster
SysBench 2x-3x faster
Data Loading 3x faster
Response Time >2x faster
Throughput Jitter >3x more consistent
Throughput at Scale 3x faster
Recovery Speed Up to 85x faster
![Page 30: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/30.jpg)
Patching
![Page 31: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/31.jpg)
Minor Version Patching
• Customer Control Patching• Execute immediately or in Weekly Maintenance Window
• Auto Patching can be enabled • Allows for hands off upgrade in next Maintenance Window
![Page 32: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/32.jpg)
Major version upgrade
Prod9.5
![Page 33: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/33.jpg)
Major version upgrade
Prod9.5
Prod9.6
pg_upgrade
Backup Backup
No PITR
![Page 34: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/34.jpg)
Major version upgrade
Prod9.5
Test9.5
Restore to a test instance
![Page 35: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/35.jpg)
Major version upgrade
Prod9.5
Test9.5
pg_upgrade
Restore to a test instance
![Page 36: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/36.jpg)
Major version upgrade
Prod9.5
Test9.5
Test9.6
pg_upgrade
Restore to a test instance
![Page 37: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/37.jpg)
Major version upgrade
Prod9.5
Test9.5
Test9.6
pg_upgrade
Restore to a test instance
Application Testing
![Page 38: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/38.jpg)
Major version upgrade
Prod9.5
Test9.5
Test9.6
pg_upgrade
Restore to a test instance
Application Testing
![Page 39: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/39.jpg)
Major version upgrade
Prod9.5
Prod9.6
pg_upgrade
Backup Backup
No PITR
Test9.5
Test9.6
pg_upgrade
Restore to a test instance
Application Testing
![Page 40: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/40.jpg)
Security
![Page 41: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/41.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
Host
Log Backups
![Page 42: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/42.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
Host
Log Backups
Security Group
![Page 43: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/43.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
![Page 44: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/44.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
VPC
![Page 45: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/45.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
VPC
Encryption at Rest
![Page 46: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/46.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
VPC
Encryption at Rest
ssl_mode=disable
![Page 47: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/47.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
VPC
Encryption at Rest
ssl_mode=disable
![Page 48: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/48.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
VPC
Encryption at Rest
ssl_mode=disable
rds.force_ssl=1 (default 0)
![Page 49: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/49.jpg)
Forcing SSL on all connections
DB Instance
SnapshotApplication
HostSSL
Log Backups
Security Group
VPC
Encryption at Rest
ssl_mode=disable
rds.force_ssl=1 (default 0)
![Page 50: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/50.jpg)
Unencrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Share with account
![Page 51: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/51.jpg)
Unencrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Snapshot
Share with account
![Page 52: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/52.jpg)
Unencrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
SnapshotDB Instance
Snapshot
Share with account
![Page 53: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/53.jpg)
Unencrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
SnapshotDB Instance
Snapshot
Share with account
Share to Public
![Page 54: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/54.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Encryption at Rest
Default
![Page 55: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/55.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Share with account
Encryption at Rest
Default
![Page 56: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/56.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Share with account
Encryption at Rest
Default
![Page 57: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/57.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Share with account
Encryption at Rest
![Page 58: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/58.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Share with account
Encryption at Rest
CustomKey
![Page 59: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/59.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
Snapshot
Share with account
Encryption at Rest
CustomKey
Add external account
![Page 60: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/60.jpg)
Encrypted Snapshot Sharing
DB Instance
Snapshot
Prod Account
Test Account
SnapshotDB Instance
Snapshot
Share with account
Encryption at Rest
CustomKey
Add external account
![Page 61: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/61.jpg)
Cross Region Replicas – Encrypted
SecondarySync
Application
Primary
AZ1 AZ2
Application
US-EAST-1
![Page 62: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/62.jpg)
Cross Region Replicas – Encrypted
SecondarySync
Application
Primary
AZ1 AZ2
Application
Read Replica
Application
AZ1Async Replication
US-EAST-1 EU-WEST-1
![Page 63: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/63.jpg)
2 Threads 4 Threads 8 Threads 16 Threads 32 Threads 64 Threads0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500PG Bench - Read & Write
RegularEncrypted
Tran
sact
ions
Per
Sec
ond
(TPS
)Encryption at rest overhead
5 to 10% Overhead on heavy write
![Page 64: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/64.jpg)
HIPAA-eligible service & FedRAMP
• RDS PostgreSQL is now a HIPAA-eligible service• https://aws.amazon.com/compliance/hipaa-compliance/
• FedRAMP in AWS GovCloud (US) region • https://aws.amazon.com/compliance/fedramp/
![Page 65: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/65.jpg)
Data movement
![Page 66: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/66.jpg)
Move data to the same or different database engine Keep your apps running during the migrationStart your first migration in 10 minutes or lessReplicate within, to, or from AWS EC2 or RDS
AWSDatabase Migration
Service(DMS)
![Page 67: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/67.jpg)
CustomerPremises
Application Users
EC2or
RDS
Internet
VPN
Start a replication instance
Keep your apps running during the migration
AWS Database Migration Service
![Page 68: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/68.jpg)
CustomerPremises
Application Users
EC2or
RDS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Keep your apps running during the migration
AWS Database Migration Service
![Page 69: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/69.jpg)
CustomerPremises
Application Users
EC2or
RDS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let the AWS Database Migration Service create tables and load data
Keep your apps running during the migration
AWS Database Migration Service
![Page 70: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/70.jpg)
CustomerPremises
Application Users
EC2or
RDS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let the AWS Database Migration Service create tables and load data Uses change data capture to keep them in sync
Keep your apps running during the migration
AWS Database Migration Service
![Page 71: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/71.jpg)
CustomerPremises
Application Users
EC2or
RDS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let the AWS Database Migration Service create tables and load data Uses change data capture to keep them in syncSwitch applications over to the target at your convenience
Keep your apps running during the migration
AWS Database Migration Service
![Page 72: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/72.jpg)
AWS Database Migration Service - PostgreSQL
• Source - on premise or EC2 PostgreSQL (9.4+) RDS (9.4.9+ or 9.5.4+ or 9.6.1)
• Destination can be EC2 or RDS• Initial bulk copy via consistent select• Uses PostgreSQL logical replication support to provide
change data capture
https://aws.amazon.com/dms/
![Page 73: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/73.jpg)
Logical Replication Support• Supported with 9.6.1+, 9.5.4+ and 9.4.9+• Set rds.logical_replication parameter to 1• As user who has rds_replication & rds_superuser role
SELECT * FROM pg_create_logical_replication_slot('test_slot', 'test_decoding');
pg_recvlogical -d postgres --slot test_slot -U master --host $rds_hostname -f - --start
• Added support for Event Triggers
![Page 74: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/74.jpg)
Logical Decoding Space Usage
![Page 75: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/75.jpg)
CloudWatch – Replication Lag
![Page 76: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/76.jpg)
CloudWatch – Slot usage for WAL
![Page 77: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/77.jpg)
Logical Replication Support - Example
RDS Postgres
RDS Postgres
Logical Replica
Redshift
DMS
![Page 78: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/78.jpg)
Logical Replication Support - Example
RDS Postgres
RDS Postgres
Logical Replica
Redshift
On PremisePostgres
DMS
![Page 79: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/79.jpg)
Logical Replication Support - Example
RDS Postgres
RDS Postgres
Logical Replica
RedshiftEC2 Postgres
On PremisePostgres
DMS
![Page 80: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/80.jpg)
Logical Replication Support - Example
RDS Postgres
RDS Postgres
Logical Replica
RedshiftEC2 Postgres
On PremisePostgres
DMS
EC2 Oracle
![Page 81: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/81.jpg)
Logical Replication Support - Example
RDS Postgres
RDS Postgres
Logical Replica
RedshiftEC2 Postgres
On PremisePostgres
DMS
EC2 Oracle
CustomLogicalHandler
NoSQL DB
![Page 82: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/82.jpg)
Schema Conversion Tool - SCT
Downloadable tool (Windows, Mac, Linux Desktop)
Source Database Target Database on Amazon RDSMicrosoft SQL Server Amazon Aurora, MySQL, PostgreSQLMySQL PostgreSQLOracle Amazon Aurora, MySQL, PostgreSQLPostgreSQL Amazon Aurora, MySQL
![Page 83: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/83.jpg)
SCT - Analysis
![Page 84: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/84.jpg)
SCT - Detailed
![Page 85: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/85.jpg)
Scale & Performance
![Page 86: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/86.jpg)
Push Button Scaling
Scale nodes vertically up or down• db.t2.micro (1 virtual cores, 1GB)• db.m4.large (2 virtual cores, 8GB) • db.r3.8xlarge(32 virtual cores, 244GB)
• db.r4.16xlarge (64 virtual cores, 488GB) Aurora
Storage Scaling EBS Based - 5GB-6TB & Up to 30,000 IOPSAurora Based – Up to 64 TB
![Page 87: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/87.jpg)
M4 Instance Class – pgbench read only
1 2 4 8 160
2000
4000
6000
8000
10000
12000
14000
db.m3.large db.m4.large
Threads
Tran
sact
ions
per
Sec
ond
(TPS
)
46% Better Price/Performance
37% TPS Increase
$0.195 $0.182
![Page 88: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/88.jpg)
Enhanced Operating System (OS) metrics
1-60 second granularity
cpuUtilization• guest• irq• system• wait• idl: • user • total • steal • nice
diskIO • writeKbPS• readIOsPS• await • readKbPS• rrqmPS • util • avgQueueLen • tps • readKb • writeKb • avgReqSz • wrqmPS • writeIOsPS
memory • writeback• cached • free • inactive• dirty • mapped • active • total • slab • buffers• pageTable• Hugepages
swap • cached • total • free
tasks• sleeping • zombie • running • stopped • total • blocked
fileSys • used• usedFiles• usedFilePercent• maxFiles • total • usedPercent
loadAverageMinute • fifteen • five • one
uptime
processList• name• cpuTime• parentID• memoryUsedPct• cpuUsedPct• id • rss• vss
![Page 89: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/89.jpg)
Process List
![Page 90: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/90.jpg)
OS metrics
![Page 91: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/91.jpg)
Performance Insights – In Preview
![Page 92: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/92.jpg)
Performance Insights – In Preview
![Page 93: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/93.jpg)
Stats on RAMDISK
• Set rds.pg_stat_ramdisk_size in MB’s• Creates a RAM disk and sets stats_temp_directory to
use it.
• Reduces IOPS
• Good for instances with many tables/indexes and databases.
![Page 94: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/94.jpg)
Burst mode: GP2 and T2
T2 – Amazon EC2 instance with burst capability• Base performance + burst • Earn credits per hour when below base performance• Can store up to 24 hours worth of credits• Amazon CloudWatch metrics to see credits and usage
GP2 – SSD-based Amazon EBS storage• 3 IOPS per GB base performance• Earn credits when usage below base• Burst to 3000+ IOPS
![Page 95: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/95.jpg)
T2 – CPU credits
CPU credit balance
T2 exhaust CPU credits
![Page 96: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/96.jpg)
Burst mode: what’s new
db.t2.large• 60 CPU Initial Credit• 36 CPU Credit earned per hour• Base Performance – 60%• 8 GB RAM• Increased IO bandwidth• Encryption at rest support
Future – t2.xlarge & t2.2xlarge
![Page 97: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/97.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
2000
4000
6000
8000
10000
12000100% Read - 20GB data
db.m1.medium + 200GB standarddb.m3.medium + 200G + 2000 IOPSdb.m3.large + 200G + 2000 IOPSdb.t2.medium + 200GB gp2
Hours
Tran
sact
ions
per
Sec
ond
(TPS
)Burst mode vs. Classic vs. Provisioned IOPS
$0.10 per hour
$0.58 per hour$0.40 per hour
$0.50 per hour
![Page 98: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/98.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
2000
4000
6000
8000
10000
12000100% Read - 20GB data
db.m1.medium + 200GB standarddb.m3.medium + 200G + 2000 IOPSdb.m3.large + 200G + 2000 IOPSdb.t2.medium + 200GB gp2db.t2.medium + 1TB gp2
Hours
Tran
sact
ions
per
Sec
ond
(TPS
)Burst mode vs. Classic vs. Provisioned IOPS
$0.10 per hour
$0.58 per hour
$0.23 per hour
$0.40 per hour
$0.50 per hour
![Page 99: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/99.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
2000
4000
6000
8000
10000
12000100% Read - 20GB data
db.m1.medium + 200GB standarddb.m3.medium + 200G + 2000 IOPSdb.m3.large + 200G + 2000 IOPSdb.t2.medium + 200GB gp2db.t2.medium + 1TB gp2db.t2.large + 1TB gp2
Hours
Tran
sact
ions
per
Sec
ond
(TPS
)Burst mode vs. Classic vs. Provisioned IOPS
$0.10 per hour
$0.58 per hour
$0.23 per hour
$0.40 per hour
$0.50 per hour
$0.30 per hour
![Page 100: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/100.jpg)
Lessons
![Page 101: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/101.jpg)
Vacuum parameters
Will auto vacuum when• autovacuum_vacuum_threshold +
autovacuum_vacuum_scale_factor * pgclass.reltuples
How hard auto vacuum works • autovacuum_max_workers• autovacuum_nap_time• autovacuum_cost_limit• autovacuum_cost_delay
Transaction IDWrap Around
![Page 102: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/102.jpg)
RDS autovacuum logging (9.4.5+)
log_autovacuum_min_duration = 5000 (i.e. 5 secs)rds.force_autovacuum_logging_level = LOG
…[14638]:ERROR: canceling autovacuum task…[14638]:CONTEXT: automatic vacuum of table "postgres.public.pgbench_tellers"…[14638]:LOG: skipping vacuum of "pgbench_branches" --- lock not available
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.Autovacuum
![Page 103: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/103.jpg)
RDS autovacuum visibility(9.3.12, 9.4.7, 9.5.2, 9.6.1)pg_stat_activity
BEFORE usename | query----------+------------------------------------------------------------- rdsadmin | <insufficient privilege> rdsadmin | <insufficient privilege> gtest | SELECT c FROM sbtest27 WHERE id BETWEEN 392582 AND 392582+4 gtest | select usename, query from pg_stat_activity
NOW usename | query----------+---------------------------------------------- rdsadmin | <insufficient privilege> gtest | select usename, query from pg_stat_activity gtest | COMMIT rdsadmin | autovacuum: ANALYZE public.sbtest16
![Page 104: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/104.jpg)
CloudWatch Metric
![Page 105: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/105.jpg)
AWS Database BLOG
https://aws.amazon.com/blogs/database/
![Page 106: Amazon RDS PostgreSQL: Enabling Innovation with Cloud Managed Databases](https://reader036.vdocuments.net/reader036/viewer/2022081512/58f9a8bb1a28ab9c288b4595/html5/thumbnails/106.jpg)
Thank you!
Questions?