Learn the specifics of Amazon RDS for PostgreSQL’s capabilities and extensions that make it powerful. This session begins with a brief overview of the RDS PostgreSQL service, how it provides High Availability & Durability and will then deep dive into the new features that we have released since re:Invent 2014, including major version upgrade and newly added PostgreSQL extensions to RDS PostgreSQL. During the session, we will also discuss lessons learned running a large fleet of PostgreSQL instances, including specific recommendations. In addition we will present benchmarking results looking at differences between the 9.3, 9.4 and 9.5 releases.
3. What’s new in storage
6TB storage
• PIOPS has 30K IOPS max
• GP2 increase storage above 3TB = increase throughput & IOPS
Encryption at rest
• Uses the AWS Key Management Service (KMS) part of AWS
Identity and Access Management (IAM)
• Your own key
• Use a default one
• Includes all data files, log files, log backups, and snapshots
10. Move data to the same or different database engine
Keep your apps running during the migration
Start your first migration in 10 minutes or less
Replicate within, to, or from AWS EC2 or RDS
AWS
Database Migration
Service
11. Customer
Premises
Application Users
EC2
or
RDS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let the AWS Database Migration
Service create tables and load data
Uses change data capture to keep
them in sync
Switch applications over to the target
at your convenience
Keep your apps running during the migration
AWS Database
Migration Service
12. AWS Database Migration Service - PostgreSQL
• Source - on premises or Amazon EC2 PostgreSQL (9.4)
• Destination can be EC2 or RDS
• Initial bulk copy via consistent select
• Uses PostgreSQL logical replication support to provide
change data capture
http://aws.amazon.com/rds/DatabaseMigrationService/preview
18. Vacuum parameters
Will auto vacuum when
• autovacuum_vacuum_threshold +
autovacuum_vacuum_scale_factor * pgclass.reltuples
How hard auto vacuum works
• autovacuum_max_workers
• autovacuum_nap_time
• autovacuum_cost_limit
• autovacuum_cost_delay
22. shared_buffers parameter
244GB RAM
PG processes
shared_buffers
Linux
pagecache
select of data – check for buffer in shared_buffers
if not in shared_buffers load from pagecache/disk
EBS
1/4
shared_buffers = working set size
23. 0
2,000
4,000
6,000
8,000
10,000
12,000
3% 6% 13% 25% 50% 75%
transactionspersecond(TPS)
shared_buffers as a percentage of system memory
pgbench write workload on r3.8xlarge
working set = 10% of memory
25 threads
50 threads
100 threads
200 threads
400 threads
800 threads
24. 0
2,000
4,000
6,000
8,000
10,000
12,000
13% 25% 50% 75%
transactionspersecond(TPS)
shared_buffers as a percentage of system memory
pgbench write workload on r3.8xlarge
working set = 50% of memory
25 threads
50 threads
100 threads
200 threads
400 threads
800 threads
25. Availability – Read and Write – Multi-AZ
Physical
Synchronous
Replication
AZ1 AZ2
DNS
cname update
Primary Update
31. Replication parameters – continued
vacuum_defer_cleanup_age
max_standby_archive_delay
max_standby_streaming_delay
hot_standby_feedback
A - Foo
A- Bar
Source
A - Foo
A- Bar
Replica
32. vacuum_defer_cleanup_age
on primary
default is 0
# of transactions
Table T1
t1 – foo, bar
t2 – foo, car
t3 – foo, dar
t4 – foo, ear
t5 – foo, far
t6 – foo, gar
t1 – foo, bar
t2 – foo, car
t3 – foo, dar
t4 – foo, ear
t5 – foo, far
36. pg_stat_statements
Change parameter shared_preload_libraries=pg_stat_statements
=>create extenstion pg_stats_statements
=>select query, calls, total_time, rows, shared_blks_read from
pg_stat_statements where total_time > 100 and query like '%usertable%';
query | calls | total_time | rows | shared_blks_read
-------------------------------------------------------------------------------------------+----------+------------------+------------+-----------------
SELECT * FROM usertable WHERE YCSB_KEY = $1 | 71356782 | 8629119.24887683 | 71356780 | 28779668
SELECT * FROM usertable WHERE YCSB_KEY >= $1 LIMIT ? | 12068394 | 62530609.930002 | 1206839246 | 171093346
UPDATE usertable SET FIELD1=$1 WHERE YCSB_KEY = $2 | 7048967 | 35813107.3580354 | 7048967 | 3825857
analyze usertable; | 1 | 2129.84 | 0 | 15679
SELECT * FROM usertable WHERE YCSB_KEY >= $1 AND md5(YCSB_KEY) = md5(YCSB_KEY) LIMIT ? | 15441280 | 39356905.8080029 | 1544127640 | 230668106
37. Burst mode: GP2 and T2
T2 – Amazon EC2 instance with burst capability
• Base performance + burst
• Earn credits per hour when below base performance
• Can store up to 24 hours worth of credits
• Amazon CloudWatch metrics to see credits and usage
GP2 – SSD-based Amazon EBS storage
• 3 IOPS per GB base performance
• Earn credits when usage below base
• Burst to 3000+ IOPS