SlideShare une entreprise Scribd logo
1  sur  70
Télécharger pour lire hors ligne
Streaming replication in practice.
PgConf.Russia 2016, Moscow
Lesovsky Alexey
lesovsky@pgco.me
Working stuff: http://goo.gl/Yy4UzH
I. How streaming replication works.
II. Replication setup.
Practice.
III. Monitoring and maintenance.
IV. Possible problems.
V. Switchiver и Failover.
Practice.
Agenda.
What is replication and it kinds?
Write Ahead Log. REDO and REDO realization in PostgreSQL.
Common replication architecture in PostgreSQL.
Description of processes that involved in replication.
Part I. How replication works.
Synchronize objects
Objects changes are moving to each others.
Replication maybe physical and logical.
What is replication.
Pros:
● Works between different major versions and architectures.
● Allow to replicate tables and tables sets.
Cons:
● Hard to implement synchronous replication.
● CPU overhead (triggers, text conversions, etc).
Examples:
● Slony, Londiste (Skytools), Bucardo, Pglogical.
Logical replication.
Pros:
● Resource usage minimal overhead.
● Easy setup, usage and maintenace.
Cons:
● Standbys are only read-only.
● Can't work with different versions and architectures.
● Can't replicate tables and tables sets.
Physical replication.
Commit all changes in database (Durability in ACID).
Flush all data from REDO buffers at COMMIT.
REDO log has history of all changes in database.
Any changes in database, written into REDO.
REDO log usage:
● in crash recovery;
● backup and Point In Time Recovery;
● replication.
Write Ahead Log. REDO.
In PostgreSQL, REDO also known as Write Ahead Log (WAL).
WAL guaranties that changes are commited before data will changed.
How it guarantied:
LSN (log sequence number) — record location (position) inside WAL;
● Any page is marked with LSN of last record that touched the page;
● Before page is written to disk, bufmgr must check that the WAL flushed to
specified LSN.
Write Ahead Log. REDO implementation in PostgreSQL.
Common view.
backend backend backend autovac ...
WAL writer*
WAL
Startup/Recovery
SR/Archiving/Hot Standby
Point in Time
WAL Sender/WAL Receiver
* - может отсутствовать
Main startup process task is run the database.
In standby mode it initializes infinite replay loop.
Read recovery.conf at REDO start.
REDO:
● read segments from pg_xlog/archive;
● start wal receiver and reading XLOG from upstream server.
When consistency point reached (min recovery ending location) allow
connections and starts checkpointer/bgwriter.
Processing all others parameters from recovery.conf.
More details see in StartupXLOG() function.
Startup process.
For any client postmaster runs dedicated process — backend.
WAL sender is backend too (it has am_walsender flag).
This backend runs exec_replication_command().
exec_replication_command() can do various things:
● create/remove replication slots;
● start basebackup;
● start physical/logical replication.
In last case, backend sends XLOG segments to the client.
Or sleeps when no new XLOG.
WAL Sender process.
Startup process checks XLOG sources.
Startup process init startup of WAL receivers.
Need recovery.conf with primary_conninfo.
WAL receiver:
● check start location for transfer XLOG;
● connects to master and sends start position;
● receive XLOG and write it to disk;
● update variable in shared memory (WalRcv→receivedUpto);
● sends statistics + feedback.
Startup process uses the variable and replay WAL to this location.
WAL Receiver process.
Questions?
backend backend backend autovac ...
WAL writer
WAL
Startup/Recovery
SR/Archiving/Hot Standby
Point in Time
WAL Sender/WAL Receiver
Setup options.
Prepare master for replication.
Tools and utilities.
Start replication and verify results.
Specific options.
Part II. Replication setup.
Synchronous/Asynchronous replication.
Cascade configurations.
Uni-directional/Bi-directional.
Setup options.
Prepare the master.
Copy the DATADIR.
Prepare standbys.
Start standby.
Result verify.
Common logic.
Dedicated user for a replication.
Edit the postgresql.conf.
Edit the pg_hba.conf.
Create replication slots (if required).
Master setup.
Dedicated user for a replication.
● CREATE ROLE ... WITH LOGIN REPLICATION PASSWORD '…';
Edit the postgresql.conf.
Edit the pg_hba.conf.
Create replication slots (if required).
Master setup.
Dedicated user for a replication.
Edit the postgresql.conf.
● wal_level = hot_standby
● max_wal_senders > 0
● Restart the PostgreSQL
Edit the pg_hba.conf.
Create replication slots (if required).
Master setup.
Dedicated user for a replication.
Edit the postgresql.conf.
Edit the pg_hba.conf.
● host replication username client_addr/mask authtype
● host replication replica 10.1.0.99/32 md5
● pg_reload_conf()
Create replication slots (if required).
Master setup.
Dedicated user for a replication.
Edit the postgresql.conf.
Edit the pg_hba.conf.
Create replication slots (if required).
● max_replication_slots > 0
● pg_create_physical_replication_slot('slotname');
● primary_slot_name = 'slotname' (recovery.conf)
Master setup.
pg_basebackup (since 9.1)
-h, --host=...; -p, --port=...; -U, --username=...; -d, --dbname=...; -D, --pgdata=...
-c, --checkpoint=fast | spread
-X, --xlog-method=fetch | stream – stream с 9.2
-R, --write-recovery-conf – с 9.3
-r, --max-rate=… – с 9.4
--xlogdir=… – с 9.4
-T, --tablespace-mapping=olddir=newdir – с 9.4
-P, --progress
pg_basebackup -P -R -X stream -c fast -h 127.0.0.1 -U replica -D /pgdb
Copy the DATADIR.
Copy with cp, scp, tar, rsync...
Snapshots:
● ZFS send/receive;
● LVM + dd.
pg_start_backup() + pg_stop_backup().
Copy the DATADIR. pg_basebackup alternatives.
Configuration files should be the same.
Configuration files:
● postgresql.conf;
● recovery.conf.
Standby setup.
Configuration files should be the same:
● Why?
● How?
Configuration files:
● postgresql.conf;
● recovery.conf.
Standby setup.
Configuration files should be the same.
Configuration files (postgresql.conf):
● hot_standby = on;
● max_standby_streaming_delay;
● wal_receiver_status_interval;
● hot_standby_feedback;
● wal_receiver_timeout;
Standby setup.
Configuration files should be the same.
Configuration files (recovery.conf):
● primary_conninfo = 'host=… port=…'
● standby_mode = on
● primary_slot_name = 'slotname'
● trigger_file = '...'
● recovery_min_apply_delay.
Standby setup.
pg_ctl — PostgreSQL native utility.
pg_ctlcluster — pg_ctl perl wrapper in Debian/Ubuntu.
sysvinit, upstart, openrc, systemd…
Standby start.
wal sender and wal receiver processes.
Check postgres log.
Simple connection with psql.
pg_stat_replication view.
Check results.
DATADIR, configs and Debian-based vs. RHEL-based.
pg_ctlcluster and «unable to connect» errors.
To less processes in «ps» output.
Specific options.
Read scalability.
Full text search.
OLAP.
Standby is not a backup.
Resume. Practical purpose of the replication.
Questions.
root password: pgconf2016
# su - postgres – working under postgres account.
$ ps auxf – what we have?
$ pwd – where we are?
$ ls -l 9.5
Replication setup. Practice.
$ vi 9.5/data/postgresql.conf
● listen_addresses = '*' – Listen on all interfaces.
● wal_level = hot_standby – Set WAL verbose level.
● max_wal_senders = 4 – Limiting walsenders.
● hot_standby = on – Allow read-only queries on standby.
Master setup.
$ psql – Creare dedicated user.
CREATE ROLE replica WITH LOGIN REPLICATION PASSWORD 'rep123';
$ vi .pgpass – Setup password file.
*:*:*:replica:rep123
$ chmod 600 .pgpass
$ vi 9.5/data/pg_hba.conf – Add auth rules for replication user.
host replication replica 127.0.0.1/32 md5
$ pg_ctl -D 9.5/data/ -m fast restart – Apply the changes.
Master setup.
$ pg_basebackup -P -R -c fast -X stream -h 127.0.0.1 -U replica -D 9.5/replica
● -c fast — do the force checkpoint.
● -X stream — copy new XLOG through dedicated connection.
● -R — create minimal recovery.conf
$ vi 9.5/replica/postgresql.conf – edit port number.
port = 5433
$ pg_ctl -D 9.5/replica/ start – start this standby.
Create the standby.
$ ps auxf – wal sender/receiver process.
$ psql -p 5433 – check the status on the standby.
select pg_is_in_recovery(); – standby nust be in recovery mode.
$ psql – check the status on the master.
select * from pg_stat_replication ; – check statistics from standby.
Check result.
Yes, replication is ready.
Questions.
«Setup and forget» - this is about PostgreSQL streaming replication.
Monitoring:
● Internal statistics;
● auxiliary functions;
● queries examples.
Maintenance:
● add or remove standbys;
● pause replication;
● add or remove slots.
Part III. Monitoring and maintenance.
System views:
● pg_stat_replication
● pg_stat_replication_slots
Monitoring.
pg_is_in_recovery()
pg_current_xlog_location()
pg_last_xact_replay_timestamp()
pg_last_xlog_receive_location()
pg_last_xlog_replay_location()
pg_xlog_location_diff()
Monitoring. Auxiliary functions.
Replication monitoring on master.
select
pid, client_addr,
pg_size_pretty(pg_xlog_location_diff(pg_current_xlog_location(),sent_location)) as
pending_xlog,
pg_size_pretty(pg_xlog_location_diff(sent_location,write_location)) as write,
pg_size_pretty(pg_xlog_location_diff(write_location,flush_location)) as flush,
pg_size_pretty(pg_xlog_location_diff(flush_location,replay_location)) as replay,
pg_size_pretty(pg_xlog_location_diff(pg_current_xlog_location(),replay_location)) as
total_lag
from pg_stat_replication;
pid | client_addr | pending_xlog | write | flush | replay | total_lag
-------+---------------+--------------+----------+---------+------------+------------
21015 | 127.0.0.1 | 0 bytes | 0 bytes | 0 bytes | 48 bytes | 48 bytes
2067 | 192.168.200.4 | 12 GB | 30 MB | 0 bytes | 156 kB | 12 GB
18635 | 192.168.100.2 | 0 bytes | 48 bytes | 0 bytes | 590 MB | 590 MB
Monitoring. Queries examples.
On standby:
pg_current_xlog_location() → pg_last_xlog_receive_location()
WAL amount:
● SELECT pg_xlog_location_diff(pg_current_xlog_location, '0/0');
Lag in seconds:
● SELECT now() - pg_last_xact_replay_timestamp();
Monitoring.
Add new standbys or removing existing standbys.
Temporary pause replication.
Maintenance.
Add new standbys or removing existing standbys.
● max_wal_senders
● max_replication_slots
● pg_create_physical_replication_slot()
● pg_drop_replication_slot()
Temporary pause replication.
Maintenance.
Add new standbys or removing existing standbys.
Temporary pause replication:
● pg_is_xlog_replay_paused()
● pg_xlog_replay_pause()
● pg_xlog_replay_resume()
Maintenance.
Questions.
Replication lag.
Replication stopping.
Disk and network problems.
100% disk usage.
Recovery conflicts.
Tables and indexes bloat.
pg_xlog/ bloat.
Part IV. Problems.
Symptoms:
● Data between standby and master are differ.
Causes:
● Long queries on standby, much writes on master;
● Hardware issues.
Solutions:
● Application optimizations.
Replication lag.
Network lag:
● full_page_writes = off;
● ssh tunnels with compression.
Storage lag:
● full_page_writes =off;
● filesystem barriers;
● writethrough/writeback;
● RAID BBU learning;
● ionice (only for cfq elevator).
Networking and Storage.
Symptoms:
● Recovery process uses 100% CPU;
● Lag increasing.
Causes:
● Heavy update/delete, too many autovacuums.
Solutions:
● Increasing wal_keep_segments;
● Temporary disabling of full_page_writes;
● Set priorities with ionice and renice.
Replication stopping.
Causes:
● Replication slots and stopped standby → save XLOG segments;
Solutions:
● Remove the slot and use wal_keep_segments.
Dirty hack:
● Filesystem's reserved blocks percentage and tune2fs.
100% disk usage.
Why conflicts occurs:
● Autovacuum;
● XLOG replay.
Solutions:
● hot_standby_feedback = on;
● Increasing max_standby_streaming_delay.
Recovery conflicts.
Causes:
● Long transactions on a standby.
Solutions:
● pgstattuple;
● VACUUM FULL, pgcompacttable, pg_reorg...;
Tables and indexes bloat.
Symptoms:
● Different size pg_xlog/ and amount of XLOG segments.
Solutions:
● Decreasing checkpoint_timeout;
● Decreasing checkpoint_completion_target.
pg_xlog/ bloat on a standby.
Questions.
What is it?
For what is needed?
How to do it?
Part V. Switchover and Failover.
Switchover and Failover.
Purposes:
● Updates of software, operating system, or hardware.
● Hardware failures.
Prerequisites.
Run chekpoint on master.
Check replication lag.
Shutdown the master.
Remove recovery.conf and restart a standby.
Switchover.
Pros:
● Old master fast reuse;
● No lost transactions.
Cons:
● Warm cache after restart;
● pg_prewarm extension (since 9.4).
Switchover.
Create trigger file
● recovery.conf: trigger_file = '…'
● Need restart after recovery.conf changes.
With pg_ctl:
● pg_ctl -D ... promote
Failover.
Pros:
● It's fast;
● Don't need a restart;
● Don't need a cache warm.
Cons:
● Lost transactions risk;
● Old master should be reinitialized (until 9.5).
Failover.
Switchover:
● create recovery.conf and start.
Failover:
● reinit as standby (until 9.5);
● pg_rewind (since 9.5).
● timeline must be differs between master and standby.
● old master shut be shutdowned correctly.
● but sometimes issues occurs.
● pg_rewind --target-pgdata=9.5/main --source-server="host=10.0.0.1"
Old master reuse.
Questions.
$ vi 9.5/replica/postgresql.conf – edit configuration before restart.
port = 5432
$ mv 9.5/replica/recovery.conf /tmp/ – remove recovery.conf from DATADIR
$ psql
> CHECKPOINT; – reduce restart time.
$ pg_ctl -D 9.5/data -m fast stop – shutdown the master.
$ pg_ctl -D 9.5/replica -m fast restart – promote new master.
$ tail -f 9.5/replica/pg_log/postgresql-Wed.log
$ ps auxf
Switchover. Practice.
$ vi 9.5/data/postgresql.conf – edit config.
port = 5433
$ mv /tmp/recovery.conf 9.5/data/ – create recovery.conf.
$ pg_ctl -D 9.5/data start – start.
$ ps auxf – check.
Switchover. Reuse old master.
$ vi 9.5/data/postgresql.conf
$ vi 9.5/replica/postgresql.conf
● wal_log_hints = on – this options required for pg_rewind
● wal_keep_segments = 32
$ pg_ctl -D 9.5/data -m fast restart
$ pg_ctl -D 9.5/replica -m fast restart
Failover. Prepare.
$ pg_ctl -D 9.5/replica -m immediate stop – «crash» a master.
$ pg_ctl -D 9.5/data promote – promote a standby.
$ psql -p 5433 – «doing the changes».
create database test;
Failover.
$ pg_ctl -D 9.5/replica start
$ pg_ctl -D 9.5/replica stop
$ pg_rewind -D 9.5/replica --source-server="host=127.0.0.1 port=5433"
$ vi 9.5/replica/postgresql.conf
port = 5432
$ mv 9.5/replica/recovery.done 9.5/replica/recovery.conf
$ vi 9.5/replica/recovery.conf
port = 5433
$ pg_ctl -D 9.5/replica start
$ ps auxf
Failover. Reuse the old master.
Thanks.
Alexey Lesovsky, PostgreSQL Consulting.
lesovsky@pgco.me
Questions.

Contenu connexe

Tendances

PGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander Kukushkin
PGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander KukushkinPGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander Kukushkin
PGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander KukushkinEqunix Business Solutions
 
Kevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres Open
Kevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres OpenKevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres Open
Kevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres OpenPostgresOpen
 
Patroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyPatroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyAlexander Kukushkin
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1Federico Campoli
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HAharoonm
 
PostgreSQL Performance Tuning
PostgreSQL Performance TuningPostgreSQL Performance Tuning
PostgreSQL Performance Tuningelliando dias
 
Understanding PostgreSQL LW Locks
Understanding PostgreSQL LW LocksUnderstanding PostgreSQL LW Locks
Understanding PostgreSQL LW LocksJignesh Shah
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performancePostgreSQL-Consulting
 
Introduction VAUUM, Freezing, XID wraparound
Introduction VAUUM, Freezing, XID wraparoundIntroduction VAUUM, Freezing, XID wraparound
Introduction VAUUM, Freezing, XID wraparoundMasahiko Sawada
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015PostgreSQL-Consulting
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Alexey Lesovsky
 
Advanced Postgres Monitoring
Advanced Postgres MonitoringAdvanced Postgres Monitoring
Advanced Postgres MonitoringDenish Patel
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PGConf APAC
 
PostgreSQL - Haute disponibilité avec Patroni
PostgreSQL - Haute disponibilité avec PatroniPostgreSQL - Haute disponibilité avec Patroni
PostgreSQL - Haute disponibilité avec Patronislardiere
 
[pgday.Seoul 2022] PostgreSQL구조 - 윤성재
[pgday.Seoul 2022] PostgreSQL구조 - 윤성재[pgday.Seoul 2022] PostgreSQL구조 - 윤성재
[pgday.Seoul 2022] PostgreSQL구조 - 윤성재PgDay.Seoul
 
Postgresql Database Administration- Day3
Postgresql Database Administration- Day3Postgresql Database Administration- Day3
Postgresql Database Administration- Day3PoguttuezhiniVP
 
MySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitationsMySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitationsJean-François Gagné
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataJignesh Shah
 

Tendances (20)

PGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander Kukushkin
PGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander KukushkinPGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander Kukushkin
PGConf.ASIA 2019 Bali - Patroni in 2019 - Alexander Kukushkin
 
Kevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres Open
Kevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres OpenKevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres Open
Kevin Kempter PostgreSQL Backup and Recovery Methods @ Postgres Open
 
Patroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyPatroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easy
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
 
PostgreSQL Performance Tuning
PostgreSQL Performance TuningPostgreSQL Performance Tuning
PostgreSQL Performance Tuning
 
Understanding PostgreSQL LW Locks
Understanding PostgreSQL LW LocksUnderstanding PostgreSQL LW Locks
Understanding PostgreSQL LW Locks
 
PostgreSQL and RAM usage
PostgreSQL and RAM usagePostgreSQL and RAM usage
PostgreSQL and RAM usage
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performance
 
Introduction VAUUM, Freezing, XID wraparound
Introduction VAUUM, Freezing, XID wraparoundIntroduction VAUUM, Freezing, XID wraparound
Introduction VAUUM, Freezing, XID wraparound
 
PostgreSQL replication
PostgreSQL replicationPostgreSQL replication
PostgreSQL replication
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.
 
Advanced Postgres Monitoring
Advanced Postgres MonitoringAdvanced Postgres Monitoring
Advanced Postgres Monitoring
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs
 
PostgreSQL - Haute disponibilité avec Patroni
PostgreSQL - Haute disponibilité avec PatroniPostgreSQL - Haute disponibilité avec Patroni
PostgreSQL - Haute disponibilité avec Patroni
 
[pgday.Seoul 2022] PostgreSQL구조 - 윤성재
[pgday.Seoul 2022] PostgreSQL구조 - 윤성재[pgday.Seoul 2022] PostgreSQL구조 - 윤성재
[pgday.Seoul 2022] PostgreSQL구조 - 윤성재
 
Postgresql Database Administration- Day3
Postgresql Database Administration- Day3Postgresql Database Administration- Day3
Postgresql Database Administration- Day3
 
MySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitationsMySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitations
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
 

Similaire à Streaming replication in practice

Out of the box replication in postgres 9.4(pg confus)
Out of the box replication in postgres 9.4(pg confus)Out of the box replication in postgres 9.4(pg confus)
Out of the box replication in postgres 9.4(pg confus)Denish Patel
 
Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)Denish Patel
 
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...Command Prompt., Inc
 
Out of the Box Replication in Postgres 9.4(pgconfsf)
Out of the Box Replication in Postgres 9.4(pgconfsf)Out of the Box Replication in Postgres 9.4(pgconfsf)
Out of the Box Replication in Postgres 9.4(pgconfsf)Denish Patel
 
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...Puppet
 
Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)Denish Patel
 
Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)Denish Patel
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Denish Patel
 
OpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQLOpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQLOpen Gurukul
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniZalando Technology
 
configuring a warm standby, the easy way
configuring a warm standby, the easy wayconfiguring a warm standby, the easy way
configuring a warm standby, the easy wayCommand Prompt., Inc
 
PostgreSQL Replication in 10 Minutes - SCALE
PostgreSQL Replication in 10  Minutes - SCALEPostgreSQL Replication in 10  Minutes - SCALE
PostgreSQL Replication in 10 Minutes - SCALEPostgreSQL Experts, Inc.
 
The Essential postgresql.conf
The Essential postgresql.confThe Essential postgresql.conf
The Essential postgresql.confRobert Treat
 
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRest
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRestPGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRest
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRestPGDay.Amsterdam
 
Q2.12: Debugging with GDB
Q2.12: Debugging with GDBQ2.12: Debugging with GDB
Q2.12: Debugging with GDBLinaro
 
Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph Ceph Community
 
Replication using PostgreSQL Replicator
Replication using PostgreSQL ReplicatorReplication using PostgreSQL Replicator
Replication using PostgreSQL ReplicatorCommand Prompt., Inc
 

Similaire à Streaming replication in practice (20)

Out of the box replication in postgres 9.4(pg confus)
Out of the box replication in postgres 9.4(pg confus)Out of the box replication in postgres 9.4(pg confus)
Out of the box replication in postgres 9.4(pg confus)
 
Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)
 
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...
 
Out of the Box Replication in Postgres 9.4(pgconfsf)
Out of the Box Replication in Postgres 9.4(pgconfsf)Out of the Box Replication in Postgres 9.4(pgconfsf)
Out of the Box Replication in Postgres 9.4(pgconfsf)
 
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
 
Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)
 
Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)Out of the Box Replication in Postgres 9.4(PgCon)
Out of the Box Replication in Postgres 9.4(PgCon)
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4
 
OpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQLOpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQL
 
The Accidental DBA
The Accidental DBAThe Accidental DBA
The Accidental DBA
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando Patroni
 
configuring a warm standby, the easy way
configuring a warm standby, the easy wayconfiguring a warm standby, the easy way
configuring a warm standby, the easy way
 
PostgreSQL Replication in 10 Minutes - SCALE
PostgreSQL Replication in 10  Minutes - SCALEPostgreSQL Replication in 10  Minutes - SCALE
PostgreSQL Replication in 10 Minutes - SCALE
 
The Essential postgresql.conf
The Essential postgresql.confThe Essential postgresql.conf
The Essential postgresql.conf
 
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRest
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRestPGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRest
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRest
 
Q2.12: Debugging with GDB
Q2.12: Debugging with GDBQ2.12: Debugging with GDB
Q2.12: Debugging with GDB
 
Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph
 
9.1 Grand Tour
9.1 Grand Tour9.1 Grand Tour
9.1 Grand Tour
 
Replication using PostgreSQL Replicator
Replication using PostgreSQL ReplicatorReplication using PostgreSQL Replicator
Replication using PostgreSQL Replicator
 
Go replicator
Go replicatorGo replicator
Go replicator
 

Plus de Alexey Lesovsky

Отладка и устранение проблем в PostgreSQL Streaming Replication.
Отладка и устранение проблем в PostgreSQL Streaming Replication.Отладка и устранение проблем в PostgreSQL Streaming Replication.
Отладка и устранение проблем в PostgreSQL Streaming Replication.Alexey Lesovsky
 
Call of Postgres: Advanced Operations (part 5)
Call of Postgres: Advanced Operations (part 5)Call of Postgres: Advanced Operations (part 5)
Call of Postgres: Advanced Operations (part 5)Alexey Lesovsky
 
Call of Postgres: Advanced Operations (part 4)
Call of Postgres: Advanced Operations (part 4)Call of Postgres: Advanced Operations (part 4)
Call of Postgres: Advanced Operations (part 4)Alexey Lesovsky
 
Call of Postgres: Advanced Operations (part 3)
Call of Postgres: Advanced Operations (part 3)Call of Postgres: Advanced Operations (part 3)
Call of Postgres: Advanced Operations (part 3)Alexey Lesovsky
 
Call of Postgres: Advanced Operations (part 2)
Call of Postgres: Advanced Operations (part 2)Call of Postgres: Advanced Operations (part 2)
Call of Postgres: Advanced Operations (part 2)Alexey Lesovsky
 
Call of Postgres: Advanced Operations (part 1)
Call of Postgres: Advanced Operations (part 1)Call of Postgres: Advanced Operations (part 1)
Call of Postgres: Advanced Operations (part 1)Alexey Lesovsky
 
Troubleshooting PostgreSQL with pgCenter
Troubleshooting PostgreSQL with pgCenterTroubleshooting PostgreSQL with pgCenter
Troubleshooting PostgreSQL with pgCenterAlexey Lesovsky
 
PostgreSQL Streaming Replication
PostgreSQL Streaming ReplicationPostgreSQL Streaming Replication
PostgreSQL Streaming ReplicationAlexey Lesovsky
 
GitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons LearnedGitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons LearnedAlexey Lesovsky
 
PostgreSQL Vacuum: Nine Circles of Hell
PostgreSQL Vacuum: Nine Circles of HellPostgreSQL Vacuum: Nine Circles of Hell
PostgreSQL Vacuum: Nine Circles of HellAlexey Lesovsky
 
Tuning Linux for Databases.
Tuning Linux for Databases.Tuning Linux for Databases.
Tuning Linux for Databases.Alexey Lesovsky
 
Managing PostgreSQL with PgCenter
Managing PostgreSQL with PgCenterManaging PostgreSQL with PgCenter
Managing PostgreSQL with PgCenterAlexey Lesovsky
 
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL VacuumNine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL VacuumAlexey Lesovsky
 
Streaming replication in practice
Streaming replication in practiceStreaming replication in practice
Streaming replication in practiceAlexey Lesovsky
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Alexey Lesovsky
 
Highload 2014. PostgreSQL: ups, DevOps.
Highload 2014. PostgreSQL: ups, DevOps.Highload 2014. PostgreSQL: ups, DevOps.
Highload 2014. PostgreSQL: ups, DevOps.Alexey Lesovsky
 
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).Alexey Lesovsky
 
Linux tuning for PostgreSQL at Secon 2015
Linux tuning for PostgreSQL at Secon 2015Linux tuning for PostgreSQL at Secon 2015
Linux tuning for PostgreSQL at Secon 2015Alexey Lesovsky
 

Plus de Alexey Lesovsky (19)

Отладка и устранение проблем в PostgreSQL Streaming Replication.
Отладка и устранение проблем в PostgreSQL Streaming Replication.Отладка и устранение проблем в PostgreSQL Streaming Replication.
Отладка и устранение проблем в PostgreSQL Streaming Replication.
 
Call of Postgres: Advanced Operations (part 5)
Call of Postgres: Advanced Operations (part 5)Call of Postgres: Advanced Operations (part 5)
Call of Postgres: Advanced Operations (part 5)
 
Call of Postgres: Advanced Operations (part 4)
Call of Postgres: Advanced Operations (part 4)Call of Postgres: Advanced Operations (part 4)
Call of Postgres: Advanced Operations (part 4)
 
Call of Postgres: Advanced Operations (part 3)
Call of Postgres: Advanced Operations (part 3)Call of Postgres: Advanced Operations (part 3)
Call of Postgres: Advanced Operations (part 3)
 
Call of Postgres: Advanced Operations (part 2)
Call of Postgres: Advanced Operations (part 2)Call of Postgres: Advanced Operations (part 2)
Call of Postgres: Advanced Operations (part 2)
 
Call of Postgres: Advanced Operations (part 1)
Call of Postgres: Advanced Operations (part 1)Call of Postgres: Advanced Operations (part 1)
Call of Postgres: Advanced Operations (part 1)
 
Troubleshooting PostgreSQL with pgCenter
Troubleshooting PostgreSQL with pgCenterTroubleshooting PostgreSQL with pgCenter
Troubleshooting PostgreSQL with pgCenter
 
PostgreSQL Streaming Replication
PostgreSQL Streaming ReplicationPostgreSQL Streaming Replication
PostgreSQL Streaming Replication
 
GitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons LearnedGitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons Learned
 
PostgreSQL Vacuum: Nine Circles of Hell
PostgreSQL Vacuum: Nine Circles of HellPostgreSQL Vacuum: Nine Circles of Hell
PostgreSQL Vacuum: Nine Circles of Hell
 
Tuning Linux for Databases.
Tuning Linux for Databases.Tuning Linux for Databases.
Tuning Linux for Databases.
 
Managing PostgreSQL with PgCenter
Managing PostgreSQL with PgCenterManaging PostgreSQL with PgCenter
Managing PostgreSQL with PgCenter
 
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL VacuumNine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
 
Streaming replication in practice
Streaming replication in practiceStreaming replication in practice
Streaming replication in practice
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.
 
Pgcenter overview
Pgcenter overviewPgcenter overview
Pgcenter overview
 
Highload 2014. PostgreSQL: ups, DevOps.
Highload 2014. PostgreSQL: ups, DevOps.Highload 2014. PostgreSQL: ups, DevOps.
Highload 2014. PostgreSQL: ups, DevOps.
 
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
 
Linux tuning for PostgreSQL at Secon 2015
Linux tuning for PostgreSQL at Secon 2015Linux tuning for PostgreSQL at Secon 2015
Linux tuning for PostgreSQL at Secon 2015
 

Dernier

Power System electrical and electronics .pptx
Power System electrical and electronics .pptxPower System electrical and electronics .pptx
Power System electrical and electronics .pptxMUKULKUMAR210
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingMarian Marinov
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxSAJITHABANUS
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...soginsider
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxYogeshKumarKJMIT
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfNaveenVerma126
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging systemgokuldongala
 
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxVertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxLMW Machine Tool Division
 
A Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationA Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationMohsinKhanA
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfodunowoeminence2019
 
How to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfHow to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfRedhwan Qasem Shaddad
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Amil baba
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Amil baba
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxKISHAN KUMAR
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Akarthi keyan
 

Dernier (20)

Power System electrical and electronics .pptx
Power System electrical and electronics .pptxPower System electrical and electronics .pptx
Power System electrical and electronics .pptx
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & Logging
 
Lecture 2 .pdf
Lecture 2                           .pdfLecture 2                           .pdf
Lecture 2 .pdf
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
 
Litature Review: Research Paper work for Engineering
Litature Review: Research Paper work for EngineeringLitature Review: Research Paper work for Engineering
Litature Review: Research Paper work for Engineering
 
Présentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdfPrésentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdf
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptx
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging system
 
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxVertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
 
A Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationA Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software Simulation
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
 
How to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfHow to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdf
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptx
 
計劃趕得上變化
計劃趕得上變化計劃趕得上變化
計劃趕得上變化
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part A
 

Streaming replication in practice

  • 1. Streaming replication in practice. PgConf.Russia 2016, Moscow Lesovsky Alexey lesovsky@pgco.me
  • 2. Working stuff: http://goo.gl/Yy4UzH I. How streaming replication works. II. Replication setup. Practice. III. Monitoring and maintenance. IV. Possible problems. V. Switchiver и Failover. Practice. Agenda.
  • 3. What is replication and it kinds? Write Ahead Log. REDO and REDO realization in PostgreSQL. Common replication architecture in PostgreSQL. Description of processes that involved in replication. Part I. How replication works.
  • 4. Synchronize objects Objects changes are moving to each others. Replication maybe physical and logical. What is replication.
  • 5. Pros: ● Works between different major versions and architectures. ● Allow to replicate tables and tables sets. Cons: ● Hard to implement synchronous replication. ● CPU overhead (triggers, text conversions, etc). Examples: ● Slony, Londiste (Skytools), Bucardo, Pglogical. Logical replication.
  • 6. Pros: ● Resource usage minimal overhead. ● Easy setup, usage and maintenace. Cons: ● Standbys are only read-only. ● Can't work with different versions and architectures. ● Can't replicate tables and tables sets. Physical replication.
  • 7. Commit all changes in database (Durability in ACID). Flush all data from REDO buffers at COMMIT. REDO log has history of all changes in database. Any changes in database, written into REDO. REDO log usage: ● in crash recovery; ● backup and Point In Time Recovery; ● replication. Write Ahead Log. REDO.
  • 8. In PostgreSQL, REDO also known as Write Ahead Log (WAL). WAL guaranties that changes are commited before data will changed. How it guarantied: LSN (log sequence number) — record location (position) inside WAL; ● Any page is marked with LSN of last record that touched the page; ● Before page is written to disk, bufmgr must check that the WAL flushed to specified LSN. Write Ahead Log. REDO implementation in PostgreSQL.
  • 9. Common view. backend backend backend autovac ... WAL writer* WAL Startup/Recovery SR/Archiving/Hot Standby Point in Time WAL Sender/WAL Receiver * - может отсутствовать
  • 10. Main startup process task is run the database. In standby mode it initializes infinite replay loop. Read recovery.conf at REDO start. REDO: ● read segments from pg_xlog/archive; ● start wal receiver and reading XLOG from upstream server. When consistency point reached (min recovery ending location) allow connections and starts checkpointer/bgwriter. Processing all others parameters from recovery.conf. More details see in StartupXLOG() function. Startup process.
  • 11. For any client postmaster runs dedicated process — backend. WAL sender is backend too (it has am_walsender flag). This backend runs exec_replication_command(). exec_replication_command() can do various things: ● create/remove replication slots; ● start basebackup; ● start physical/logical replication. In last case, backend sends XLOG segments to the client. Or sleeps when no new XLOG. WAL Sender process.
  • 12. Startup process checks XLOG sources. Startup process init startup of WAL receivers. Need recovery.conf with primary_conninfo. WAL receiver: ● check start location for transfer XLOG; ● connects to master and sends start position; ● receive XLOG and write it to disk; ● update variable in shared memory (WalRcv→receivedUpto); ● sends statistics + feedback. Startup process uses the variable and replay WAL to this location. WAL Receiver process.
  • 13. Questions? backend backend backend autovac ... WAL writer WAL Startup/Recovery SR/Archiving/Hot Standby Point in Time WAL Sender/WAL Receiver
  • 14. Setup options. Prepare master for replication. Tools and utilities. Start replication and verify results. Specific options. Part II. Replication setup.
  • 16. Prepare the master. Copy the DATADIR. Prepare standbys. Start standby. Result verify. Common logic.
  • 17. Dedicated user for a replication. Edit the postgresql.conf. Edit the pg_hba.conf. Create replication slots (if required). Master setup.
  • 18. Dedicated user for a replication. ● CREATE ROLE ... WITH LOGIN REPLICATION PASSWORD '…'; Edit the postgresql.conf. Edit the pg_hba.conf. Create replication slots (if required). Master setup.
  • 19. Dedicated user for a replication. Edit the postgresql.conf. ● wal_level = hot_standby ● max_wal_senders > 0 ● Restart the PostgreSQL Edit the pg_hba.conf. Create replication slots (if required). Master setup.
  • 20. Dedicated user for a replication. Edit the postgresql.conf. Edit the pg_hba.conf. ● host replication username client_addr/mask authtype ● host replication replica 10.1.0.99/32 md5 ● pg_reload_conf() Create replication slots (if required). Master setup.
  • 21. Dedicated user for a replication. Edit the postgresql.conf. Edit the pg_hba.conf. Create replication slots (if required). ● max_replication_slots > 0 ● pg_create_physical_replication_slot('slotname'); ● primary_slot_name = 'slotname' (recovery.conf) Master setup.
  • 22. pg_basebackup (since 9.1) -h, --host=...; -p, --port=...; -U, --username=...; -d, --dbname=...; -D, --pgdata=... -c, --checkpoint=fast | spread -X, --xlog-method=fetch | stream – stream с 9.2 -R, --write-recovery-conf – с 9.3 -r, --max-rate=… – с 9.4 --xlogdir=… – с 9.4 -T, --tablespace-mapping=olddir=newdir – с 9.4 -P, --progress pg_basebackup -P -R -X stream -c fast -h 127.0.0.1 -U replica -D /pgdb Copy the DATADIR.
  • 23. Copy with cp, scp, tar, rsync... Snapshots: ● ZFS send/receive; ● LVM + dd. pg_start_backup() + pg_stop_backup(). Copy the DATADIR. pg_basebackup alternatives.
  • 24. Configuration files should be the same. Configuration files: ● postgresql.conf; ● recovery.conf. Standby setup.
  • 25. Configuration files should be the same: ● Why? ● How? Configuration files: ● postgresql.conf; ● recovery.conf. Standby setup.
  • 26. Configuration files should be the same. Configuration files (postgresql.conf): ● hot_standby = on; ● max_standby_streaming_delay; ● wal_receiver_status_interval; ● hot_standby_feedback; ● wal_receiver_timeout; Standby setup.
  • 27. Configuration files should be the same. Configuration files (recovery.conf): ● primary_conninfo = 'host=… port=…' ● standby_mode = on ● primary_slot_name = 'slotname' ● trigger_file = '...' ● recovery_min_apply_delay. Standby setup.
  • 28. pg_ctl — PostgreSQL native utility. pg_ctlcluster — pg_ctl perl wrapper in Debian/Ubuntu. sysvinit, upstart, openrc, systemd… Standby start.
  • 29. wal sender and wal receiver processes. Check postgres log. Simple connection with psql. pg_stat_replication view. Check results.
  • 30. DATADIR, configs and Debian-based vs. RHEL-based. pg_ctlcluster and «unable to connect» errors. To less processes in «ps» output. Specific options.
  • 31. Read scalability. Full text search. OLAP. Standby is not a backup. Resume. Practical purpose of the replication.
  • 33. root password: pgconf2016 # su - postgres – working under postgres account. $ ps auxf – what we have? $ pwd – where we are? $ ls -l 9.5 Replication setup. Practice.
  • 34. $ vi 9.5/data/postgresql.conf ● listen_addresses = '*' – Listen on all interfaces. ● wal_level = hot_standby – Set WAL verbose level. ● max_wal_senders = 4 – Limiting walsenders. ● hot_standby = on – Allow read-only queries on standby. Master setup.
  • 35. $ psql – Creare dedicated user. CREATE ROLE replica WITH LOGIN REPLICATION PASSWORD 'rep123'; $ vi .pgpass – Setup password file. *:*:*:replica:rep123 $ chmod 600 .pgpass $ vi 9.5/data/pg_hba.conf – Add auth rules for replication user. host replication replica 127.0.0.1/32 md5 $ pg_ctl -D 9.5/data/ -m fast restart – Apply the changes. Master setup.
  • 36. $ pg_basebackup -P -R -c fast -X stream -h 127.0.0.1 -U replica -D 9.5/replica ● -c fast — do the force checkpoint. ● -X stream — copy new XLOG through dedicated connection. ● -R — create minimal recovery.conf $ vi 9.5/replica/postgresql.conf – edit port number. port = 5433 $ pg_ctl -D 9.5/replica/ start – start this standby. Create the standby.
  • 37. $ ps auxf – wal sender/receiver process. $ psql -p 5433 – check the status on the standby. select pg_is_in_recovery(); – standby nust be in recovery mode. $ psql – check the status on the master. select * from pg_stat_replication ; – check statistics from standby. Check result.
  • 38. Yes, replication is ready. Questions.
  • 39. «Setup and forget» - this is about PostgreSQL streaming replication. Monitoring: ● Internal statistics; ● auxiliary functions; ● queries examples. Maintenance: ● add or remove standbys; ● pause replication; ● add or remove slots. Part III. Monitoring and maintenance.
  • 40. System views: ● pg_stat_replication ● pg_stat_replication_slots Monitoring.
  • 42. Replication monitoring on master. select pid, client_addr, pg_size_pretty(pg_xlog_location_diff(pg_current_xlog_location(),sent_location)) as pending_xlog, pg_size_pretty(pg_xlog_location_diff(sent_location,write_location)) as write, pg_size_pretty(pg_xlog_location_diff(write_location,flush_location)) as flush, pg_size_pretty(pg_xlog_location_diff(flush_location,replay_location)) as replay, pg_size_pretty(pg_xlog_location_diff(pg_current_xlog_location(),replay_location)) as total_lag from pg_stat_replication; pid | client_addr | pending_xlog | write | flush | replay | total_lag -------+---------------+--------------+----------+---------+------------+------------ 21015 | 127.0.0.1 | 0 bytes | 0 bytes | 0 bytes | 48 bytes | 48 bytes 2067 | 192.168.200.4 | 12 GB | 30 MB | 0 bytes | 156 kB | 12 GB 18635 | 192.168.100.2 | 0 bytes | 48 bytes | 0 bytes | 590 MB | 590 MB Monitoring. Queries examples.
  • 43. On standby: pg_current_xlog_location() → pg_last_xlog_receive_location() WAL amount: ● SELECT pg_xlog_location_diff(pg_current_xlog_location, '0/0'); Lag in seconds: ● SELECT now() - pg_last_xact_replay_timestamp(); Monitoring.
  • 44. Add new standbys or removing existing standbys. Temporary pause replication. Maintenance.
  • 45. Add new standbys or removing existing standbys. ● max_wal_senders ● max_replication_slots ● pg_create_physical_replication_slot() ● pg_drop_replication_slot() Temporary pause replication. Maintenance.
  • 46. Add new standbys or removing existing standbys. Temporary pause replication: ● pg_is_xlog_replay_paused() ● pg_xlog_replay_pause() ● pg_xlog_replay_resume() Maintenance.
  • 48. Replication lag. Replication stopping. Disk and network problems. 100% disk usage. Recovery conflicts. Tables and indexes bloat. pg_xlog/ bloat. Part IV. Problems.
  • 49. Symptoms: ● Data between standby and master are differ. Causes: ● Long queries on standby, much writes on master; ● Hardware issues. Solutions: ● Application optimizations. Replication lag.
  • 50. Network lag: ● full_page_writes = off; ● ssh tunnels with compression. Storage lag: ● full_page_writes =off; ● filesystem barriers; ● writethrough/writeback; ● RAID BBU learning; ● ionice (only for cfq elevator). Networking and Storage.
  • 51. Symptoms: ● Recovery process uses 100% CPU; ● Lag increasing. Causes: ● Heavy update/delete, too many autovacuums. Solutions: ● Increasing wal_keep_segments; ● Temporary disabling of full_page_writes; ● Set priorities with ionice and renice. Replication stopping.
  • 52. Causes: ● Replication slots and stopped standby → save XLOG segments; Solutions: ● Remove the slot and use wal_keep_segments. Dirty hack: ● Filesystem's reserved blocks percentage and tune2fs. 100% disk usage.
  • 53. Why conflicts occurs: ● Autovacuum; ● XLOG replay. Solutions: ● hot_standby_feedback = on; ● Increasing max_standby_streaming_delay. Recovery conflicts.
  • 54. Causes: ● Long transactions on a standby. Solutions: ● pgstattuple; ● VACUUM FULL, pgcompacttable, pg_reorg...; Tables and indexes bloat.
  • 55. Symptoms: ● Different size pg_xlog/ and amount of XLOG segments. Solutions: ● Decreasing checkpoint_timeout; ● Decreasing checkpoint_completion_target. pg_xlog/ bloat on a standby.
  • 57. What is it? For what is needed? How to do it? Part V. Switchover and Failover.
  • 58. Switchover and Failover. Purposes: ● Updates of software, operating system, or hardware. ● Hardware failures. Prerequisites.
  • 59. Run chekpoint on master. Check replication lag. Shutdown the master. Remove recovery.conf and restart a standby. Switchover.
  • 60. Pros: ● Old master fast reuse; ● No lost transactions. Cons: ● Warm cache after restart; ● pg_prewarm extension (since 9.4). Switchover.
  • 61. Create trigger file ● recovery.conf: trigger_file = '…' ● Need restart after recovery.conf changes. With pg_ctl: ● pg_ctl -D ... promote Failover.
  • 62. Pros: ● It's fast; ● Don't need a restart; ● Don't need a cache warm. Cons: ● Lost transactions risk; ● Old master should be reinitialized (until 9.5). Failover.
  • 63. Switchover: ● create recovery.conf and start. Failover: ● reinit as standby (until 9.5); ● pg_rewind (since 9.5). ● timeline must be differs between master and standby. ● old master shut be shutdowned correctly. ● but sometimes issues occurs. ● pg_rewind --target-pgdata=9.5/main --source-server="host=10.0.0.1" Old master reuse.
  • 65. $ vi 9.5/replica/postgresql.conf – edit configuration before restart. port = 5432 $ mv 9.5/replica/recovery.conf /tmp/ – remove recovery.conf from DATADIR $ psql > CHECKPOINT; – reduce restart time. $ pg_ctl -D 9.5/data -m fast stop – shutdown the master. $ pg_ctl -D 9.5/replica -m fast restart – promote new master. $ tail -f 9.5/replica/pg_log/postgresql-Wed.log $ ps auxf Switchover. Practice.
  • 66. $ vi 9.5/data/postgresql.conf – edit config. port = 5433 $ mv /tmp/recovery.conf 9.5/data/ – create recovery.conf. $ pg_ctl -D 9.5/data start – start. $ ps auxf – check. Switchover. Reuse old master.
  • 67. $ vi 9.5/data/postgresql.conf $ vi 9.5/replica/postgresql.conf ● wal_log_hints = on – this options required for pg_rewind ● wal_keep_segments = 32 $ pg_ctl -D 9.5/data -m fast restart $ pg_ctl -D 9.5/replica -m fast restart Failover. Prepare.
  • 68. $ pg_ctl -D 9.5/replica -m immediate stop – «crash» a master. $ pg_ctl -D 9.5/data promote – promote a standby. $ psql -p 5433 – «doing the changes». create database test; Failover.
  • 69. $ pg_ctl -D 9.5/replica start $ pg_ctl -D 9.5/replica stop $ pg_rewind -D 9.5/replica --source-server="host=127.0.0.1 port=5433" $ vi 9.5/replica/postgresql.conf port = 5432 $ mv 9.5/replica/recovery.done 9.5/replica/recovery.conf $ vi 9.5/replica/recovery.conf port = 5433 $ pg_ctl -D 9.5/replica start $ ps auxf Failover. Reuse the old master.
  • 70. Thanks. Alexey Lesovsky, PostgreSQL Consulting. lesovsky@pgco.me Questions.