diff --git a/docs/SETTINGS.rst b/docs/SETTINGS.rst index fac8a90f..84265de9 100644 --- a/docs/SETTINGS.rst +++ b/docs/SETTINGS.rst @@ -23,6 +23,10 @@ Bootstrap configuration - **use\_slots**: whether or not to use replication_slots. Must be False for PostgreSQL 9.3. You should comment out max_replication_slots before it becomes ineligible for leader status. - **recovery\_conf**: additional configuration settings written to recovery.conf when configuring follower. - **parameters**: list of configuration settings for Postgres. Many of these are required for replication to work. +- **method**: custom script to use for bootstrpapping this cluster. + See :ref:`custom bootstrap methods documentation ` for details. + When ``initdb`` is specified revert to the default ``initdb`` command. ``initdb`` is also triggered when no ``method`` + parameter is present in the configuration file. - **initdb**: List options to be passed on to initdb. - **- data-checksums**: Must be enabled when pg_rewind is needed on 9.3. - **- encoding: UTF8**: default encoding for new databases. @@ -80,7 +84,9 @@ PostgreSQL - **on\_start**: run this script when the cluster starts. - **on\_stop**: run this script when the cluster stops. - **connect\_address**: IP address + port through which Postgres is accessible from other nodes and applications. -- **create\_replica\_methods**: an ordered list of the create methods for turning a Patroni node into a new replica. "basebackup" is the default method; other methods are assumed to refer to scripts, each of which is configured as its own config item. +- **create\_replica\_method**: an ordered list of the create methods for turning a Patroni node into a new replica. + "basebackup" is the default method; other methods are assumed to refer to scripts, each of which is configured as its + own config item. See :ref:`custom replica creation methods documentation ` for further explanation. - **data\_dir**: The location of the Postgres data directory, either existing or to be initialized by Patroni. - **config\_dir**: The location of the Postgres configuration directory, defaults to the data directory. Must be writable by Patroni. - **bin\_dir**: Path to PostgreSQL binaries. (pg_ctl, pg_rewind, pg_basebackup, postgres) The default value is an empty string meaning that PATH environment variable will be used to find the executables. @@ -90,7 +96,7 @@ PostgreSQL - **recovery\_conf**: additional configuration settings written to recovery.conf when configuring follower. - **custom\_conf** : path to an optional custom ``postgresql.conf`` file, that will be used in place of ``postgresql.base.conf``. The file must exist on all cluster nodes, be readable by PostgreSQL and will be included from its location on the real ``postgresql.conf``. Note that Patroni will not monitor this file for changes, nor backup it. However, its settings can still be overriden by Patroni's own configuration facilities - see `dynamic configuration `__ for details. - **parameters**: list of configuration settings for Postgres. Many of these are required for replication to work. -- **pg\_hba**: list of lines that Patroni will use to generate ``pg_hba.conf``. This parameter has higher priority than ``bootstrap.pg_hba``. Together with `dynamic configuration `__ it simplifies management of ``pg_hba.conf``. +- **pg\_hba**: list of lines that Patroni will use to generate ``pg_hba.conf``. This parameter has higher priority than ``bootstrap.pg_hba``. Together with :ref:`dynamic configuration ` it simplifies management of ``pg_hba.conf``. - **- host all all 0.0.0.0/0 md5**. - **- host replication replicator 127.0.0.1/32 md5**: A line like this is required for replication. - **pg\_ctl\_timeout**: How long should pg_ctl wait when doing ``start``, ``stop`` or ``restart``. Default value is 60 seconds. diff --git a/docs/index.rst b/docs/index.rst index 6764b30d..f46babb9 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -21,6 +21,7 @@ We call Patroni a "template" because it is far from being a one-size-fits-all or dynamic_configuration ENVIRONMENT SETTINGS + replica_bootstrap replication_modes pause releases diff --git a/docs/replica_bootstrap.rst b/docs/replica_bootstrap.rst new file mode 100644 index 00000000..1aecf629 --- /dev/null +++ b/docs/replica_bootstrap.rst @@ -0,0 +1,98 @@ +Replica imaging and bootstrap +============================= + +Patroni allows customizing creation of a new replica. It also supports defining what happens when the new empty cluster +is being bootstrapped. The distinction between two is well defined: Patroni creates replicas only if the ``initialize`` +key is present in Etcd for the cluster. If there is no ``initialize`` key - Patroni calls bootstrap exclusively on the +first node that takes the initialize key lock. + +.. _custom_bootstrap: + +Bootstrap +--------- + +PostgreSQL provides ``initdb`` command to initialize a new cluster and Patroni calls it by default. In certain cases, +particularly when creating a new cluster as a copy of an existing one, it is necessary to replace a built-in method with +custom actions. Patroni supports executing user-defined scripts to bootstrap new clusters, supplying some required +arguments to them, i.e. the name of the cluster and the path to the data directory. This is configured in the +``bootstrap`` section of the Patroni configuration. For example: + +.. code:: YAML + + bootstrap: + method: + : + command: [param1 [, ...]] + recovery_conf: + recovery_target_action: promote + recovery_target_timeline: latest + restore_command: + + +Each bootstrap method must define at least a ``name`` and a ``command``. A special ``initdb`` method is available to trigger +the default behavior, in which case ``method`` parameter can be omitted altogether. The ``command`` can be specified using either +an absolute path, or the one relative to the ``patroni`` command location. In addition to the fixed parameters defined +in the configuration files, Patroni supplies two cluster-specific ones: + +--scope + Name of the cluster to be bootstrapped +--datadir + Path to the data directory of the cluster instance to be bootstrapped + +If the bootstrap script returns 0, Patroni tries to configure and start the PostgreSQL instance produced by it. If any +of the intermediate steps fail, or the script returns a non-zero value, Patroni assumes that the bootstrap has failed, +cleans up after itself and releases the initialize lock to give another node the opportunity to bootstrap. + +If a ``recovery_conf`` block is defined in the same section as the custom bootstrap method, Patroni will generate a +``recovery.conf`` before starting the newly bootstrapped instance. Typically, such recovery.conf should contain at least +one of the ``recovery_target_*`` parameters, together with the ``recovery_target_timeline`` set to ``promote``. + + .. note:: Bootstrap methods are neither chained, nor fallen-back to the default one in case the primary one fails + + +.. _custom_replica_creation: + +Building replicas +----------------- + +Patroni uses tried and proven ``pg_basebackup`` in order to create new replicas. One downside of it is that it requires +a running master node. Another one is the lack of 'on-the-fly' compression for the backup data and no built-in cleanup +for outdated backup files. Some people prefer other backup solutions, such as ``WAL-E``, ``pgBackRest``, ``Barman`` and +others, or simply roll their own scripts. In order to accommodate all those use-cases Patroni supports running custom +scripts to clone a new replica. Those are configured in the ``postgresql`` configuration block: + +.. code:: YAML + + postgresql: + create_replica_method: + - wal_e + - basebackup + wal_e: + command: patroni_wale_restore + no_master: 1 + envdir: {{WALE_ENV_DIR}} + use_iam: 1 + + +The ``create_replica_method`` defines available replica creation methods and the order of executing them. Patroni will +stop on the first one that returns 0. The basebackup is the built-in method and doesn't require any configuration. The +rest of the methods should define a separate section in the configuration file, listing the command to execute and any +custom parameters that should be passed to that command. All parameters will be passed in a ``--name=value`` format. +Besides user-defined parameters, Patroni supplies a couple of cluster-specific ones: + +--scope + Which cluster this replica belongs to +--datadir + Path to the data directory of the replica +--role + Always 'replica' +--connstring + Connection string to connect to the cluster member to clone from (master or other replica). The user in the + connection string can execute SQL and replication protocol commands. + +A special ``no_master`` parameter, if defined, allows Patroni to call the replica creation method even if there is no +running master or replicas. In that case, an empty string will be passed in a connection string. This is useful for +restoring the formerly running cluster from the binary backup. + +If all replica creation methods fail, Patroni will try again all methods in order during the next event loop cycle. + diff --git a/docs/replication_modes.rst b/docs/replication_modes.rst index 06acbce3..805735d7 100644 --- a/docs/replication_modes.rst +++ b/docs/replication_modes.rst @@ -44,7 +44,7 @@ When ``synchronous_mode`` is on and a standby crashes, commits will block until You can ensure that a standby never becomes the synchronous standby by setting ``nosync`` tag to true. This is recommended to set for standbys that are behind slow network connections and would cause performance degradation when becoming a synchronous standby. -Synchronous mode can be switched on and off via Patroni REST interface. See `dynamic configuration `__ for instructions. +Synchronous mode can be switched on and off via Patroni REST interface. See :ref:`dynamic configuration ` for instructions. Synchronous mode implementation