We currently have a script named `patroni_barman_recover` in Patroni, which is intended to be used as a custom bootstrap method, or as a custom replica creation method.
Now there is need of one more Barman related script in Patroni to handle switching of config models in Barman upon `on_role_change` events.
However, instead of creating another Patroni script, let's say `patroni_barman_config_switch`, and duplicating a lot of logic in the code, we decided to refactor the code so:
* Instead of two separate scripts (`patroni_barman_recover` and `patroni_barman_config_switch`), we have a single script (`patroni_barman`) with 2 sub-commands (`recover` and `config-switch`)
This is the overview of changes that have been performed:
* File `patroni.scripts.barman_recover` has been removed, and its logic has been split into a few files:
* `patroni.scripts.barman.cli`: handles the entrypoint of the new `patroni_barman` command, exposing the argument parser and calling the appropriate functions depending on the sub-command
* `patroni.scripts.barman.utils`: implements utilitary enums, functions and classes wich can be used by `cli` and by sub-commands implementation:
* retry mechanism
* logging set up
* communication with pg-backup-api
* `patroni.scripts.barman.recover`: implements the `recover` sub-command only
* File `patroni.tests.test_barman_recover` has been renamed as `patroni.tests.test_barman`
* File `patroni.scripts.barman.config_switch` was created to implement the `config-switch` sub-command only
* `setup.py` has been changed so it generates a `patroni_barman` application instead of `patroni_barman_recover`
* Docs and unit tests were updated accordingly
References: PAT-154.
A contrib script, which can be used as a custom bootstrap method, or as a custom create replica method.
The script communicates with the pg-backup-api on the Barman node so Patroni is able to restore a Barman backup remotely.
The `--help` option of the script, along with the script docstring, should provide some context on how to use fill its parameters.
Patroni docs were updated accordingly to share examples about how to configure the script as a custom bootstrap method, or as a custom create replica method.
References: PAT-216.
Previous to this commit, if a user would ever like to add parameters to the custom bootstrap script call, they would need to configure Patroni like this:
```
bootstrap:
method: custom_method_name
custom_method_name:
command: /path/to/my/custom_script --arg1=value1 --arg2=value2 ...
```
This commit extends that so we achieve a similar behavior that is seen when using `create_replica_methods`, i.e., we also allow the following syntax:
```
bootstrap:
method: custom_method_name
custom_method_name:
command: /path/to/my/custom_script
arg1: value1
arg2: value2
```
All keys in the mapping which are not recognized by Patroni, will be dealt with as if they were additional named arguments to be passed down to the `command` call.
References: PAT-218.
This PR introduces a documentation page for `patronictl` application.
We adopted a top-down approach when writing this document. We start by describing the outer most parts, and then keep writing new sections that specialize the knowledge.
We basically added a section called `patronictl` to the left menu. Inside that section we created a page with this structure:
- `patronictl`: describes what it is
- `Configuraiton`: how to configure `patronictl`
- `Usage`: how to use the CLI. Inside this section, there are subsections for each of the subcommands exposed by `patronictl`, and each of them are described using the following subsubsections:
- `Synopsis`: syntax of the command and its positional and optional arguments
- `Description`: a description of what the command does
- `Parameters`: a detailed description of the arguments and how to use them
- `Examples`: one or more examples of execution of the command
References: PAT-200.
keep as much backward compatibility as possible.
Following changes were made:
1. All internal checks are performed as `role in ('master', 'primary')`
2. All internal variables/functions/methods are renamed
3. `GET /metrics` endpoint returns `patroni_primary` in addition to `patroni_master`.
4. Logs are changed to use leader/primary/member/remote depending on the context
5. Unit-tests are using only role = 'primary' instead of 'master' to verify that 1 works.
6. patronictl still supports old syntax, but also accepts `--leader` and `--primary`.
7. `master_(start|stop)_timeout` is automatically translated to `primary_(start|stop)_timeout` if the last one is not set.
8. updated the documentation and some examples
Future plan: in the next major release switch role name from `master` to `primary` and maybe drop `master` altogether.
The Kubernetes implementation will require more work and keep two labels in parallel. Label values should probably be configurable as described in https://github.com/zalando/patroni/issues/2495.
This adds a paragraph to the Standby Cluster section clarifying that the standby cluster is independent of the primary cluster and not visible from the primary cluster's Patroni interface.
Close#2090
If an existing instance was configured with WAL residing outside of
PGDATA then currently a 'reinit' would lose such symlinks. So add some
bits of information on that to draw attention to this cornercase issue
and also add the --waldir option to the sample `postgresql.basebackup`
configuration sections to increase visibility.
Discussion: https://github.com/zalando/patroni/issues/1817
* pgbackrest support
pgBackrest can restore in existing $PGDATA folder, this allows speedy restore as files which have not changed since last backup are skipped, to support this feature new param keep_data has been introduced. When keep_data=True, cleanup of $PGDATA will be skipped.
Patroni passes some extra parameters to custom_replica_methods when calling, this causes an error due to pgbackrest strict parameter checking. New param no_params=True can be set to skip parameters passing.
Fixes https://github.com/zalando/patroni/issues/625
Implementation of "standby cluster" described in #657. Standby cluster consists
of a "standby leader", that replicates from a "remote master" (which is not a
part of current patroni cluster and can be anywhere), and cascade replicas,
that replicate from the corresponding standby leader. "Standby leader" behaves
pretty much like a regular leader, which means that it holds a leader lock in
DSC, in case if disappears there will be an election of a new "standby
leader".
One can define such a cluster using the section "standby_cluster" in patroni
config file. This section provides parameters for standby cluster, that will be
applied only once during bootstrap and can be changed only through DSC.
pgBackRest's restore command generates the appropriate recovery.conf based
on the parameters you provide to pgBackRest. When calling pgBackRest's restore command
via Patroni's custom bootstrap, it deletes that recovery.conf. Specifying the recovery.conf
information in the patroni.yml is less than ideal. It prevent's leveraging pgBackRests
work to ensure recovery.conf files are properly generated. It also can lead to transient
config data in the patroni.yml under certain restore cases, such as a PITR restore
of Cluster B to Cluster A, where the restore_commnand in A needs to reference B.
The parameter is optional. The default behavior is to delete the recovery.conf.
Fixes https://github.com/zalando/patroni/issues/637
