Due to postgres --describe-config not showing GUCs defined as GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE, Patroni was always ignoring some GUCs that a user might want to have configured with non-default values.
- remove postgres --describe-config validation.
- define minor versions for availability bounds of some back-patched GUCs
Provide info about the PG parameters that caused "pending restart"
flag to be set. Both `patronictl list` and `/patroni` REST API endpoint
now show the parameters names and the diff as the "pending restart
reason".
When running `pg_basebackup` to bootstrap a replica, Patroni sanitizes
the custom user options that come from `postgresql.basebackup` configuration
section using the `process_user_options` method.
However, there is a bug in that method: it filters out not allowed options
that are in the format `- setting`, but not the ones in the format
`- setting: value` from `postgresql.basebackup`.
An example of that issue is the `dbname` setting. If you specify something
like this in the configuration file:
```yaml
postgresql:
basebackup:
- dbname: "host=RANDOM"
```
You end up with `--dbname` being specified twice for `pg_basebackup`, with
`--dbname='host=RANDOM'` taking precedence as it comes up later in the
command.
This commit fixes that issue by adding a `continue` statement when
the setting in format `- setting: value` is not allowed, thus skipping
it.
---------
Signed-off-by: Israel Barth Rubio <israel.barth@enterprisedb.com>
There are cases when Citus wants to have a connection to the local postgres. By default it uses `localhost` for that, which is not alwasy available. To solve it we will set `citus.local_hostname` GUC to custom value, which is the same as Patroni uses to connect to Postgres.
Make it hold connection kwargs for local connections and all `NamedConnection` objects use them automatically.
Also get rid of redundant `ConfigHandler.local_connect_kwargs`.
On top of that we will introduce a dedicated connection for the REST API thread.
* Use YAML files to validate Postgres GUCs through Patroni.
Patroni used to have a static list of Postgres GUCs validators in
`patroni.postgresql.validator`.
One problem with that approach, for example, is that it would not
allow GUCs from custom Postgres builds to be validated/accepted.
The idea that we had to work around that issue was to move the
validators from the source code to an external and extendable source.
With that Patroni will start reading the current validators from that
external source plus whatever custom validators are found.
From this commit onwards Patroni will read and parse all YAML files
that are found under the `patroni/postgresql/available_parameters`
directory to build its Postgres GUCs validation rules.
All the details about how this work can be found in the docstring
of the introduced function `_load_postgres_gucs_validators`.
Previously it was called via `pg_ctl`, what required a special quoting of parameters passed to `initdb`.
Co-authored-by: Israel <israel.barth@enterprisedb.com>
* Ignore D401 in flake8-docstrings
* Fix newly reported flake8 issues, ignore the old W503 rule
* rely on concatenation of adjecent strings
* Format behave scripts
* Reformat ha.py according to new rules
Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com>
It could happen that Patroni is started up before PGDATA was mounted. In this case Patroni can't determine major Postgres version from PG_VERSION file. Later, when PGDATA is mounted, Patroni was trying to create the recovery.conf even if the actual Postgres major version is newver than 12.
To mitigate the problem we double check that the `Postgresql._major_version` is set before writing recovery configuration or starting postgres up.
Close https://github.com/zalando/patroni/issues/2434
Not doing so makes it hard to implement callbacks in bash and eventually can lead to the situation when two callbacks are running at the same time. In case if we failed to kill the child process we will still wait for it to finish.
The same problem could happen with custom bootstrap, therefore if we happen to kill the custom bootstrap process we also kill all child subprocesses.
Closes https://github.com/zalando/patroni/issues/1238
Recently it has happened two times when people tried to deploy the new cluster but postgres data directory wasn't empty and also wasn't valid. In this case Patroni was still creating initialize key in DCS and trying to start the postgres up.
Now it will complain about non-empty invalid postgres data directory and exit.
Close https://github.com/zalando/patroni/issues/1216
the main purpose of this PR is simplifying #1068
It is mostly necessary for future support of pg12, where there will be no recovery.conf anymore, but `keep_existing_recovery_conf` parameter still needs to be supported due to backward compatibility.
* Convert postgresql.py into a package
* Factor out cancellable process into a separate class
* Factor out connection handler into a separate class
* Move postmaster into postgresql package
* Factor out pg_rewind into a separate class
* Factor out bootstrap into a separate class
* Factor out slots handler into a separate class
* Factor out postgresql config handler into a separate class
* Move callback_executor into postgresql package
This is just a careful refactoring, without code changes.
