Release v3.0.3 (#2719)

* Bump version
* Bump pyright version and fix newly reported issues
* Update release notes
* Fix typos, extend release process desc
* Add readthedocs configuration file v2
* Fix Dockerfile.citus files

.github/workflows/tests.yaml

@@ -173,4 +173,4 @@ jobs:
 
     - uses: jakebailey/pyright-action@v1
       with:
-        version: 1.1.309
+        version: 1.1.315

.readthedocs.yaml

@@ -0,0 +1,21 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+
+formats:
+  - epub
+  - pdf
+  - htmlzip

Dockerfile.citus

@@ -25,7 +25,7 @@ RUN set -ex \
             | grep -Ev '^python3-(sphinx|etcd|consul|kazoo|kubernetes)' \
             | xargs apt-get install -y vim curl less jq locales haproxy sudo \
                             python3-etcd python3-kazoo python3-pip busybox \
-                            net-tools iputils-ping --fix-missing \
+                            net-tools iputils-ping lsb-release --fix-missing \
     && if [ $(dpkg --print-architecture) = 'arm64' ]; then \
         apt-get install -y postgresql-server-dev-$PG_MAJOR \
                            git gcc make autoconf \

docs/releases.rst

@@ -3,6 +3,60 @@
 Release notes
 =============
 
+Version 3.0.3
+-------------
+
+**New features**
+
+- Compatibility with PostgreSQL 16 beta1 (Alexander Kukushkin)
+
+  Extended GUC's validator rules.
+
+- Make PostgreSQL GUC's validator extensible (Israel Barth Rubio)
+
+  Validator rules are loaded from YAML files located in ``patroni/postgresql/available_parameters/`` directory. Files are ordered in alphabetical order and applied one after another. It makes possible to have custom validators for non-standard Postgres distributions.
+
+- Added ``restapi.request_queue_size`` option (Andrey Zhidenkov)
+
+  Sets request queue size for TCP socket used by Patroni REST API. Once the queue is full, further requests get a "Connection denied" error. The default value is 5.
+
+- Call ``initdb`` directly when initializing a new cluster (Matt Baker)
+
+  Previously it was called via ``pg_ctl``, what required a special quoting of parameters passed to ``initdb``.
+
+- Added before stop hook (Le Duane)
+
+  The hook could be configured via ``postgresql.before_stop`` and is executed right before ``pg_ctl stop``. The exit code doesn't impact shutdown process.
+
+- Added support for custom Postgres binary names (Israel Barth Rubio, Polina Bungina)
+
+  When using a custom Postgres distribution it may be the case that the Postgres binaries are compiled with different names other than the ones used by the community Postgres distribution. Custom binary names could be configured using ``postgresql.bin_name.*`` and ``PATRONI_POSTGRESQL_BIN_*`` environment variables.
+
+
+**Improvements**
+
+- Various improvements of ``patroni --validate-config`` (Polina Bungina)
+
+  -  Make ``bootstrap.initdb`` optional. It is only required for new clusters, but ``patroni --validate-config`` was complaining if it was missing in the config.
+  -  Don't error out when ``postgresql.bin_dir`` is empty or not set. Try to first find Postgres binaries in the default PATH instead.
+  -  Make ``postgresql.authentication.rewind`` section optional. If it is missing, Patroni is using the superuser.
+
+- Improved error reporting in ``patronictl`` (Israel Barth Rubio)
+
+  The ``\n`` symbol was rendered as it is, instead of the actual newline symbol.
+
+
+**Bugfixes**
+
+- Fixed issue in Citus support (Alexander Kukushkin)
+
+  If the REST API call from the promoted worker to the coordinator failed during switchover it was leaving the given Citus group blocked during indefinite time.
+
+- Allow `etcd3` URL in `--dcs-url` option of `patronictl` (Israel Barth Rubio)
+
+  If users attempted to pass a `etcd3` URL through `--dcs-url` option of `patronictl` they would face an exception.
+
+
 Version 3.0.2
 -------------
 

kubernetes/Dockerfile.citus

@@ -7,7 +7,7 @@ RUN export DEBIAN_FRONTEND=noninteractive \
     && apt-get upgrade -y \
     && apt-cache depends patroni | sed -n -e 's/.* Depends: \(python3-.\+\)$/\1/p' \
             | grep -Ev '^python3-(sphinx|etcd|consul|kazoo|kubernetes)' \
-            | xargs apt-get install -y busybox vim-tiny curl jq less locales git python3-pip python3-wheel \
+            | xargs apt-get install -y busybox vim-tiny curl jq less locales git python3-pip python3-wheel lsb-release \
     ## Make sure we have a en_US.UTF-8 locale available
     && localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 \
     && echo "deb [signed-by=/etc/apt/trusted.gpg.d/citusdata_community.gpg] https://packagecloud.io/citusdata/community/debian/ $(lsb_release -cs) main" > /etc/apt/sources.list.d/citusdata_community.list \

patroni/dcs/consul.py

@@ -67,7 +67,7 @@ class HTTPClient(object):
         if ca_cert:
             kwargs['ca_certs'] = ca_cert
         kwargs['cert_reqs'] = ssl.CERT_REQUIRED if verify or ca_cert else ssl.CERT_NONE
-        self.http = urllib3.PoolManager(num_pools=10, maxsize=10, **kwargs)
+        self.http = urllib3.PoolManager(num_pools=10, maxsize=10, headers={}, **kwargs)
         self._ttl = 30
 
     def set_read_timeout(self, timeout: float) -> None:

patroni/dcs/kubernetes.py

@@ -75,7 +75,7 @@ class K8sConfig(object):
         pass
 
     def __init__(self) -> None:
-        self.pool_config: Dict[str, Union[str, int]] = {'maxsize': 10, 'num_pools': 10}  # urllib3.PoolManager config
+        self.pool_config: Dict[str, Any] = {'maxsize': 10, 'num_pools': 10}  # urllib3.PoolManager config
         self._token_expires_at = datetime.datetime.max
         self._headers: Dict[str, str] = {}
         self._make_headers()
@@ -277,12 +277,13 @@ class K8sClient(object):
 
         def _get_api_servers(self, api_servers_cache: List[str]) -> List[str]:
             _, per_node_timeout, per_node_retries = self._calculate_timeouts(len(api_servers_cache))
-            kwargs = {'headers': self._make_headers({}), 'preload_content': True, 'retries': per_node_retries,
+            headers = self._make_headers({})
+            kwargs = {'preload_content': True, 'retries': per_node_retries,
                       'timeout': urllib3.Timeout(connect=max(1.0, per_node_timeout / 2.0), total=per_node_timeout)}
             path = self._API_URL_PREFIX + 'default/endpoints/kubernetes'
             for base_uri in api_servers_cache:
                 try:
-                    response = self.pool_manager.request('GET', base_uri + path, **kwargs)
+                    response = self.pool_manager.request('GET', base_uri + path, headers=headers, **kwargs)
                     endpoint = self._handle_server_response(response, True)
                     if TYPE_CHECKING:  # pragma: no cover
                         assert isinstance(endpoint, K8sObject)

patroni/version.py

@@ -2,4 +2,4 @@
 
 :var __version__: the current Patroni version.
 """
-__version__ = '3.0.2'
+__version__ = '3.0.3'

release.sh

@@ -1,11 +1,12 @@
 #!/bin/bash
 
 # Release process:
-# 1. Open a PR that updates release notes and Patroni version
-# 2. Merge it
-# 3. Run release.sh
-# 4. After the new tag is pushed, the .github/workflows/release.yaml will run tests and upload the new package to test.pypi.org
-# 5. Once the release is created, the .github/workflows/release.yaml will run tests and upload the new package to pypi.org
+# 1. Open a PR that updates release notes, Patroni version and pyright version in the tests workflow.
+# 2. Resolve possible typing issues.
+# 3. Merge the PR.
+# 4. Run release.sh
+# 5. After the new tag is pushed, the .github/workflows/release.yaml will run tests and upload the new package to test.pypi.org
+# 6. Once the release is created, the .github/workflows/release.yaml will run tests and upload the new package to pypi.org
 
 ## Bail out on any non-zero exitcode from the called processes
 set -xe