[feat] Hetzner

Add Hetzner CCM
Add Hetzner RobotLB

Signed-off-by: Ahmad Murzahmatov <gwynbleidd2106@yandex.com>

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,24 @@
+<!-- Thank you for making a contribution! Here are some tips for you:
+- Start the PR title with the [label] of Cozystack component:
+  - For system components: [platform], [system], [linstor], [cilium], [kube-ovn], [dashboard], [cluster-api], etc.
+  - For managed apps: [apps], [tenant], [kubernetes], [postgres], [virtual-machine] etc.
+  - For development and maintenance: [tests], [ci], [docs], [maintenance].
+- If it's a work in progress, consider creating this PR as a draft.
+- Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft.
+- Add the label `backport` if it's a bugfix that needs to be backported to a previous version.
+-->
+
+## What this PR does
+
+
+### Release note
+
+<!--  Write a release note:
+- Explain what has changed internally and for users.
+- Start with the same [label] as in the PR title
+- Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
+-->
+
+```release-note
+[]
+```

.github/workflows/pre-commit.yml

@@ -2,7 +2,7 @@ name: Pre-Commit Checks
 
 on:
   pull_request:
-    types: [labeled, opened, synchronize, reopened]
+    types: [opened, synchronize, reopened]
 
 concurrency:
   group: pre-commit-${{ github.workflow }}-${{ github.event.pull_request.number }}

.github/workflows/pull-requests-release.yaml

@@ -1,100 +1,17 @@
-name: Releasing PR
+name: "Releasing PR"
 
 on:
   pull_request:
-    types: [labeled, opened, synchronize, reopened, closed]
+    types: [closed]
+    paths-ignore:
+      - 'docs/**/*'
 
+# Cancel in‑flight runs for the same PR when a new push arrives.
 concurrency:
-  group: pull-requests-release-${{ github.workflow }}-${{ github.event.pull_request.number }}
+  group: pr-${{ github.workflow }}-${{ github.event.pull_request.number }}
   cancel-in-progress: true
 
 jobs:
-  verify:
-    name: Test Release
-    runs-on: [self-hosted]
-    permissions:
-      contents: read
-      packages: write
-
-    if: |
-      contains(github.event.pull_request.labels.*.name, 'release') &&
-      github.event.action != 'closed'
-
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-          fetch-tags: true
-
-      - name: Login to GitHub Container Registry
-        uses: docker/login-action@v3
-        with:
-          username: ${{ github.repository_owner }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-          registry: ghcr.io
-
-      - name: Extract tag from PR branch
-        id: get_tag
-        uses: actions/github-script@v7
-        with:
-          script: |
-            const branch = context.payload.pull_request.head.ref;
-            const m = branch.match(/^release-(\d+\.\d+\.\d+(?:[-\w\.]+)?)$/);
-            if (!m) {
-              core.setFailed(`❌ Branch '${branch}' does not match 'release-X.Y.Z[-suffix]'`);
-              return;
-            }
-            const tag = `v${m[1]}`;
-            core.setOutput('tag', tag);
-
-      - name: Find draft release and get asset IDs
-        id: fetch_assets
-        uses: actions/github-script@v7
-        with:
-          github-token: ${{ secrets.GH_PAT }}
-          script: |
-            const tag = '${{ steps.get_tag.outputs.tag }}';
-            const releases = await github.rest.repos.listReleases({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              per_page: 100
-            });
-            const draft = releases.data.find(r => r.tag_name === tag && r.draft);
-            if (!draft) {
-              core.setFailed(`Draft release '${tag}' not found`);
-              return;
-            }
-            const findAssetId = (name) =>
-              draft.assets.find(a => a.name === name)?.id;
-            const installerId = findAssetId("cozystack-installer.yaml");
-            const diskId = findAssetId("nocloud-amd64.raw.xz");
-            if (!installerId || !diskId) {
-              core.setFailed("Missing required assets");
-              return;
-            }
-            core.setOutput("installer_id", installerId);
-            core.setOutput("disk_id", diskId);
-      
-      - name: Download assets from GitHub API
-        run: |
-          mkdir -p _out/assets
-          curl -sSL \
-            -H "Authorization: token ${GH_PAT}" \
-            -H "Accept: application/octet-stream" \
-            -o _out/assets/cozystack-installer.yaml \
-            "https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/assets/${{ steps.fetch_assets.outputs.installer_id }}"
-          curl -sSL \
-            -H "Authorization: token ${GH_PAT}" \
-            -H "Accept: application/octet-stream" \
-            -o _out/assets/nocloud-amd64.raw.xz \
-            "https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/assets/${{ steps.fetch_assets.outputs.disk_id }}"
-        env:
-          GH_PAT: ${{ secrets.GH_PAT }}
-
-      - name: Run tests
-        run: make test
-
   finalize:
     name: Finalize Release
     runs-on: [self-hosted]

.github/workflows/pull-requests.yaml

@@ -2,10 +2,13 @@ name: Pull Request
 
 on:
   pull_request:
-    types: [labeled, opened, synchronize, reopened]
+    types: [opened, synchronize, reopened]
+    paths-ignore:
+      - 'docs/**/*'
 
+# Cancel in‑flight runs for the same PR when a new push arrives.
 concurrency:
-  group: pull-requests-${{ github.workflow }}-${{ github.event.pull_request.number }}
+  group: pr-${{ github.workflow }}-${{ github.event.pull_request.number }}
   cancel-in-progress: true
 
 jobs:
@@ -43,6 +46,17 @@ jobs:
 
       - name: Build Talos image
         run: make -C packages/core/installer talos-nocloud
+
+      - name: Save git diff as patch
+        if: "!contains(github.event.pull_request.labels.*.name, 'release')"
+        run: git diff HEAD > _out/assets/pr.patch
+
+      - name: Upload git diff patch
+        if: "!contains(github.event.pull_request.labels.*.name, 'release')"
+        uses: actions/upload-artifact@v4
+        with:
+          name: pr-patch
+          path: _out/assets/pr.patch
      
       - name: Upload installer
         uses: actions/upload-artifact@v4
@@ -55,93 +69,268 @@ jobs:
         with:
           name: talos-image
           path: _out/assets/nocloud-amd64.raw.xz
- 
-  prepare_env:
-    name: Prepare environment
-    runs-on: [self-hosted]
-    needs: build
 
-    # Never run when the PR carries the "release" label.
-    if: |
-      !contains(github.event.pull_request.labels.*.name, 'release')
+  resolve_assets:
+    name: "Resolve assets"
+    runs-on: ubuntu-latest
+    if: contains(github.event.pull_request.labels.*.name, 'release')
+    outputs:
+     installer_id: ${{ steps.fetch_assets.outputs.installer_id }}
+     disk_id:      ${{ steps.fetch_assets.outputs.disk_id }}
 
     steps:
       - name: Checkout code
+        if: contains(github.event.pull_request.labels.*.name, 'release')
         uses: actions/checkout@v4
         with:
           fetch-depth: 0
           fetch-tags: true
 
-      - name: Download installer
-        uses: actions/download-artifact@v4
+      - name: Extract tag from PR branch (release PR)
+        if: contains(github.event.pull_request.labels.*.name, 'release')
+        id: get_tag
+        uses: actions/github-script@v7
         with:
-          name: cozystack-installer
-          path: _out/assets/
+          script: |
+            const branch = context.payload.pull_request.head.ref;
+            const m = branch.match(/^release-(\d+\.\d+\.\d+(?:[-\w\.]+)?)$/);
+            if (!m) {
+              core.setFailed(`❌ Branch '${branch}' does not match 'release-X.Y.Z[-suffix]'`);
+              return;
+            }
+            core.setOutput('tag', `v${m[1]}`);
 
-      - name: Download Talos image
+      - name: Find draft release & asset IDs (release PR)
+        if: contains(github.event.pull_request.labels.*.name, 'release')
+        id: fetch_assets
+        uses: actions/github-script@v7
+        with:
+          github-token: ${{ secrets.GH_PAT }}
+          script: |
+            const tag = '${{ steps.get_tag.outputs.tag }}';
+            const releases = await github.rest.repos.listReleases({
+              owner: context.repo.owner,
+              repo:  context.repo.repo,
+              per_page: 100
+            });
+            const draft = releases.data.find(r => r.tag_name === tag && r.draft);
+            if (!draft) {
+              core.setFailed(`Draft release '${tag}' not found`);
+              return;
+            }
+            const find = (n) => draft.assets.find(a => a.name === n)?.id;
+            const installerId = find('cozystack-installer.yaml');
+            const diskId      = find('nocloud-amd64.raw.xz');
+            if (!installerId || !diskId) {
+              core.setFailed('Required assets missing in draft release');
+              return;
+            }
+            core.setOutput('installer_id', installerId);
+            core.setOutput('disk_id',      diskId);
+
+
+  prepare_env:
+    name: "Prepare environment"
+    runs-on: [self-hosted]
+    permissions:
+      contents: read
+      packages: read
+    needs: ["build", "resolve_assets"]
+    if: ${{ always() && (needs.build.result == 'success' || needs.resolve_assets.result == 'success') }}
+
+    steps:
+      # ▸ Checkout and prepare the codebase
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      # ▸ Regular PR path – download artefacts produced by the *build* job
+      - name: "Download Talos image (regular PR)"
+        if: "!contains(github.event.pull_request.labels.*.name, 'release')"
         uses: actions/download-artifact@v4
         with:
           name: talos-image
-          path: _out/assets/
+          path: _out/assets
+
+      - name: Download PR patch
+        if: "!contains(github.event.pull_request.labels.*.name, 'release')"
+        uses: actions/download-artifact@v4
+        with:
+          name: pr-patch
+          path: _out/assets
+
+      - name: Apply patch
+        if: "!contains(github.event.pull_request.labels.*.name, 'release')"
+        run: |
+          git apply _out/assets/pr.patch
+
+      # ▸ Release PR path – fetch artefacts from the corresponding draft release
+      - name: Download assets from draft release (release PR)
+        if: contains(github.event.pull_request.labels.*.name, 'release')
+        run: |
+          mkdir -p _out/assets
+          curl -sSL -H "Authorization: token ${GH_PAT}" -H "Accept: application/octet-stream" \
+            -o _out/assets/nocloud-amd64.raw.xz \
+            "https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/assets/${{ needs.resolve_assets.outputs.disk_id }}"
+        env:
+          GH_PAT: ${{ secrets.GH_PAT }}
 
       - name: Set sandbox ID
         run: echo "SANDBOX_NAME=cozy-e2e-sandbox-$(echo "${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}" | sha256sum | cut -c1-10)" >> $GITHUB_ENV
 
+      # ▸ Start actual job steps
+      - name: Prepare workspace
+        run: |
+          rm -rf /tmp/$SANDBOX_NAME
+          cp -r ${{ github.workspace }} /tmp/$SANDBOX_NAME
+
       - name: Prepare environment
-        run: make SANDBOX_NAME=$SANDBOX_NAME prepare-env
+        run: |
+          cd /tmp/$SANDBOX_NAME
+          attempt=0
+          until make SANDBOX_NAME=$SANDBOX_NAME prepare-env; do
+            attempt=$((attempt + 1))
+            if [ $attempt -ge 3 ]; then
+              echo "❌ Attempt $attempt failed, exiting..."
+              exit 1
+            fi
+            echo "❌ Attempt $attempt failed, retrying..."
+          done
+          echo "✅ The task completed successfully after $attempt attempts"
 
   install_cozystack:
-    name: Install Cozystack
+    name: "Install Cozystack"
     runs-on: [self-hosted]
-    needs: prepare_env
-
-    # Never run when the PR carries the "release" label.
-    if: |
-      !contains(github.event.pull_request.labels.*.name, 'release')
+    permissions:
+      contents: read
+      packages: read
+    needs: ["prepare_env", "resolve_assets"]
+    if: ${{ always() && needs.prepare_env.result == 'success' }}
 
     steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-          fetch-tags: true
+      - name: Prepare _out/assets directory
+        run: mkdir -p _out/assets
 
+      # ▸ Regular PR path – download artefacts produced by the *build* job
+      - name: "Download installer (regular PR)"
+        if: "!contains(github.event.pull_request.labels.*.name, 'release')"
+        uses: actions/download-artifact@v4
+        with:
+          name: cozystack-installer
+          path: _out/assets
+
+      # ▸ Release PR path – fetch artefacts from the corresponding draft release
+      - name: Download assets from draft release (release PR)
+        if: contains(github.event.pull_request.labels.*.name, 'release')
+        run: |
+          mkdir -p _out/assets
+          curl -sSL -H "Authorization: token ${GH_PAT}" -H "Accept: application/octet-stream" \
+            -o _out/assets/cozystack-installer.yaml \
+            "https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/assets/${{ needs.resolve_assets.outputs.installer_id }}"
+        env:
+          GH_PAT: ${{ secrets.GH_PAT }}
+ 
+      # ▸ Start actual job steps
       - name: Set sandbox ID
         run: echo "SANDBOX_NAME=cozy-e2e-sandbox-$(echo "${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}" | sha256sum | cut -c1-10)" >> $GITHUB_ENV
 
-      - name: Install Cozystack
-        run: make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME install-cozystack
+      - name: Sync _out/assets directory
+        run: |
+          mkdir -p /tmp/$SANDBOX_NAME/_out/assets
+          mv _out/assets/* /tmp/$SANDBOX_NAME/_out/assets/
 
-  test_apps:
-    name: Test applications
-    runs-on: [self-hosted]
-    needs: install_cozystack
+      - name: Install Cozystack into sandbox
+        run: |
+          cd /tmp/$SANDBOX_NAME
+          attempt=0
+          until make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME install-cozystack; do
+            attempt=$((attempt + 1))
+            if [ $attempt -ge 3 ]; then
+              echo "❌ Attempt $attempt failed, exiting..."
+              exit 1
+            fi
+            echo "❌ Attempt $attempt failed, retrying..."
+          done
+          echo "✅ The task completed successfully after $attempt attempts."
 
-    # Never run when the PR carries the "release" label.
-    if: |
-      !contains(github.event.pull_request.labels.*.name, 'release')
+  detect_test_matrix:
+    name: "Detect e2e test matrix"
+    runs-on: ubuntu-latest
+    outputs:
+      matrix: ${{ steps.set.outputs.matrix }}
 
     steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-          fetch-tags: true
+      - uses: actions/checkout@v4
+      - id: set
+        run: |
+          apps=$(find hack/e2e-apps -maxdepth 1 -mindepth 1 -name '*.bats' | \
+            awk -F/ '{sub(/\..+/, "", $NF); print $NF}' | jq -R . | jq -cs .)
+          echo "matrix={\"app\":$apps}" >> "$GITHUB_OUTPUT"
 
+  test_apps:
+    strategy:
+      matrix: ${{ fromJson(needs.detect_test_matrix.outputs.matrix) }}
+    name: Test ${{ matrix.app }}
+    runs-on: [self-hosted]
+    needs: [install_cozystack,detect_test_matrix]
+    if: ${{ always() && (needs.install_cozystack.result == 'success' && needs.detect_test_matrix.result == 'success') }}
+
+    steps:
       - name: Set sandbox ID
         run: echo "SANDBOX_NAME=cozy-e2e-sandbox-$(echo "${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}" | sha256sum | cut -c1-10)" >> $GITHUB_ENV
 
       - name: E2E Apps
-        run: make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME test-apps
+        run: |
+          cd /tmp/$SANDBOX_NAME
+          attempt=0
+          until make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME test-apps-${{ matrix.app }}; do
+            attempt=$((attempt + 1))
+            if [ $attempt -ge 3 ]; then
+              echo "❌ Attempt $attempt failed, exiting..."
+              exit 1
+            fi
+            echo "❌ Attempt $attempt failed, retrying..."
+          done
+          echo "✅ The task completed successfully after $attempt attempts"
+
+  collect_debug_information:
+    name: Collect debug information
+    runs-on: [self-hosted]
+    needs: [test_apps]
+    if: ${{ always() }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set sandbox ID
+        run: echo "SANDBOX_NAME=cozy-e2e-sandbox-$(echo "${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}" | sha256sum | cut -c1-10)" >> $GITHUB_ENV
+
+      - name: Collect report
+        run: |
+          cd /tmp/$SANDBOX_NAME
+          make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME collect-report
+
+      - name: Upload cozyreport.tgz
+        uses: actions/upload-artifact@v4
+        with:
+          name: cozyreport
+          path: /tmp/${{ env.SANDBOX_NAME }}/_out/cozyreport.tgz
+
+      - name: Collect images list
+        run: |
+          cd /tmp/$SANDBOX_NAME
+          make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME collect-images
+
+      - name: Upload image list
+        uses: actions/upload-artifact@v4
+        with:
+          name: image-list
+          path: /tmp/${{ env.SANDBOX_NAME }}/_out/images.txt
 
   cleanup:
     name: Tear down environment
     runs-on: [self-hosted]
-    needs: test_apps
-
-    # Never run when the PR carries the "release" label.
-    if: |
-      !contains(github.event.pull_request.labels.*.name, 'release')
+    needs: [collect_debug_information]
+    if: ${{ always() && needs.test_apps.result == 'success' }}
 
     steps:
       - name: Checkout code
@@ -153,5 +342,10 @@ jobs:
       - name: Set sandbox ID
         run: echo "SANDBOX_NAME=cozy-e2e-sandbox-$(echo "${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}" | sha256sum | cut -c1-10)" >> $GITHUB_ENV
 
-      - name: E2E Apps
+      - name: Tear down sandbox
         run: make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME delete
+
+      - name: Remove workspace
+        run: rm -rf /tmp/$SANDBOX_NAME
+
+

.github/workflows/tags.yaml

@@ -112,9 +112,13 @@ jobs:
       # Commit built artifacts
       - name: Commit release artifacts
         if: steps.check_release.outputs.skip == 'false'
+        env:
+          GH_PAT: ${{ secrets.GH_PAT }}
         run: |
-          git config user.name  "github-actions"
-          git config user.email "github-actions@github.com"
+          git config user.name  "cozystack-bot"
+          git config user.email "217169706+cozystack-bot@users.noreply.github.com"
+          git remote set-url origin https://cozystack-bot:${GH_PAT}@github.com/${GITHUB_REPOSITORY}
+          git config --unset-all http.https://github.com/.extraheader || true
           git add .
           git commit -m "Prepare release ${GITHUB_REF#refs/tags/}" -s || echo "No changes to commit"
           git push origin HEAD || true
@@ -189,7 +193,12 @@ jobs:
       # Create release-X.Y.Z branch and push (force-update)
       - name: Create release branch
         if: steps.check_release.outputs.skip == 'false'
+        env:
+          GH_PAT: ${{ secrets.GH_PAT }}
         run: |
+          git config user.name  "cozystack-bot"
+          git config user.email "217169706+cozystack-bot@users.noreply.github.com"
+          git remote set-url origin https://cozystack-bot:${GH_PAT}@github.com/${GITHUB_REPOSITORY}
           BRANCH="release-${GITHUB_REF#refs/tags/v}"
           git branch -f "$BRANCH"
           git push -f origin "$BRANCH"
@@ -199,6 +208,7 @@ jobs:
         if: steps.check_release.outputs.skip == 'false'
         uses: actions/github-script@v7
         with:
+          github-token: ${{ secrets.GH_PAT }}
           script: |
             const version = context.ref.replace('refs/tags/v', '');
             const base    = '${{ steps.get_base.outputs.branch }}';

.pre-commit-config.yaml

@@ -12,10 +12,10 @@ repos:
       name: Run 'make generate' in all app directories
       entry: |
         /bin/bash -c '
-          for dir in ./packages/apps/*/; do
+          for dir in ./packages/apps/*/ ./packages/extra/*/ ./packages/system/cozystack-api/; do
             if [ -d "$dir" ]; then
               echo "Running make generate in $dir"
-              (cd "$dir" && make generate)
+              make generate -C "$dir"
             fi
           done
           git diff --color=always | cat

Makefile

@@ -9,7 +9,6 @@ build-deps:
 
 build: build-deps
 	make -C packages/apps/http-cache image
-	make -C packages/apps/postgres image
 	make -C packages/apps/mysql image
 	make -C packages/apps/clickhouse image
 	make -C packages/apps/kubernetes image

docs/changelogs/template.md

@@ -0,0 +1,11 @@
+## Major Features and Improvements
+
+## Security
+
+## Fixes
+
+## Dependencies
+
+## Documentation
+
+## Development, Testing, and CI/CD

docs/changelogs/v0.31.1.md

@@ -0,0 +1,8 @@
+## Fixes
+
+* [build] Update Talos Linux v1.10.3 and fix assets. (@kvaps in https://github.com/cozystack/cozystack/pull/1006)
+* [ci] Fix uploading released artifacts to GitHub. (@kvaps in https://github.com/cozystack/cozystack/pull/1009)
+* [ci] Separate build and testing jobs. (@kvaps in https://github.com/cozystack/cozystack/pull/1005)
+* [docs] Write a full release post for v0.31.1. (@NickVolynkin in https://github.com/cozystack/cozystack/pull/999)
+
+**Full Changelog**: https://github.com/cozystack/cozystack/compare/v0.31.0...v0.31.1

docs/changelogs/v0.31.2.md

@@ -0,0 +1,12 @@
+## Security
+
+* Resolve a security problem that allowed a tenant administrator to gain enhanced privileges outside the tenant. (@kvaps in https://github.com/cozystack/cozystack/pull/1062, backported in https://github.com/cozystack/cozystack/pull/1066)
+
+## Fixes
+
+* [platform] Fix dependencies in `distro-full` bundle. (@klinch0 in  https://github.com/cozystack/cozystack/pull/1056, backported in https://github.com/cozystack/cozystack/pull/1064)
+* [platform] Fix RBAC for annotating namespaces. (@kvaps in https://github.com/cozystack/cozystack/pull/1031, backported in https://github.com/cozystack/cozystack/pull/1037)
+* [platform] Reduce system resource consumption by using smaller resource presets for VerticalPodAutoscaler, SeaweedFS, and KubeOVN. (@klinch0 in https://github.com/cozystack/cozystack/pull/1054, backported in https://github.com/cozystack/cozystack/pull/1058)
+* [dashboard] Fix a number of issues in the Cozystack Dashboard (@kvaps in https://github.com/cozystack/cozystack/pull/1042, backported in https://github.com/cozystack/cozystack/pull/1066)
+* [apps] Specify minimal working resource presets. (@kvaps in https://github.com/cozystack/cozystack/pull/1040, backported in https://github.com/cozystack/cozystack/pull/1041)
+* [apps] Update built-in documentation and configuration reference for managed Clickhouse application. (@NickVolynkin in https://github.com/cozystack/cozystack/pull/1059, backported in https://github.com/cozystack/cozystack/pull/1065)

docs/changelogs/v0.32.0.md

@@ -0,0 +1,71 @@
+Cozystack v0.32.0 is a significant release that brings new features, key fixes, and updates to underlying components.
+
+## Major Features and Improvements
+
+* [platform] Use `cozypkg` instead of Helm (@kvaps in https://github.com/cozystack/cozystack/pull/1057)
+* [platform] Introduce the HelmRelease reconciler for system components. (@kvaps in https://github.com/cozystack/cozystack/pull/1033)
+* [kubernetes] Enable using container registry mirrors by tenant Kubernetes clusters. Configure containerd for tenant Kubernetes clusters. (@klinch0 in https://github.com/cozystack/cozystack/pull/979, patched by @lllamnyp in https://github.com/cozystack/cozystack/pull/1032)
+* [platform] Allow users to specify CPU requests in VCPUs. Use a library chart for resource management. (@lllamnyp in https://github.com/cozystack/cozystack/pull/972 and https://github.com/cozystack/cozystack/pull/1025)
+* [platform] Annotate all child objects of apps with uniform labels for tracking by WorkloadMonitors. (@lllamnyp in https://github.com/cozystack/cozystack/pull/1018 and https://github.com/cozystack/cozystack/pull/1024)
+* [platform] Introduce `cluster-domain` option and un-hardcode `cozy.local`. (@kvaps in https://github.com/cozystack/cozystack/pull/1039)
+* [platform] Get instance type when reconciling WorkloadMonitor (https://github.com/cozystack/cozystack/pull/1030)
+* [virtual-machine] Add RBAC rules to allow port forwarding in KubeVirt for SSH via `virtctl`. (@mattia-eleuteri in https://github.com/cozystack/cozystack/pull/1027, patched by @klinch0 in https://github.com/cozystack/cozystack/pull/1028)
+* [monitoring] Add events and audit inputs (@kevin880202 in https://github.com/cozystack/cozystack/pull/948)
+
+## Security
+
+* Resolve a security problem that allowed tenant administrator to gain enhanced privileges outside the tenant. (@kvaps in https://github.com/cozystack/cozystack/pull/1062)
+
+## Fixes
+
+* [dashboard] Fix a number of issues in the Cozystack Dashboard (@kvaps in https://github.com/cozystack/cozystack/pull/1042)
+* [kafka] Specify minimal working resource presets. (@kvaps in https://github.com/cozystack/cozystack/pull/1040)
+* [cilium] Fixed Gateway API manifest. (@zdenekjanda in https://github.com/cozystack/cozystack/pull/1016)
+* [platform] Fix RBAC for annotating namespaces. (@kvaps in https://github.com/cozystack/cozystack/pull/1031)
+* [platform] Fix dependencies for paas-hosted bundle. (@kvaps in https://github.com/cozystack/cozystack/pull/1034)
+* [platform] Reduce system resource consumption by using lesser resource presets for VerticalPodAutoscaler, SeaweedFS, and KubeOVN. (@klinch0 in https://github.com/cozystack/cozystack/pull/1054)
+* [virtual-machine] Fix handling of cloudinit and ssh-key input for `virtual-machine` and `vm-instance` applications. (@gwynbleidd2106 in https://github.com/cozystack/cozystack/pull/1019 and https://github.com/cozystack/cozystack/pull/1020)
+* [apps] Fix Clickhouse version parsing. (@kvaps in https://github.com/cozystack/cozystack/commit/28302e776e9d2bb8f424cf467619fa61d71ac49a)
+* [apps] Add resource quotas for PostgreSQL jobs and fix application readme generation check in CI. (@klinch0 in https://github.com/cozystack/cozystack/pull/1051)
+* [kube-ovn] Enable database health check. (@kvaps in https://github.com/cozystack/cozystack/pull/1047)
+* [kubernetes] Fix upstream issue by updating Kubevirt-CCM. (@kvaps in https://github.com/cozystack/cozystack/pull/1052)
+* [kubernetes] Fix resources and introduce a migration when upgrading tenant Kubernetes to v0.32.4. (@kvaps in https://github.com/cozystack/cozystack/pull/1073)
+* [cluster-api] Add a missing migration for `capi-providers`. (@kvaps in https://github.com/cozystack/cozystack/pull/1072)
+
+## Dependencies
+
+* Introduce cozykpg, update to v1.1.0. (@kvaps in https://github.com/cozystack/cozystack/pull/1057 and https://github.com/cozystack/cozystack/pull/1063)
+* Update flux-operator to 0.22.0, Flux to 2.6.x. (@kingdonb in https://github.com/cozystack/cozystack/pull/1035)
+* Update Talos Linux to v1.10.3. (@kvaps in https://github.com/cozystack/cozystack/pull/1006)
+* Update Cilium to v1.17.4. (@kvaps in https://github.com/cozystack/cozystack/pull/1046)
+* Update MetalLB to v0.15.2. (@kvaps in https://github.com/cozystack/cozystack/pull/1045)
+* Update Kube-OVN to v1.13.13. (@kvaps in https://github.com/cozystack/cozystack/pull/1047)
+
+## Documentation
+
+* [Oracle Cloud Infrastructure installation guide](https://cozystack.io/docs/operations/talos/installation/oracle-cloud/). (@kvaps, @lllamnyp, and @NickVolynkin in https://github.com/cozystack/website/pull/168)
+* [Cluster configuration with `talosctl`](https://cozystack.io/docs/operations/talos/configuration/talosctl/). (@NickVolynkin in https://github.com/cozystack/website/pull/211)
+* [Configuring container registry mirrors for tenant Kubernetes clusters](https://cozystack.io/docs/operations/talos/configuration/air-gapped/#5-configure-container-registry-mirrors-for-tenant-kubernetes). (@klinch0 in https://github.com/cozystack/website/pull/210)
+* [Explain application management strategies and available versions for managed applications.](https://cozystack.io/docs/guides/applications/). (@NickVolynkin in https://github.com/cozystack/website/pull/219)
+* [How to clean up etcd state](https://cozystack.io/docs/operations/faq/#how-to-clean-up-etcd-state). (@gwynbleidd2106 in https://github.com/cozystack/website/pull/214)
+* [State that Cozystack is a CNCF Sandbox project](https://github.com/cozystack/cozystack?tab=readme-ov-file#cozystack). (@NickVolynkin in https://github.com/cozystack/cozystack/pull/1055)
+
+## Development, Testing, and CI/CD
+
+* [tests] Add tests for applications `virtual-machine`, `vm-disk`, `vm-instance`, `postgresql`, `mysql`, and `clickhouse`. (@gwynbleidd2106 in https://github.com/cozystack/cozystack/pull/1048, patched by @kvaps in https://github.com/cozystack/cozystack/pull/1074)
+* [tests] Fix concurrency for the `docker login` action. (@kvaps in https://github.com/cozystack/cozystack/pull/1014)
+* [tests] Increase QEMU system disk size in tests. (@kvaps in https://github.com/cozystack/cozystack/pull/1011)
+* [tests] Increase the waiting timeout for VMs in tests. (@kvaps in https://github.com/cozystack/cozystack/pull/1038)
+* [ci] Separate build and testing jobs in CI. (@kvaps in https://github.com/cozystack/cozystack/pull/1005 and https://github.com/cozystack/cozystack/pull/1010)
+* [ci] Fix the release assets. (@kvaps in https://github.com/cozystack/cozystack/pull/1006 and https://github.com/cozystack/cozystack/pull/1009)
+
+## New Contributors
+
+* @kevin880202 made their first contribution in https://github.com/cozystack/cozystack/pull/948
+* @mattia-eleuteri made their first contribution in https://github.com/cozystack/cozystack/pull/1027
+
+**Full Changelog**: https://github.com/cozystack/cozystack/compare/v0.31.0...v0.32.0
+
+<!-- 
+HEAD https://github.com/cozystack/cozystack/commit/3ce6dbe8
+-->

docs/changelogs/v0.32.1.md

@@ -0,0 +1,38 @@
+## Major Features and Improvements
+
+* [postgres] Introduce new functionality for backup and restore in PostgreSQL. (@klinch0 in https://github.com/cozystack/cozystack/pull/1086)
+* [apps] Refactor resources in managed applications. (@kvaps in https://github.com/cozystack/cozystack/pull/1106)
+* [system] Make VMAgent's `extraArgs` tunable. (@lllamnyp in https://github.com/cozystack/cozystack/pull/1091)
+
+## Fixes
+
+* [postgres] Escape users and database names. (@kvaps in https://github.com/cozystack/cozystack/pull/1087)
+* [tenant] Fix monitoring agents HelmReleases for tenant clusters. (@klinch0 in https://github.com/cozystack/cozystack/pull/1079)
+* [kubernetes] Wrap cert-manager CRDs in a conditional. (@lllamnyp in https://github.com/cozystack/cozystack/pull/1076)
+* [kubernetes] Remove `useCustomSecretForPatchContainerd` option and enable it by default. (@kvaps in https://github.com/cozystack/cozystack/pull/1104)
+* [apps] Increase default resource presets for Clickhouse and Kafka from `nano` to `small`. Update OpenAPI specs and readme's. (@kvaps in https://github.com/cozystack/cozystack/pull/1103 and https://github.com/cozystack/cozystack/pull/1105)
+* [linstor] Add configurable DRBD network options for connection and timeout settings, replacing scripted logic for detecting devices that lost connection. (@kvaps in https://github.com/cozystack/cozystack/pull/1094)
+
+## Dependencies
+
+* Update cozy-proxy to v0.2.0 (@kvaps in https://github.com/cozystack/cozystack/pull/1081)
+* Update Kafka Operator to 0.45.1-rc1 (@kvaps in https://github.com/cozystack/cozystack/pull/1082 and https://github.com/cozystack/cozystack/pull/1102)
+* Update Flux Operator to 0.23.0 (@kingdonb in https://github.com/cozystack/cozystack/pull/1078)
+
+## Documentation
+
+* [docs] Release notes for v0.32.0 and two beta-versions. (@NickVolynkin in https://github.com/cozystack/cozystack/pull/1043)
+
+## Development, Testing, and CI/CD
+
+* [tests] Add Kafka, Redis. (@gwynbleidd2106 in https://github.com/cozystack/cozystack/pull/1077)
+* [tests] Increase disk space for VMs in tests. (@kvaps in https://github.com/cozystack/cozystack/pull/1097)
+* [tests] Upd Kubernetes v1.33. (@kvaps in https://github.com/cozystack/cozystack/pull/1083)
+* [tests] increase postgres timeouts. (@kvaps in https://github.com/cozystack/cozystack/pull/1108)
+* [tests] don't wait for postgres ro service. (@kvaps in https://github.com/cozystack/cozystack/pull/1109)
+* [ci] Setup systemd timer to tear down sandbox. (@lllamnyp in https://github.com/cozystack/cozystack/pull/1092)
+* [ci] Split testing job into several. (@lllamnyp in https://github.com/cozystack/cozystack/pull/1075)
+* [ci] Run E2E tests as separate parallel jobs. (@lllamnyp in https://github.com/cozystack/cozystack/pull/1093)
+* [ci] Refactor GitHub workflows. (@kvaps in https://github.com/cozystack/cozystack/pull/1107)
+
+**Full Changelog**: https://github.com/cozystack/cozystack/compare/v0.32.0...v0.32.1

hack/cdi_golden_image_create.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+set -e
+
+name="$1"
+url="$2"
+
+if [ -z "$name" ] || [ -z "$url" ]; then
+  echo "Usage: <name> <url>"
+  echo "Example: 'ubuntu' 'https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img'"
+  exit 1
+fi
+
+#### create DV ubuntu source for CDI image cloning
+kubectl create -f - <<EOF
+apiVersion: cdi.kubevirt.io/v1beta1
+kind: DataVolume
+metadata:
+  name: "vm-image-$name"
+  namespace: cozy-public
+  annotations:
+    cdi.kubevirt.io/storage.bind.immediate.requested: "true"
+spec:
+  source:
+    http:
+      url: "$url"
+  storage:
+    resources:
+      requests:
+        storage: 5Gi
+    storageClassName: replicated
+EOF

hack/collect-images.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+
+for node in 11 12 13; do
+  talosctl -n 192.168.123.${node} -e 192.168.123.${node} images ls >> images.tmp
+  talosctl -n 192.168.123.${node} -e 192.168.123.${node} images --namespace system ls >> images.tmp
+done
+
+while read _ name sha _ ; do echo $sha $name ; done < images.tmp | sort -u > images.txt

hack/cozyreport.sh

@@ -0,0 +1,147 @@
+#!/bin/sh
+REPORT_DATE=$(date +%Y-%m-%d_%H-%M-%S)
+REPORT_NAME=${1:-cozyreport-$REPORT_DATE}
+REPORT_PDIR=$(mktemp -d)
+REPORT_DIR=$REPORT_PDIR/$REPORT_NAME
+
+# -- check dependencies
+command -V kubectl >/dev/null || exit $?
+command -V tar >/dev/null || exit $?
+
+# -- cozystack module
+
+echo "Collecting Cozystack information..."
+mkdir -p $REPORT_DIR/cozystack
+kubectl get deploy -n cozy-system cozystack -o jsonpath='{.spec.template.spec.containers[0].image}' > $REPORT_DIR/cozystack/image.txt 2>&1
+kubectl get cm -n cozy-system --no-headers | awk '$1 ~ /^cozystack/' |
+  while read NAME _; do
+    DIR=$REPORT_DIR/cozystack/configs
+    mkdir -p $DIR
+    kubectl get cm -n cozy-system $NAME -o yaml > $DIR/$NAME.yaml 2>&1
+  done
+
+# -- kubernetes module
+
+echo "Collecting Kubernetes information..."
+mkdir -p $REPORT_DIR/kubernetes
+kubectl version > $REPORT_DIR/kubernetes/version.txt 2>&1
+
+echo "Collecting nodes..."
+kubectl get nodes -o wide > $REPORT_DIR/kubernetes/nodes.txt 2>&1
+kubectl get nodes --no-headers | awk '$2 != "Ready"' |
+  while read NAME _; do
+    DIR=$REPORT_DIR/kubernetes/nodes/$NAME
+    mkdir -p $DIR
+    kubectl get node $NAME -o yaml > $DIR/node.yaml 2>&1
+    kubectl describe node $NAME > $DIR/describe.txt 2>&1
+  done
+
+echo "Collecting namespaces..."
+kubectl get ns -o wide > $REPORT_DIR/kubernetes/namespaces.txt 2>&1
+kubectl get ns --no-headers | awk '$2 != "Active"' |
+  while read NAME _; do
+    DIR=$REPORT_DIR/kubernetes/namespaces/$NAME
+    mkdir -p $DIR
+    kubectl get ns $NAME -o yaml > $DIR/namespace.yaml 2>&1
+    kubectl describe ns $NAME > $DIR/describe.txt 2>&1
+  done
+
+echo "Collecting helmreleases..."
+kubectl get hr -A > $REPORT_DIR/kubernetes/helmreleases.txt 2>&1
+kubectl get hr -A | awk '$4 != "True"' | \
+  while read NAMESPACE NAME _; do
+    DIR=$REPORT_DIR/kubernetes/helmreleases/$NAMESPACE/$NAME
+    mkdir -p $DIR
+    kubectl get hr -n $NAMESPACE $NAME -o yaml > $DIR/hr.yaml 2>&1
+    kubectl describe hr -n $NAMESPACE $NAME > $DIR/describe.txt 2>&1
+  done
+
+echo "Collecting pods..."
+kubectl get pod -A -o wide > $REPORT_DIR/kubernetes/pods.txt 2>&1
+kubectl get pod -A --no-headers | awk '$4 !~ /Running|Succeeded|Completed/' |
+  while read NAMESPACE NAME _ STATE _; do
+    DIR=$REPORT_DIR/kubernetes/pods/$NAMESPACE/$NAME
+    mkdir -p $DIR
+    CONTAINERS=$(kubectl get pod -o jsonpath='{.spec.containers[*].name}' -n $NAMESPACE $NAME)
+    kubectl get pod -n $NAMESPACE $NAME -o yaml > $DIR/pod.yaml 2>&1
+    kubectl describe pod -n $NAMESPACE $NAME > $DIR/describe.txt 2>&1
+    if [ "$STATE" != "Pending" ]; then
+      for CONTAINER in $CONTAINERS; do
+        kubectl logs -n $NAMESPACE $NAME $CONTAINER > $DIR/logs-$CONTAINER.txt 2>&1
+        kubectl logs -n $NAMESPACE $NAME $CONTAINER --previous > $DIR/logs-$CONTAINER-previous.txt 2>&1
+      done
+    fi
+  done
+
+echo "Collecting virtualmachines..."
+kubectl get vm -A > $REPORT_DIR/kubernetes/vms.txt 2>&1
+kubectl get vm -A --no-headers | awk '$5 != "True"' |
+  while read NAMESPACE NAME _; do
+    DIR=$REPORT_DIR/kubernetes/vm/$NAMESPACE/$NAME
+    mkdir -p $DIR
+    kubectl get vm -n $NAMESPACE $NAME -o yaml > $DIR/vm.yaml 2>&1
+    kubectl describe vm -n $NAMESPACE $NAME > $DIR/describe.txt 2>&1
+  done
+
+echo "Collecting virtualmachine instances..."
+kubectl get vmi -A > $REPORT_DIR/kubernetes/vmis.txt 2>&1
+kubectl get vmi -A --no-headers | awk '$4 != "Running"' |
+  while read NAMESPACE NAME _; do
+    DIR=$REPORT_DIR/kubernetes/vmi/$NAMESPACE/$NAME
+    mkdir -p $DIR
+    kubectl get vmi -n $NAMESPACE $NAME -o yaml > $DIR/vmi.yaml 2>&1
+    kubectl describe vmi -n $NAMESPACE $NAME > $DIR/describe.txt 2>&1
+  done
+
+echo "Collecting services..."
+kubectl get svc -A > $REPORT_DIR/kubernetes/services.txt 2>&1
+kubectl get svc -A --no-headers | awk '$4 == "<pending>"' |
+  while read NAMESPACE NAME _; do
+    DIR=$REPORT_DIR/kubernetes/services/$NAMESPACE/$NAME
+    mkdir -p $DIR
+    kubectl get svc -n $NAMESPACE $NAME -o yaml > $DIR/service.yaml 2>&1
+    kubectl describe svc -n $NAMESPACE $NAME > $DIR/describe.txt 2>&1
+  done
+
+echo "Collecting pvcs..."
+kubectl get pvc -A > $REPORT_DIR/kubernetes/pvcs.txt 2>&1
+kubectl get pvc -A | awk '$3 != "Bound"'  |
+  while read NAMESPACE NAME _; do
+    DIR=$REPORT_DIR/kubernetes/pvc/$NAMESPACE/$NAME
+    mkdir -p $DIR
+    kubectl get pvc -n $NAMESPACE $NAME -o yaml > $DIR/pvc.yaml 2>&1
+    kubectl describe pvc -n $NAMESPACE $NAME > $DIR/describe.txt 2>&1
+  done
+
+# -- kamaji module
+
+if kubectl get deploy -n cozy-linstor linstor-controller >/dev/null 2>&1; then
+  echo "Collecting kamaji resources..."
+  DIR=$REPORT_DIR/kamaji
+  mkdir -p $DIR
+  kubectl logs -n cozy-kamaji deployment/kamaji > $DIR/kamaji-controller.log 2>&1
+  kubectl get kamajicontrolplanes.controlplane.cluster.x-k8s.io -A > $DIR/kamajicontrolplanes.txt 2>&1
+  kubectl get kamajicontrolplanes.controlplane.cluster.x-k8s.io -A -o yaml > $DIR/kamajicontrolplanes.yaml 2>&1
+  kubectl get tenantcontrolplanes.kamaji.clastix.io -A > $DIR/tenantcontrolplanes.txt 2>&1
+  kubectl get tenantcontrolplanes.kamaji.clastix.io -A -o yaml > $DIR/tenantcontrolplanes.yaml 2>&1
+fi
+
+# -- linstor module
+
+if kubectl get deploy -n cozy-linstor linstor-controller >/dev/null 2>&1; then
+  echo "Collecting linstor resources..."
+  DIR=$REPORT_DIR/linstor
+  mkdir -p $DIR
+  kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor --no-color n l > $DIR/nodes.txt 2>&1
+  kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor --no-color sp l > $DIR/storage-pools.txt 2>&1
+  kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor --no-color r l > $DIR/resources.txt 2>&1
+fi
+
+# -- finalization
+
+echo "Creating archive..."
+tar -czf $REPORT_NAME.tgz -C $REPORT_PDIR .
+echo "Report created: $REPORT_NAME.tgz"
+
+echo "Cleaning up..."
+rm -rf $REPORT_PDIR

hack/e2e-apps.bats

@@ -1,353 +0,0 @@
-#!/usr/bin/env bats
-
-# -----------------------------------------------------------------------------
-# Cozystack end‑to‑end provisioning test (Bats)
-# -----------------------------------------------------------------------------
-
-@test "Create tenant with isolated mode enabled" {
-  kubectl -n tenant-root get tenants.apps.cozystack.io test || 
-  kubectl create -f - <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: Tenant
-metadata:
-  name: test
-  namespace: tenant-root
-spec:
-  etcd: false
-  host: ""
-  ingress: false
-  isolated: true
-  monitoring: false
-  resourceQuotas: {}
-  seaweedfs: false
-EOF
-  kubectl wait hr/tenant-test -n tenant-root --timeout=1m --for=condition=ready
-  kubectl wait namespace tenant-test --timeout=20s --for=jsonpath='{.status.phase}'=Active
-}
-
-@test "Create a tenant Kubernetes control plane" {
-  kubectl -n tenant-test get kuberneteses.apps.cozystack.io test || 
-  kubectl create -f - <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: Kubernetes
-metadata:
-  name: test
-  namespace: tenant-test
-spec:
-  addons:
-    certManager:
-      enabled: false
-      valuesOverride: {}
-    cilium:
-      valuesOverride: {}
-    fluxcd:
-      enabled: false
-      valuesOverride: {}
-    gatewayAPI:
-      enabled: false
-    gpuOperator:
-      enabled: false
-      valuesOverride: {}
-    ingressNginx:
-      enabled: true
-      hosts: []
-      valuesOverride: {}
-    monitoringAgents:
-      enabled: false
-      valuesOverride: {}
-    verticalPodAutoscaler:
-      valuesOverride: {}
-  controlPlane:
-    apiServer:
-      resources: {}
-      resourcesPreset: small
-    controllerManager:
-      resources: {}
-      resourcesPreset: micro
-    konnectivity:
-      server:
-        resources: {}
-        resourcesPreset: micro
-    replicas: 2
-    scheduler:
-      resources: {}
-      resourcesPreset: micro
-  host: ""
-  nodeGroups:
-    md0:
-      ephemeralStorage: 20Gi
-      gpus: []
-      instanceType: u1.medium
-      maxReplicas: 10
-      minReplicas: 0
-      resources:
-        cpu: ""
-        memory: ""
-      roles:
-      - ingress-nginx
-  storageClass: replicated
-EOF
-  kubectl wait namespace tenant-test --timeout=20s --for=jsonpath='{.status.phase}'=Active
-  timeout 10 sh -ec 'until kubectl get kamajicontrolplane -n tenant-test kubernetes-test; do sleep 1; done'
-  kubectl wait --for=condition=TenantControlPlaneCreated kamajicontrolplane -n tenant-test kubernetes-test --timeout=4m
-  kubectl wait tcp -n tenant-test kubernetes-test --timeout=2m --for=jsonpath='{.status.kubernetesResources.version.status}'=Ready
-  kubectl wait deploy --timeout=4m --for=condition=available -n tenant-test kubernetes-test kubernetes-test-cluster-autoscaler kubernetes-test-kccm kubernetes-test-kcsi-controller
-  kubectl wait machinedeployment kubernetes-test-md0 -n tenant-test --timeout=1m --for=jsonpath='{.status.replicas}'=2
-  kubectl wait machinedeployment kubernetes-test-md0 -n tenant-test --timeout=10m --for=jsonpath='{.status.v1beta2.readyReplicas}'=2
-  kubectl -n tenant-test delete kuberneteses.apps.cozystack.io test
-}
-
-@test "Create a VM Disk" {
-  name='test'
-  kubectl -n tenant-test get vmdisks.apps.cozystack.io $name || 
-  kubectl create -f - <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: VMDisk
-metadata:
-  name: $name
-  namespace: tenant-test
-spec:
-  source:
-    http:
-      url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
-  optical: false
-  storage: 5Gi
-  storageClass: replicated
-EOF
-  sleep 5
-  kubectl -n tenant-test wait hr vm-disk-$name --timeout=5s --for=condition=ready
-  kubectl -n tenant-test wait dv vm-disk-$name --timeout=150s --for=condition=ready
-  kubectl -n tenant-test wait pvc vm-disk-$name --timeout=100s --for=jsonpath='{.status.phase}'=Bound
-}
-
-@test "Create a VM Instance" {
-  diskName='test'
-  name='test'
-  kubectl -n tenant-test get vminstances.apps.cozystack.io $name || 
-  kubectl create -f - <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: VMInstance
-metadata:
-  name: $name
-  namespace: tenant-test
-spec:
-  external: false
-  externalMethod: PortList
-  externalPorts:
-  - 22
-  running: true
-  instanceType: "u1.medium"
-  instanceProfile: ubuntu
-  disks:
-    - name: $diskName
-  gpus: []
-  resources:
-    cpu: ""
-    memory: ""
-  sshKeys:
-  - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF
-    test@test
-  cloudInit: |
-    #cloud-config
-    users:
-      - name: test
-        shell: /bin/bash
-        sudo: ['ALL=(ALL) NOPASSWD: ALL']
-        groups: sudo
-        ssh_authorized_keys:
-          - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF test@test
-  cloudInitSeed: ""
-EOF
-  sleep 5
-  timeout 20 sh -ec "until kubectl -n tenant-test get vmi vm-instance-$name -o jsonpath='{.status.interfaces[0].ipAddress}' | grep -q '[0-9]'; do sleep 5; done"
-  kubectl -n tenant-test wait hr vm-instance-$name --timeout=5s --for=condition=ready
-  kubectl -n tenant-test wait vm vm-instance-$name --timeout=20s --for=condition=ready
-  kubectl -n tenant-test delete vminstances.apps.cozystack.io $name 
-  kubectl -n tenant-test delete vmdisks.apps.cozystack.io $diskName 
-}
-
-@test "Create a Virtual Machine" {
-  name='test'
-  kubectl -n tenant-test get virtualmachines.apps.cozystack.io $name || 
-  kubectl create -f - <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: VirtualMachine
-metadata:
-  name: $name
-  namespace: tenant-test
-spec:
-  external: false
-  externalMethod: PortList
-  externalPorts:
-  - 22
-  instanceType: "u1.medium"
-  instanceProfile: ubuntu
-  systemDisk:
-    image: ubuntu
-    storage: 5Gi
-    storageClass: replicated
-  gpus: []
-  resources:
-    cpu: ""
-    memory: ""
-  sshKeys:
-  - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF
-    test@test
-  cloudInit: |
-    #cloud-config
-    users:
-      - name: test
-        shell: /bin/bash
-        sudo: ['ALL=(ALL) NOPASSWD: ALL']
-        groups: sudo
-        ssh_authorized_keys:
-          - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF test@test
-  cloudInitSeed: ""
-EOF
-  sleep 5
-  kubectl -n tenant-test wait hr virtual-machine-$name --timeout=10s --for=condition=ready
-  kubectl -n tenant-test wait dv virtual-machine-$name --timeout=150s --for=condition=ready
-  kubectl -n tenant-test wait pvc virtual-machine-$name --timeout=100s --for=jsonpath='{.status.phase}'=Bound
-  kubectl -n tenant-test wait vm virtual-machine-$name --timeout=100s --for=condition=ready
-  timeout 120 sh -ec "until kubectl -n tenant-test get vmi virtual-machine-$name -o jsonpath='{.status.interfaces[0].ipAddress}' | grep -q '[0-9]'; do sleep 10; done"
-  kubectl -n tenant-test delete virtualmachines.apps.cozystack.io $name 
-}
-
-@test "Create DB PostgreSQL" {
-  name='test'
-  kubectl -n tenant-test get postgreses.apps.cozystack.io $name || 
-  kubectl create -f - <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: Postgres
-metadata:
-  name: $name
-  namespace: tenant-test
-spec:
-  external: false
-  size: 10Gi
-  replicas: 2
-  storageClass: ""
-  postgresql:
-    parameters:
-      max_connections: 100
-  quorum:
-    minSyncReplicas: 0
-    maxSyncReplicas: 0
-  users:
-    testuser:
-      password: xai7Wepo
-  databases:
-    testdb:
-      roles:
-        admin:
-        - testuser
-  backup:
-    enabled: false
-    s3Region: us-east-1
-    s3Bucket: s3.example.org/postgres-backups
-    schedule: "0 2 * * *"
-    cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
-    s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
-    s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
-    resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
-  resources: {}
-  resourcesPreset: "nano"
-EOF
-  sleep 5
-  kubectl -n tenant-test wait hr postgres-$name --timeout=100s --for=condition=ready
-  kubectl -n tenant-test wait job.batch postgres-$name-init-job --timeout=50s --for=condition=Complete
-  timeout 40 sh -ec "until kubectl -n tenant-test get svc postgres-$name-r -o jsonpath='{.spec.ports[0].port}' | grep -q '5432'; do sleep 10; done"
-  timeout 40 sh -ec "until kubectl -n tenant-test get svc postgres-$name-ro -o jsonpath='{.spec.ports[0].port}' | grep -q '5432'; do sleep 10; done"
-  timeout 40 sh -ec "until kubectl -n tenant-test get svc postgres-$name-rw -o jsonpath='{.spec.ports[0].port}' | grep -q '5432'; do sleep 10; done"
-  timeout 120 sh -ec "until kubectl -n tenant-test get endpoints postgres-$name-r -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
-  timeout 120 sh -ec "until kubectl -n tenant-test get endpoints postgres-$name-ro -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
-  timeout 120 sh -ec "until kubectl -n tenant-test get endpoints postgres-$name-rw -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
-  kubectl -n tenant-test delete postgreses.apps.cozystack.io $name
-}
-
-@test "Create DB MySQL" {
-  name='test'
-  kubectl -n tenant-test get mysqls.apps.cozystack.io $name || 
-  kubectl create -f- <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: MySQL
-metadata:
-  name: $name
-  namespace: tenant-test
-spec:
-  external: false
-  size: 10Gi
-  replicas: 2
-  storageClass: ""
-  users:
-    testuser:
-      maxUserConnections: 1000
-      password: xai7Wepo
-  databases:
-    testdb:
-      roles:
-        admin:
-        - testuser
-  backup:
-    enabled: false
-    s3Region: us-east-1
-    s3Bucket: s3.example.org/postgres-backups
-    schedule: "0 2 * * *"
-    cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
-    s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
-    s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
-    resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
-  resources: {}
-  resourcesPreset: "nano"
-EOF
-  sleep 5
-  kubectl -n tenant-test wait hr mysql-$name --timeout=30s --for=condition=ready
-  timeout 80 sh -ec "until kubectl -n tenant-test get svc mysql-$name -o jsonpath='{.spec.ports[0].port}' | grep -q '3306'; do sleep 10; done"
-  timeout 80 sh -ec "until kubectl -n tenant-test get endpoints mysql-$name -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
-  kubectl -n tenant-test wait statefulset.apps/mysql-$name --timeout=110s --for=jsonpath='{.status.replicas}'=2
-  timeout 80 sh -ec "until kubectl -n tenant-test get svc mysql-$name-metrics -o jsonpath='{.spec.ports[0].port}' | grep -q '9104'; do sleep 10; done"
-  timeout 40 sh -ec "until kubectl -n tenant-test get endpoints mysql-$name-metrics -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
-  kubectl -n tenant-test wait deployment.apps/mysql-$name-metrics --timeout=90s --for=jsonpath='{.status.replicas}'=1
-  kubectl -n tenant-test delete mysqls.apps.cozystack.io $name
-}
-
-@test "Create DB ClickHouse" {
-  name='test'
-  kubectl -n tenant-test get clickhouses.apps.cozystack.io $name || 
-  kubectl create -f- <<EOF
-apiVersion: apps.cozystack.io/v1alpha1
-kind: ClickHouse
-metadata:
-  name: $name
-  namespace: tenant-test
-spec:
-  size: 10Gi
-  logStorageSize: 2Gi
-  shards: 1
-  replicas: 2
-  storageClass: ""
-  logTTL: 15
-  users:
-    testuser:
-      password: xai7Wepo
-  backup:
-    enabled: false
-    s3Region: us-east-1
-    s3Bucket: s3.example.org/clickhouse-backups
-    schedule: "0 2 * * *"
-    cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
-    s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
-    s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
-    resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
-  resources: {}
-  resourcesPreset: "nano"
-EOF
-  sleep 5
-  kubectl -n tenant-test wait hr clickhouse-$name --timeout=20s --for=condition=ready
-  timeout 180 sh -ec "until kubectl -n tenant-test get svc chendpoint-clickhouse-$name -o jsonpath='{.spec.ports[*].port}' | grep -q '8123 9000'; do sleep 10; done"
-  kubectl -n tenant-test wait statefulset.apps/chi-clickhouse-$name-clickhouse-0-0 --timeout=120s --for=jsonpath='{.status.replicas}'=1
-  timeout 80 sh -ec "until kubectl -n tenant-test get endpoints chi-clickhouse-$name-clickhouse-0-0 -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
-  timeout 100 sh -ec "until kubectl -n tenant-test get svc chi-clickhouse-$name-clickhouse-0-0 -o jsonpath='{.spec.ports[*].port}' | grep -q '9000 8123 9009'; do sleep 10; done"
-  timeout 80 sh -ec "until kubectl -n tenant-test get sts chi-clickhouse-$name-clickhouse-0-1 ; do sleep 10; done"
-  kubectl -n tenant-test wait statefulset.apps/chi-clickhouse-$name-clickhouse-0-1 --timeout=140s --for=jsonpath='{.status.replicas}'=1
-}

hack/e2e-apps/clickhouse.bats

@@ -0,0 +1,41 @@
+#!/usr/bin/env bats
+
+@test "Create DB ClickHouse" {
+  name='test'
+  kubectl apply -f- <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: ClickHouse
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  size: 10Gi
+  logStorageSize: 2Gi
+  shards: 1
+  replicas: 2
+  storageClass: ""
+  logTTL: 15
+  users:
+    testuser:
+      password: xai7Wepo
+  backup:
+    enabled: false
+    s3Region: us-east-1
+    s3Bucket: s3.example.org/clickhouse-backups
+    schedule: "0 2 * * *"
+    cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+    s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
+    s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
+    resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
+  resources: {}
+  resourcesPreset: "nano"
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr clickhouse-$name --timeout=20s --for=condition=ready
+  timeout 180 sh -ec "until kubectl -n tenant-test get svc chendpoint-clickhouse-$name -o jsonpath='{.spec.ports[*].port}' | grep -q '8123 9000'; do sleep 10; done"
+  kubectl -n tenant-test wait statefulset.apps/chi-clickhouse-$name-clickhouse-0-0 --timeout=120s --for=jsonpath='{.status.replicas}'=1
+  timeout 80 sh -ec "until kubectl -n tenant-test get endpoints chi-clickhouse-$name-clickhouse-0-0 -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  timeout 100 sh -ec "until kubectl -n tenant-test get svc chi-clickhouse-$name-clickhouse-0-0 -o jsonpath='{.spec.ports[*].port}' | grep -q '9000 8123 9009'; do sleep 10; done"
+  timeout 80 sh -ec "until kubectl -n tenant-test get sts chi-clickhouse-$name-clickhouse-0-1 ; do sleep 10; done"
+  kubectl -n tenant-test wait statefulset.apps/chi-clickhouse-$name-clickhouse-0-1 --timeout=140s --for=jsonpath='{.status.replicas}'=1
+}

hack/e2e-apps/kafka.bats

@@ -0,0 +1,51 @@
+#!/usr/bin/env bats
+
+@test "Create Kafka" {
+  name='test'
+  kubectl apply -f- <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: Kafka
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  external: false
+  kafka:
+    size: 10Gi
+    replicas: 2
+    storageClass: ""
+    resources: {}
+    resourcesPreset: "nano"
+  zookeeper:
+    size: 5Gi
+    replicas: 2
+    storageClass: ""
+    resources:
+    resourcesPreset: "nano"
+  topics:
+    - name: testResults
+      partitions: 1
+      replicas: 2
+      config:
+        min.insync.replicas: 2
+    - name: testOrders
+      config:
+        cleanup.policy: compact
+        segment.ms: 3600000
+        max.compaction.lag.ms: 5400000
+        min.insync.replicas: 2
+      partitions: 1
+      replicas: 2
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr kafka-$name --timeout=30s --for=condition=ready
+  kubectl wait kafkas -n tenant-test test --timeout=60s --for=condition=ready
+  timeout 60 sh -ec "until kubectl -n tenant-test get pvc data-kafka-$name-zookeeper-0; do sleep 10; done"
+  kubectl -n tenant-test wait pvc data-kafka-$name-zookeeper-0 --timeout=50s --for=jsonpath='{.status.phase}'=Bound
+  timeout 40 sh -ec "until kubectl -n tenant-test get svc kafka-$name-zookeeper-client -o jsonpath='{.spec.ports[0].port}' | grep -q '2181'; do sleep 10; done"
+  timeout 40 sh -ec "until kubectl -n tenant-test get svc kafka-$name-zookeeper-nodes -o jsonpath='{.spec.ports[*].port}' | grep -q '2181 2888 3888'; do sleep 10; done"
+  timeout 80 sh -ec "until kubectl -n tenant-test get endpoints kafka-$name-zookeeper-nodes -o jsonpath='{.subsets[*].addresses[0].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  kubectl -n tenant-test delete kafka.apps.cozystack.io $name
+  kubectl -n tenant-test delete pvc data-kafka-$name-zookeeper-0
+  kubectl -n tenant-test delete pvc data-kafka-$name-zookeeper-1
+}

hack/e2e-apps/kubernetes.bats

@@ -0,0 +1,72 @@
+#!/usr/bin/env bats
+
+@test "Create a tenant Kubernetes control plane" {
+  kubectl apply -f - <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: Kubernetes
+metadata:
+  name: test
+  namespace: tenant-test
+spec:
+  addons:
+    certManager:
+      enabled: false
+      valuesOverride: {}
+    cilium:
+      valuesOverride: {}
+    fluxcd:
+      enabled: false
+      valuesOverride: {}
+    gatewayAPI:
+      enabled: false
+    gpuOperator:
+      enabled: false
+      valuesOverride: {}
+    ingressNginx:
+      enabled: true
+      hosts: []
+      valuesOverride: {}
+    monitoringAgents:
+      enabled: false
+      valuesOverride: {}
+    verticalPodAutoscaler:
+      valuesOverride: {}
+  controlPlane:
+    apiServer:
+      resources: {}
+      resourcesPreset: small
+    controllerManager:
+      resources: {}
+      resourcesPreset: micro
+    konnectivity:
+      server:
+        resources: {}
+        resourcesPreset: micro
+    replicas: 2
+    scheduler:
+      resources: {}
+      resourcesPreset: micro
+  host: ""
+  nodeGroups:
+    md0:
+      ephemeralStorage: 20Gi
+      gpus: []
+      instanceType: u1.medium
+      maxReplicas: 10
+      minReplicas: 0
+      resources:
+        cpu: ""
+        memory: ""
+      roles:
+      - ingress-nginx
+  storageClass: replicated
+EOF
+  kubectl wait namespace tenant-test --timeout=20s --for=jsonpath='{.status.phase}'=Active
+  timeout 10 sh -ec 'until kubectl get kamajicontrolplane -n tenant-test kubernetes-test; do sleep 1; done'
+  kubectl wait --for=condition=TenantControlPlaneCreated kamajicontrolplane -n tenant-test kubernetes-test --timeout=4m
+  kubectl wait tcp -n tenant-test kubernetes-test --timeout=2m --for=jsonpath='{.status.kubernetesResources.version.status}'=Ready
+  kubectl wait deploy --timeout=4m --for=condition=available -n tenant-test kubernetes-test kubernetes-test-cluster-autoscaler kubernetes-test-kccm kubernetes-test-kcsi-controller
+  kubectl wait machinedeployment kubernetes-test-md0 -n tenant-test --timeout=1m --for=jsonpath='{.status.replicas}'=2
+  kubectl wait machinedeployment kubernetes-test-md0 -n tenant-test --timeout=10m --for=jsonpath='{.status.v1beta2.readyReplicas}'=2
+  kubectl -n tenant-test delete kuberneteses.apps.cozystack.io test
+}

hack/e2e-apps/mysql.bats

@@ -0,0 +1,46 @@
+#!/usr/bin/env bats
+
+@test "Create DB MySQL" {
+  name='test'
+  kubectl apply -f- <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: MySQL
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  external: false
+  size: 10Gi
+  replicas: 2
+  storageClass: ""
+  users:
+    testuser:
+      maxUserConnections: 1000
+      password: xai7Wepo
+  databases:
+    testdb:
+      roles:
+        admin:
+        - testuser
+  backup:
+    enabled: false
+    s3Region: us-east-1
+    s3Bucket: s3.example.org/postgres-backups
+    schedule: "0 2 * * *"
+    cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+    s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
+    s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
+    resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
+  resources: {}
+  resourcesPreset: "nano"
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr mysql-$name --timeout=30s --for=condition=ready
+  timeout 80 sh -ec "until kubectl -n tenant-test get svc mysql-$name -o jsonpath='{.spec.ports[0].port}' | grep -q '3306'; do sleep 10; done"
+  timeout 80 sh -ec "until kubectl -n tenant-test get endpoints mysql-$name -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  kubectl -n tenant-test wait statefulset.apps/mysql-$name --timeout=110s --for=jsonpath='{.status.replicas}'=2
+  timeout 80 sh -ec "until kubectl -n tenant-test get svc mysql-$name-metrics -o jsonpath='{.spec.ports[0].port}' | grep -q '9104'; do sleep 10; done"
+  timeout 40 sh -ec "until kubectl -n tenant-test get endpoints mysql-$name-metrics -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  kubectl -n tenant-test wait deployment.apps/mysql-$name-metrics --timeout=90s --for=jsonpath='{.status.replicas}'=1
+  kubectl -n tenant-test delete mysqls.apps.cozystack.io $name
+}

hack/e2e-apps/postgres.bats

@@ -0,0 +1,54 @@
+#!/usr/bin/env bats
+
+@test "Create DB PostgreSQL" {
+  name='test'
+  kubectl apply -f - <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: Postgres
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  external: false
+  size: 10Gi
+  replicas: 2
+  storageClass: ""
+  postgresql:
+    parameters:
+      max_connections: 100
+  quorum:
+    minSyncReplicas: 0
+    maxSyncReplicas: 0
+  users:
+    testuser:
+      password: xai7Wepo
+  databases:
+    testdb:
+      roles:
+        admin:
+        - testuser
+  backup:
+    enabled: false
+    s3Region: us-east-1
+    s3Bucket: s3.example.org/postgres-backups
+    schedule: "0 2 * * *"
+    cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+    s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
+    s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
+    resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
+  resources: {}
+  resourcesPreset: "nano"
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr postgres-$name --timeout=100s --for=condition=ready
+  kubectl -n tenant-test wait job.batch postgres-$name-init-job --timeout=50s --for=condition=Complete
+  timeout 40 sh -ec "until kubectl -n tenant-test get svc postgres-$name-r -o jsonpath='{.spec.ports[0].port}' | grep -q '5432'; do sleep 10; done"
+  timeout 40 sh -ec "until kubectl -n tenant-test get svc postgres-$name-ro -o jsonpath='{.spec.ports[0].port}' | grep -q '5432'; do sleep 10; done"
+  timeout 40 sh -ec "until kubectl -n tenant-test get svc postgres-$name-rw -o jsonpath='{.spec.ports[0].port}' | grep -q '5432'; do sleep 10; done"
+  timeout 120 sh -ec "until kubectl -n tenant-test get endpoints postgres-$name-r -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  # for some reason it takes longer for the read-only endpoint to be ready
+  #timeout 120 sh -ec "until kubectl -n tenant-test get endpoints postgres-$name-ro -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  timeout 120 sh -ec "until kubectl -n tenant-test get endpoints postgres-$name-rw -o jsonpath='{.subsets[*].addresses[*].ip}' | grep -q '[0-9]'; do sleep 10; done"
+  kubectl -n tenant-test delete postgreses.apps.cozystack.io $name
+  kubectl -n tenant-test delete job.batch/postgres-$name-init-job
+}

hack/e2e-apps/redis.bats

@@ -0,0 +1,26 @@
+#!/usr/bin/env bats
+
+@test "Create Redis" {
+  name='test'
+  kubectl apply -f- <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: Redis
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  external: false
+  size: 1Gi
+  replicas: 2
+  storageClass: ""
+  authEnabled: true
+  resources: {}
+  resourcesPreset: "nano"
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr redis-$name --timeout=20s --for=condition=ready
+  kubectl -n tenant-test wait pvc redisfailover-persistent-data-rfr-redis-$name-0 --timeout=50s --for=jsonpath='{.status.phase}'=Bound
+  kubectl -n tenant-test wait deploy rfs-redis-$name --timeout=90s --for=condition=available
+  kubectl -n tenant-test wait sts rfr-redis-$name --timeout=90s --for=jsonpath='{.status.replicas}'=2
+  kubectl -n tenant-test delete redis.apps.cozystack.io $name
+}

hack/e2e-apps/virtualmachine.bats

@@ -0,0 +1,47 @@
+#!/usr/bin/env bats
+
+@test "Create a Virtual Machine" {
+  name='test'
+  kubectl apply -f - <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: VirtualMachine
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  external: false
+  externalMethod: PortList
+  externalPorts:
+  - 22
+  instanceType: "u1.medium"
+  instanceProfile: ubuntu
+  systemDisk:
+    image: ubuntu
+    storage: 5Gi
+    storageClass: replicated
+  gpus: []
+  resources:
+    cpu: ""
+    memory: ""
+  sshKeys:
+  - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF
+    test@test
+  cloudInit: |
+    #cloud-config
+    users:
+      - name: test
+        shell: /bin/bash
+        sudo: ['ALL=(ALL) NOPASSWD: ALL']
+        groups: sudo
+        ssh_authorized_keys:
+          - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF test@test
+  cloudInitSeed: ""
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr virtual-machine-$name --timeout=10s --for=condition=ready
+  kubectl -n tenant-test wait dv virtual-machine-$name --timeout=150s --for=condition=ready
+  kubectl -n tenant-test wait pvc virtual-machine-$name --timeout=100s --for=jsonpath='{.status.phase}'=Bound
+  kubectl -n tenant-test wait vm virtual-machine-$name --timeout=100s --for=condition=ready
+  timeout 120 sh -ec "until kubectl -n tenant-test get vmi virtual-machine-$name -o jsonpath='{.status.interfaces[0].ipAddress}' | grep -q '[0-9]'; do sleep 10; done"
+  kubectl -n tenant-test delete virtualmachines.apps.cozystack.io $name 
+}

hack/e2e-apps/vminstance.bats

@@ -0,0 +1,68 @@
+#!/usr/bin/env bats
+
+@test "Create a VM Disk" {
+  name='test'
+  kubectl apply -f - <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: VMDisk
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  source:
+    http:
+      url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
+  optical: false
+  storage: 5Gi
+  storageClass: replicated
+EOF
+  sleep 5
+  kubectl -n tenant-test wait hr vm-disk-$name --timeout=5s --for=condition=ready
+  kubectl -n tenant-test wait dv vm-disk-$name --timeout=150s --for=condition=ready
+  kubectl -n tenant-test wait pvc vm-disk-$name --timeout=100s --for=jsonpath='{.status.phase}'=Bound
+}
+
+@test "Create a VM Instance" {
+  diskName='test'
+  name='test'
+  kubectl apply -f - <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: VMInstance
+metadata:
+  name: $name
+  namespace: tenant-test
+spec:
+  external: false
+  externalMethod: PortList
+  externalPorts:
+  - 22
+  running: true
+  instanceType: "u1.medium"
+  instanceProfile: ubuntu
+  disks:
+    - name: $diskName
+  gpus: []
+  resources:
+    cpu: ""
+    memory: ""
+  sshKeys:
+  - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF
+    test@test
+  cloudInit: |
+    #cloud-config
+    users:
+      - name: test
+        shell: /bin/bash
+        sudo: ['ALL=(ALL) NOPASSWD: ALL']
+        groups: sudo
+        ssh_authorized_keys:
+          - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPht0dPk5qQ+54g1hSX7A6AUxXJW5T6n/3d7Ga2F8gTF test@test
+  cloudInitSeed: ""
+EOF
+  sleep 5
+  timeout 20 sh -ec "until kubectl -n tenant-test get vmi vm-instance-$name -o jsonpath='{.status.interfaces[0].ipAddress}' | grep -q '[0-9]'; do sleep 5; done"
+  kubectl -n tenant-test wait hr vm-instance-$name --timeout=5s --for=condition=ready
+  kubectl -n tenant-test wait vm vm-instance-$name --timeout=20s --for=condition=ready
+  kubectl -n tenant-test delete vminstances.apps.cozystack.io $name 
+  kubectl -n tenant-test delete vmdisks.apps.cozystack.io $diskName 
+}

hack/e2e-cluster.bats

@@ -1,391 +0,0 @@
-#!/usr/bin/env bats
-# -----------------------------------------------------------------------------
-# Cozystack end‑to‑end provisioning test (Bats)
-# -----------------------------------------------------------------------------
-
-@test "Required installer assets exist" {
-  if [ ! -f _out/assets/cozystack-installer.yaml ]; then
-    echo "Missing: _out/assets/cozystack-installer.yaml" >&2
-    exit 1
-  fi
-
-  if [ ! -f _out/assets/nocloud-amd64.raw.xz ]; then
-    echo "Missing: _out/assets/nocloud-amd64.raw.xz" >&2
-    exit 1
-  fi
-}
-
-@test "IPv4 forwarding is enabled" {
-  if [ "$(cat /proc/sys/net/ipv4/ip_forward)" != 1 ]; then
-    echo "IPv4 forwarding is disabled!" >&2
-    echo >&2
-    echo "Enable it with:" >&2
-    echo "  echo 1 > /proc/sys/net/ipv4/ip_forward" >&2
-    exit 1
-  fi
-}
-
-@test "Clean previous VMs" {
- kill $(cat srv1/qemu.pid srv2/qemu.pid srv3/qemu.pid 2>/dev/null) 2>/dev/null || true
- rm -rf srv1 srv2 srv3
-}
-
-@test "Prepare networking and masquerading" {
-  ip link del cozy-br0 2>/dev/null || true
-  ip link add cozy-br0 type bridge
-  ip link set cozy-br0 up
-  ip address add 192.168.123.1/24 dev cozy-br0
-
-  # Masquerading rule – idempotent (delete first, then add)
-  iptables -t nat -D POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE 2>/dev/null || true
-  iptables -t nat -A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
-}
-
-@test "Prepare cloud‑init drive for VMs" {
-  mkdir -p srv1 srv2 srv3
-
-  # Generate cloud‑init ISOs
-  for i in 1 2 3; do
-    echo "hostname: srv${i}" > "srv${i}/meta-data"
-
-    cat > "srv${i}/user-data" <<'EOF'
-#cloud-config
-EOF
-
-    cat > "srv${i}/network-config" <<EOF
-version: 2
-ethernets:
-  eth0:
-    dhcp4: false
-    addresses:
-      - "192.168.123.1${i}/26"
-    gateway4: "192.168.123.1"
-    nameservers:
-      search: [cluster.local]
-      addresses: [8.8.8.8]
-EOF
-
-    ( cd "srv${i}" && genisoimage \
-        -output seed.img \
-        -volid cidata -rational-rock -joliet \
-        user-data meta-data network-config )
-  done
-}
-
-@test "Use Talos NoCloud image from assets" {
-  if [ ! -f _out/assets/nocloud-amd64.raw.xz ]; then
-    echo "Missing _out/assets/nocloud-amd64.raw.xz" 2>&1
-    exit 1
-  fi
-
-  rm -f nocloud-amd64.raw
-  cp _out/assets/nocloud-amd64.raw.xz .
-  xz --decompress nocloud-amd64.raw.xz
-}
-
-@test "Prepare VM disks" {
-  for i in 1 2 3; do
-    cp nocloud-amd64.raw srv${i}/system.img
-    qemu-img resize srv${i}/system.img 50G
-    qemu-img create srv${i}/data.img 100G
-  done
-}
-
-@test "Create tap devices" {
-  for i in 1 2 3; do
-    ip link del cozy-srv${i} 2>/dev/null || true
-    ip tuntap add dev cozy-srv${i} mode tap
-    ip link set cozy-srv${i} up
-    ip link set cozy-srv${i} master cozy-br0
-  done
-}
-
-@test "Boot QEMU VMs" {
-  for i in 1 2 3; do
-    qemu-system-x86_64 -machine type=pc,accel=kvm -cpu host -smp 8 -m 24576 \
-      -device virtio-net,netdev=net0,mac=52:54:00:12:34:5${i} \
-      -netdev tap,id=net0,ifname=cozy-srv${i},script=no,downscript=no \
-      -drive file=srv${i}/system.img,if=virtio,format=raw \
-      -drive file=srv${i}/seed.img,if=virtio,format=raw \
-      -drive file=srv${i}/data.img,if=virtio,format=raw \
-      -display none -daemonize -pidfile srv${i}/qemu.pid
-  done
-
-  # Give qemu a few seconds to start up networking
-  sleep 5
-}
-
-@test "Wait until Talos API port 50000 is reachable on all machines" {
-  timeout 60 sh -ec 'until nc -nz 192.168.123.11 50000 && nc -nz 192.168.123.12 50000 && nc -nz 192.168.123.13 50000; do sleep 1; done'
-}
-
-@test "Generate Talos cluster configuration" {
-  # Cluster‑wide patches
-  cat > patch.yaml <<'EOF'
-machine:
-  kubelet:
-    nodeIP:
-      validSubnets:
-      - 192.168.123.0/24
-    extraConfig:
-      maxPods: 512
-  kernel:
-    modules:
-    - name: openvswitch
-    - name: drbd
-      parameters:
-        - usermode_helper=disabled
-    - name: zfs
-    - name: spl
-  registries:
-    mirrors:
-      docker.io:
-        endpoints:
-        - https://mirror.gcr.io
-  files:
-  - content: |
-      [plugins]
-        [plugins."io.containerd.cri.v1.runtime"]
-          device_ownership_from_security_context = true
-    path: /etc/cri/conf.d/20-customization.part
-    op: create
-
-cluster:
-  apiServer:
-    extraArgs:
-      oidc-issuer-url: "https://keycloak.example.org/realms/cozy"
-      oidc-client-id: "kubernetes"
-      oidc-username-claim: "preferred_username"
-      oidc-groups-claim: "groups"
-  network:
-    cni:
-      name: none
-    dnsDomain: cozy.local
-    podSubnets:
-    - 10.244.0.0/16
-    serviceSubnets:
-    - 10.96.0.0/16
-EOF
-
-  # Control‑plane‑only patches
-  cat > patch-controlplane.yaml <<'EOF'
-machine:
-  nodeLabels:
-    node.kubernetes.io/exclude-from-external-load-balancers:
-      $patch: delete
-  network:
-    interfaces:
-    - interface: eth0
-      vip:
-        ip: 192.168.123.10
-cluster:
-  allowSchedulingOnControlPlanes: true
-  controllerManager:
-    extraArgs:
-      bind-address: 0.0.0.0
-  scheduler:
-    extraArgs:
-      bind-address: 0.0.0.0
-  apiServer:
-    certSANs:
-    - 127.0.0.1
-  proxy:
-    disabled: true
-  discovery:
-    enabled: false
-  etcd:
-    advertisedSubnets:
-    - 192.168.123.0/24
-EOF
-
-  # Generate secrets once
-  if [ ! -f secrets.yaml ]; then
-    talosctl gen secrets
-  fi
-
-  rm -f controlplane.yaml worker.yaml talosconfig kubeconfig
-  talosctl gen config --with-secrets secrets.yaml cozystack https://192.168.123.10:6443 \
-           --config-patch=@patch.yaml --config-patch-control-plane @patch-controlplane.yaml
-}
-
-@test "Apply Talos configuration to the node" {
-  # Apply the configuration to all three nodes
-  for node in 11 12 13; do
-    talosctl apply -f controlplane.yaml -n 192.168.123.${node} -e 192.168.123.${node} -i
-  done
-
-  # Wait for Talos services to come up again
-  timeout 60 sh -ec 'until nc -nz 192.168.123.11 50000 && nc -nz 192.168.123.12 50000 && nc -nz 192.168.123.13 50000; do sleep 1; done'
-}
-
-@test "Bootstrap Talos cluster" {
-  # Bootstrap etcd on the first node
-  timeout 10 sh -ec 'until talosctl bootstrap -n 192.168.123.11 -e 192.168.123.11; do sleep 1; done'
-
-  # Wait until etcd is healthy
-  timeout 180 sh -ec 'until talosctl etcd members -n 192.168.123.11,192.168.123.12,192.168.123.13 -e 192.168.123.10 >/dev/null 2>&1; do sleep 1; done'
-  timeout 60 sh -ec 'while talosctl etcd members -n 192.168.123.11,192.168.123.12,192.168.123.13 -e 192.168.123.10 2>&1 | grep -q "rpc error"; do sleep 1; done'
-
-  # Retrieve kubeconfig
-  rm -f kubeconfig
-  talosctl kubeconfig kubeconfig -e 192.168.123.10 -n 192.168.123.10
-
-  # Wait until all three nodes register in Kubernetes
-  timeout 60 sh -ec 'until [ $(kubectl get node --no-headers | wc -l) -eq 3 ]; do sleep 1; done'
-}
-
-@test "Install Cozystack" {
-  # Create namespace & configmap required by installer
-  kubectl create namespace cozy-system --dry-run=client -o yaml | kubectl apply -f -
-  kubectl create configmap cozystack -n cozy-system \
-          --from-literal=bundle-name=paas-full \
-          --from-literal=ipv4-pod-cidr=10.244.0.0/16 \
-          --from-literal=ipv4-pod-gateway=10.244.0.1 \
-          --from-literal=ipv4-svc-cidr=10.96.0.0/16 \
-          --from-literal=ipv4-join-cidr=100.64.0.0/16 \
-          --from-literal=root-host=example.org \
-          --from-literal=api-server-endpoint=https://192.168.123.10:6443 \
-          --dry-run=client -o yaml | kubectl apply -f -
-
-  # Apply installer manifests from file
-  kubectl apply -f _out/assets/cozystack-installer.yaml
-
-  # Wait for the installer deployment to become available
-  kubectl wait deployment/cozystack -n cozy-system --timeout=1m --for=condition=Available
-
-  # Wait until HelmReleases appear & reconcile them
-  timeout 60 sh -ec 'until kubectl get hr -A | grep -q cozys; do sleep 1; done'
-  sleep 5
-  kubectl get hr -A | awk 'NR>1 {print "kubectl wait --timeout=15m --for=condition=ready -n "$1" hr/"$2" &"} END {print "wait"}' | sh -ex
-
-  # Fail the test if any HelmRelease is not Ready
-  if kubectl get hr -A | grep -v " True " | grep -v NAME; then
-    kubectl get hr -A
-    fail "Some HelmReleases failed to reconcile"
-  fi
-}
-
-@test "Wait for Cluster‑API provider deployments" {
-  # Wait for Cluster‑API provider deployments
-  timeout 60 sh -ec 'until kubectl get deploy -n cozy-cluster-api capi-controller-manager capi-kamaji-controller-manager capi-kubeadm-bootstrap-controller-manager capi-operator-cluster-api-operator capk-controller-manager >/dev/null 2>&1; do sleep 1; done'
-  kubectl wait deployment/capi-controller-manager deployment/capi-kamaji-controller-manager deployment/capi-kubeadm-bootstrap-controller-manager deployment/capi-operator-cluster-api-operator deployment/capk-controller-manager -n cozy-cluster-api --timeout=1m --for=condition=available
-}
-
-@test "Wait for LINSTOR and configure storage" {
-  # Linstor controller and nodes
-  kubectl wait deployment/linstor-controller -n cozy-linstor --timeout=5m --for=condition=available
-  timeout 60 sh -ec 'until [ $(kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor node list | grep -c Online) -eq 3 ]; do sleep 1; done'
-
-  for node in srv1 srv2 srv3; do
-    kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor ps cdp zfs ${node} /dev/vdc --pool-name data --storage-pool data
-  done
-
-  # Storage classes
-  kubectl apply -f - <<'EOF'
----
-apiVersion: storage.k8s.io/v1
-kind: StorageClass
-metadata:
-  name: local
-  annotations:
-    storageclass.kubernetes.io/is-default-class: "true"
-provisioner: linstor.csi.linbit.com
-parameters:
-  linstor.csi.linbit.com/storagePool: "data"
-  linstor.csi.linbit.com/layerList: "storage"
-  linstor.csi.linbit.com/allowRemoteVolumeAccess: "false"
-volumeBindingMode: WaitForFirstConsumer
-allowVolumeExpansion: true
----
-apiVersion: storage.k8s.io/v1
-kind: StorageClass
-metadata:
-  name: replicated
-provisioner: linstor.csi.linbit.com
-parameters:
-  linstor.csi.linbit.com/storagePool: "data"
-  linstor.csi.linbit.com/autoPlace: "3"
-  linstor.csi.linbit.com/layerList: "drbd storage"
-  linstor.csi.linbit.com/allowRemoteVolumeAccess: "true"
-  property.linstor.csi.linbit.com/DrbdOptions/auto-quorum: suspend-io
-  property.linstor.csi.linbit.com/DrbdOptions/Resource/on-no-data-accessible: suspend-io
-  property.linstor.csi.linbit.com/DrbdOptions/Resource/on-suspended-primary-outdated: force-secondary
-  property.linstor.csi.linbit.com/DrbdOptions/Net/rr-conflict: retry-connect
-volumeBindingMode: WaitForFirstConsumer
-allowVolumeExpansion: true
-EOF
-}
-
-@test "Wait for MetalLB and configure address pool" {
-  # MetalLB address pool
-  kubectl apply -f - <<'EOF'
----
-apiVersion: metallb.io/v1beta1
-kind: L2Advertisement
-metadata:
-  name: cozystack
-  namespace: cozy-metallb
-spec:
-  ipAddressPools: [cozystack]
----
-apiVersion: metallb.io/v1beta1
-kind: IPAddressPool
-metadata:
-  name: cozystack
-  namespace: cozy-metallb
-spec:
-  addresses: [192.168.123.200-192.168.123.250]
-  autoAssign: true
-  avoidBuggyIPs: false
-EOF
-}
-
-@test "Check Cozystack API service" {
-  kubectl wait --for=condition=Available apiservices/v1alpha1.apps.cozystack.io --timeout=2m
-}
-
-@test "Configure Tenant and wait for applications" {
-  # Patch root tenant and wait for its releases
-  kubectl patch tenants/root -n tenant-root --type merge -p '{"spec":{"host":"example.org","ingress":true,"monitoring":true,"etcd":true,"isolated":true}}'
-
-  timeout 60 sh -ec 'until kubectl get hr -n tenant-root etcd ingress monitoring tenant-root >/dev/null 2>&1; do sleep 1; done'
-  kubectl wait hr/etcd hr/ingress hr/tenant-root -n tenant-root --timeout=2m --for=condition=ready
-
-  if ! kubectl wait hr/monitoring -n tenant-root --timeout=2m --for=condition=ready; then
-    flux reconcile hr monitoring -n tenant-root --force
-    kubectl wait hr/monitoring -n tenant-root --timeout=2m --for=condition=ready
-  fi
-
-  # Expose Cozystack services through ingress
-  kubectl patch configmap/cozystack -n cozy-system --type merge -p '{"data":{"expose-services":"api,dashboard,cdi-uploadproxy,vm-exportproxy,keycloak"}}'
-
-  # NGINX ingress controller
-  timeout 60 sh -ec 'until kubectl get deploy root-ingress-controller -n tenant-root >/dev/null 2>&1; do sleep 1; done'
-  kubectl wait deploy/root-ingress-controller -n tenant-root --timeout=5m --for=condition=available
-
-  # etcd statefulset
-  kubectl wait sts/etcd -n tenant-root --for=jsonpath='{.status.readyReplicas}'=3 --timeout=5m
-
-  # VictoriaMetrics components
-  kubectl wait vmalert/vmalert-shortterm vmalertmanager/alertmanager -n tenant-root --for=jsonpath='{.status.updateStatus}'=operational --timeout=5m
-  kubectl wait vlogs/generic -n tenant-root --for=jsonpath='{.status.updateStatus}'=operational --timeout=5m
-  kubectl wait vmcluster/shortterm vmcluster/longterm -n tenant-root --for=jsonpath='{.status.clusterStatus}'=operational --timeout=5m
-
-  # Grafana
-  kubectl wait clusters.postgresql.cnpg.io/grafana-db -n tenant-root --for=condition=ready --timeout=5m
-  kubectl wait deploy/grafana-deployment -n tenant-root --for=condition=available --timeout=5m
-
-  # Verify Grafana via ingress
-  ingress_ip=$(kubectl get svc root-ingress-controller -n tenant-root -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
-  if ! curl -sS -k "https://${ingress_ip}" -H 'Host: grafana.example.org' --max-time 30 | grep -q Found; then
-    echo "Failed to access Grafana via ingress at ${ingress_ip}" >&2
-    exit 1
-  fi
-}
-
-@test "Keycloak OIDC stack is healthy" {
-  kubectl patch configmap/cozystack -n cozy-system --type merge -p '{"data":{"oidc-enabled":"true"}}'
-
-  timeout 120 sh -ec 'until kubectl get hr -n cozy-keycloak keycloak keycloak-configure keycloak-operator >/dev/null 2>&1; do sleep 1; done'
-  kubectl wait hr/keycloak hr/keycloak-configure hr/keycloak-operator -n cozy-keycloak --timeout=10m --for=condition=ready
-}

hack/e2e-install-cozystack.bats

@@ -1,5 +1,12 @@
 #!/usr/bin/env bats
 
+@test "Required installer assets exist" {
+  if [ ! -f _out/assets/cozystack-installer.yaml ]; then
+    echo "Missing: _out/assets/cozystack-installer.yaml" >&2
+    exit 1
+  fi
+}
+
 @test "Install Cozystack" {
   # Create namespace & configmap required by installer
   kubectl create namespace cozy-system --dry-run=client -o yaml | kubectl apply -f -
@@ -20,14 +27,14 @@
   kubectl wait deployment/cozystack -n cozy-system --timeout=1m --for=condition=Available
 
   # Wait until HelmReleases appear & reconcile them
-  timeout 60 sh -ec 'until kubectl get hr -A | grep -q cozys; do sleep 1; done'
+  timeout 60 sh -ec 'until kubectl get hr -A -l cozystack.io/system-app=true | grep -q cozys; do sleep 1; done'
   sleep 5
-  kubectl get hr -A | awk 'NR>1 {print "kubectl wait --timeout=15m --for=condition=ready -n "$1" hr/"$2" &"} END {print "wait"}' | sh -ex
+  kubectl get hr -A -l cozystack.io/system-app=true | awk 'NR>1 {print "kubectl wait --timeout=15m --for=condition=ready -n "$1" hr/"$2" &"} END {print "wait"}' | sh -ex
 
   # Fail the test if any HelmRelease is not Ready
   if kubectl get hr -A | grep -v " True " | grep -v NAME; then
     kubectl get hr -A
-    fail "Some HelmReleases failed to reconcile"
+    echo "Some HelmReleases failed to reconcile" >&2
   fi
 }
 
@@ -42,7 +49,11 @@
   kubectl wait deployment/linstor-controller -n cozy-linstor --timeout=5m --for=condition=available
   timeout 60 sh -ec 'until [ $(kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor node list | grep -c Online) -eq 3 ]; do sleep 1; done'
 
+  created_pools=$(kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor sp l -s data --pastable | awk '$2 == "data" {printf " " $4} END{printf " "}')
   for node in srv1 srv2 srv3; do
+    case $created_pools in
+      *" $node "*) echo "Storage pool 'data' already exists on node $node"; continue;;
+    esac
     kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor ps cdp zfs ${node} /dev/vdc --pool-name data --storage-pool data
   done
 
@@ -155,3 +166,24 @@ EOF
   timeout 120 sh -ec 'until kubectl get hr -n cozy-keycloak keycloak keycloak-configure keycloak-operator >/dev/null 2>&1; do sleep 1; done'
   kubectl wait hr/keycloak hr/keycloak-configure hr/keycloak-operator -n cozy-keycloak --timeout=10m --for=condition=ready
 }
+
+@test "Create tenant with isolated mode enabled" {
+  kubectl -n tenant-root get tenants.apps.cozystack.io test || 
+  kubectl apply -f - <<EOF
+apiVersion: apps.cozystack.io/v1alpha1
+kind: Tenant
+metadata:
+  name: test
+  namespace: tenant-root
+spec:
+  etcd: false
+  host: ""
+  ingress: false
+  isolated: true
+  monitoring: false
+  resourceQuotas: {}
+  seaweedfs: false
+EOF
+  kubectl wait hr/tenant-test -n tenant-root --timeout=1m --for=condition=ready
+  kubectl wait namespace tenant-test --timeout=20s --for=jsonpath='{.status.phase}'=Active
+}

hack/e2e-prepare-cluster.bats

@@ -4,11 +4,6 @@
 # -----------------------------------------------------------------------------
 
 @test "Required installer assets exist" {
-  if [ ! -f _out/assets/cozystack-installer.yaml ]; then
-    echo "Missing: _out/assets/cozystack-installer.yaml" >&2
-    exit 1
-  fi
-
   if [ ! -f _out/assets/nocloud-amd64.raw.xz ]; then
     echo "Missing: _out/assets/nocloud-amd64.raw.xz" >&2
     exit 1
@@ -141,7 +136,25 @@ machine:
     mirrors:
       docker.io:
         endpoints:
-        - https://mirror.gcr.io
+        - https://dockerio.nexus.lllamnyp.su
+      cr.fluentbit.io:
+        endpoints:
+        - https://fluentbit.nexus.lllamnyp.su
+      docker-registry3.mariadb.com:
+        endpoints:
+        - https://mariadb.nexus.lllamnyp.su
+      gcr.io:
+        endpoints:
+        - https://gcr.nexus.lllamnyp.su
+      ghcr.io:
+        endpoints:
+        - https://ghcr.nexus.lllamnyp.su
+      quay.io:
+        endpoints:
+        - https://quay.nexus.lllamnyp.su
+      registry.k8s.io:
+        endpoints:
+        - https://k8s.nexus.lllamnyp.su
   files:
   - content: |
       [plugins]

packages/apps/clickhouse/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.10.0
+version: 0.11.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/clickhouse/Makefile

@@ -5,6 +5,7 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json
 
 image:
 	docker buildx build images/clickhouse-backup \

packages/apps/clickhouse/README.md

@@ -1,18 +1,19 @@
-# Managed Clickhouse Service
+# Managed ClickHouse Service
 
 ClickHouse is an open source high-performance and column-oriented SQL database management system (DBMS).
 It is used for online analytical processing (OLAP).
-Cozystack platform uses Altinity operator to provide ClickHouse.
 
-### How to restore backup:
+### How to restore backup from S3
 
-1. Find a snapshot:
-    ```
+1.  Find the snapshot:
+
+    ```bash
     restic -r s3:s3.example.org/clickhouse-backups/table_name snapshots
     ```
 
 2.  Restore it:
-    ```
+
+    ```bash
     restic -r s3:s3.example.org/clickhouse-backups/table_name restore latest --target /tmp/
     ```
 
@@ -39,32 +40,41 @@ For more details, read [Restic: Effective Backup from Stdin](https://blog.aenix.
 
 ### Backup parameters
 
-| Name                     | Description                                                                 | Value                                                  |
-| ------------------------ | --------------------------------------------------------------------------- | ------------------------------------------------------ |
-| `backup.enabled`         | Enable periodic backups                                                     | `false`                                                |
-| `backup.s3Region`        | AWS S3 region where backups are stored                                      | `us-east-1`                                            |
-| `backup.s3Bucket`        | S3 bucket used for storing backups                                          | `s3.example.org/clickhouse-backups`                    |
-| `backup.schedule`        | Cron schedule for automated backups                                         | `0 2 * * *`                                            |
-| `backup.cleanupStrategy` | Retention strategy for cleaning up old backups                              | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
-| `backup.s3AccessKey`     | Access key for S3, used for authentication                                  | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
-| `backup.s3SecretKey`     | Secret key for S3, used for authentication                                  | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
-| `backup.resticPassword`  | Password for Restic backup encryption                                       | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
-| `resources`              | Explicit CPU/memory resource requests and limits for the Clickhouse service | `{}`                                                   |
-| `resourcesPreset`        | Use a common resources preset when `resources` is not set explicitly.       | `nano`                                                 |
+| Name                     | Description                                                                                                                             | Value                                                  |
+| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
+| `backup.enabled`         | Enable periodic backups                                                                                                                 | `false`                                                |
+| `backup.s3Region`        | AWS S3 region where backups are stored                                                                                                  | `us-east-1`                                            |
+| `backup.s3Bucket`        | S3 bucket used for storing backups                                                                                                      | `s3.example.org/clickhouse-backups`                    |
+| `backup.schedule`        | Cron schedule for automated backups                                                                                                     | `0 2 * * *`                                            |
+| `backup.cleanupStrategy` | Retention strategy for cleaning up old backups                                                                                          | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
+| `backup.s3AccessKey`     | Access key for S3, used for authentication                                                                                              | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
+| `backup.s3SecretKey`     | Secret key for S3, used for authentication                                                                                              | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
+| `backup.resticPassword`  | Password for Restic backup encryption                                                                                                   | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
+| `resources`              | Explicit CPU and memory configuration for each ClickHouse replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`                                                   |
+| `resourcesPreset`        | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.       | `small`                                                |
 
+## Parameter examples and reference
 
-In production environments, it's recommended to set `resources` explicitly.
-Example of `resources`:
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
 
 ```yaml
 resources:
-  limits:
-    cpu: 4000m
-    memory: 4Gi
-  requests:
-    cpu: 100m
-    memory: 512Mi
+  cpu: 4000m
+  memory: 4Gi
 ```
 
-Allowed values for `resourcesPreset` are `none`, `nano`, `micro`, `small`, `medium`, `large`, `xlarge`, `2xlarge`.
-This value is ignored if `resources` value is set. 
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |

packages/apps/clickhouse/images/clickhouse-backup.tag

@@ -1 +1 @@
-ghcr.io/cozystack/cozystack/clickhouse-backup:0.10.0@sha256:3faf7a4cebf390b9053763107482de175aa0fdb88c1e77424fd81100b1c3a205
+ghcr.io/cozystack/cozystack/clickhouse-backup:0.11.0@sha256:3faf7a4cebf390b9053763107482de175aa0fdb88c1e77424fd81100b1c3a205

packages/apps/clickhouse/templates/clickhouse.yaml

@@ -132,11 +132,7 @@ spec:
           containers:
             - name: clickhouse
               image: clickhouse/clickhouse-server:24.9.2.42
-              {{- if .Values.resources }}
-              resources: {{- include "cozy-lib.resources.sanitize" (list .Values.resources $) | nindent 16 }}
-              {{- else if ne .Values.resourcesPreset "none" }}
-              resources: {{- include "cozy-lib.resources.preset" (list .Values.resourcesPreset $) | nindent 16 }}
-              {{- end }}
+              resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 16 }}
               volumeMounts:
                 - name: data-volume-template
                   mountPath: /var/lib/clickhouse

packages/apps/clickhouse/values.schema.json

@@ -79,13 +79,23 @@
         },
         "resources": {
             "type": "object",
-            "description": "Explicit CPU/memory resource requests and limits for the Clickhouse service",
+            "description": "Explicit CPU and memory configuration for each ClickHouse replica. When left empty, the preset defined in `resourcesPreset` is applied.",
             "default": {}
         },
         "resourcesPreset": {
             "type": "string",
-            "description": "Use a common resources preset when `resources` is not set explicitly.",
-            "default": "nano"
+            "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+            "default": "small",
+            "enum": [
+                "none",
+                "nano",
+                "micro",
+                "small",
+                "medium",
+                "large",
+                "xlarge",
+                "2xlarge"
+            ]
         }
     }
-}
+}

packages/apps/clickhouse/values.yaml

@@ -47,15 +47,11 @@ backup:
   s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
   resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
 
-## @param resources Explicit CPU/memory resource requests and limits for the Clickhouse service
+## @param resources Explicit CPU and memory configuration for each ClickHouse replica. When left empty, the preset defined in `resourcesPreset` is applied.
 resources: {}
  # resources:
- #   limits:
- #     cpu: 4000m
- #     memory: 4Gi
- #   requests:
- #     cpu: 100m
- #     memory: 512Mi
+ #   cpu: 4000m
+ #   memory: 4Gi
 
-## @param resourcesPreset Use a common resources preset when `resources` is not set explicitly.
-resourcesPreset: "nano"
+## @param resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
+resourcesPreset: "small"

packages/apps/ferretdb/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.7.0
+version: 0.8.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/ferretdb/Makefile

@@ -2,3 +2,4 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json

packages/apps/ferretdb/README.md

@@ -1,17 +1,21 @@
 # Managed FerretDB Service
 
+FerretDB is an open source MongoDB alternative.
+It translates MongoDB wire protocol queries to SQL and can be used as a direct replacement for MongoDB 5.0+.
+Internally, FerretDB service is backed by Postgres.
+
 ## Parameters
 
 ### Common parameters
 
-| Name                     | Description                                                                                                             | Value   |
-| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | ------- |
-| `external`               | Enable external access from outside the cluster                                                                         | `false` |
-| `size`                   | Persistent Volume size                                                                                                  | `10Gi`  |
-| `replicas`               | Number of Postgres replicas                                                                                             | `2`     |
-| `storageClass`           | StorageClass used to store the data                                                                                     | `""`    |
-| `quorum.minSyncReplicas` | Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed.           | `0`     |
-| `quorum.maxSyncReplicas` | Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the number of instances). | `0`     |
+| Name                     | Description                                                                                                                 | Value   |
+| ------------------------ | --------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `external`               | Enable external access from outside the cluster                                                                             | `false` |
+| `size`                   | Persistent Volume size                                                                                                      | `10Gi`  |
+| `replicas`               | Number of replicas                                                                                                          | `2`     |
+| `storageClass`           | StorageClass used to store the data                                                                                         | `""`    |
+| `quorum.minSyncReplicas` | Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed                | `0`     |
+| `quorum.maxSyncReplicas` | Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the total number of replicas) | `0`     |
 
 ### Configuration parameters
 
@@ -21,17 +25,43 @@
 
 ### Backup parameters
 
-| Name                     | Description                                                                                                                                                                                                       | Value                                                  |
-| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
-| `backup.enabled`         | Enable pereiodic backups                                                                                                                                                                                          | `false`                                                |
-| `backup.s3Region`        | The AWS S3 region where backups are stored                                                                                                                                                                        | `us-east-1`                                            |
-| `backup.s3Bucket`        | The S3 bucket used for storing backups                                                                                                                                                                            | `s3.example.org/postgres-backups`                      |
-| `backup.schedule`        | Cron schedule for automated backups                                                                                                                                                                               | `0 2 * * *`                                            |
-| `backup.cleanupStrategy` | The strategy for cleaning up old backups                                                                                                                                                                          | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
-| `backup.s3AccessKey`     | The access key for S3, used for authentication                                                                                                                                                                    | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
-| `backup.s3SecretKey`     | The secret key for S3, used for authentication                                                                                                                                                                    | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
-| `backup.resticPassword`  | The password for Restic backup encryption                                                                                                                                                                         | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
-| `resources`              | Resources                                                                                                                                                                                                         | `{}`                                                   |
-| `resourcesPreset`        | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano`                                                 |
+| Name                     | Description                                                                                                                           | Value                                                  |
+| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
+| `backup.enabled`         | Enable periodic backups                                                                                                               | `false`                                                |
+| `backup.s3Region`        | The AWS S3 region where backups are stored                                                                                            | `us-east-1`                                            |
+| `backup.s3Bucket`        | The S3 bucket used for storing backups                                                                                                | `s3.example.org/postgres-backups`                      |
+| `backup.schedule`        | Cron schedule for automated backups                                                                                                   | `0 2 * * *`                                            |
+| `backup.cleanupStrategy` | The strategy for cleaning up old backups                                                                                              | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
+| `backup.s3AccessKey`     | The access key for S3, used for authentication                                                                                        | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
+| `backup.s3SecretKey`     | The secret key for S3, used for authentication                                                                                        | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
+| `backup.resticPassword`  | The password for Restic backup encryption                                                                                             | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
+| `resources`              | Explicit CPU and memory configuration for each FerretDB replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`                                                   |
+| `resourcesPreset`        | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.     | `nano`                                                 |
 
 
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |

packages/apps/ferretdb/templates/postgres.yaml

@@ -18,11 +18,7 @@ spec:
   {{- end }}
   minSyncReplicas: {{ .Values.quorum.minSyncReplicas }}
   maxSyncReplicas: {{ .Values.quorum.maxSyncReplicas }}
-  {{- if .Values.resources }}
-  resources: {{- include "cozy-lib.resources.sanitize" (list .Values.resources $) | nindent 4 }}
-  {{- else if ne .Values.resourcesPreset "none" }}
-  resources: {{- include "cozy-lib.resources.preset" (list .Values.resourcesPreset $) | nindent 4 }}
-  {{- end }}
+  resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 4 }}
   monitoring:
     enablePodMonitor: true
 

packages/apps/ferretdb/values.schema.json

@@ -14,7 +14,7 @@
         },
         "replicas": {
             "type": "number",
-            "description": "Number of Postgres replicas",
+            "description": "Number of replicas",
             "default": 2
         },
         "storageClass": {
@@ -27,12 +27,12 @@
             "properties": {
                 "minSyncReplicas": {
                     "type": "number",
-                    "description": "Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed.",
+                    "description": "Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed",
                     "default": 0
                 },
                 "maxSyncReplicas": {
                     "type": "number",
-                    "description": "Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the number of instances).",
+                    "description": "Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the total number of replicas)",
                     "default": 0
                 }
             }
@@ -42,7 +42,7 @@
             "properties": {
                 "enabled": {
                     "type": "boolean",
-                    "description": "Enable pereiodic backups",
+                    "description": "Enable periodic backups",
                     "default": false
                 },
                 "s3Region": {
@@ -84,13 +84,23 @@
         },
         "resources": {
             "type": "object",
-            "description": "Resources",
+            "description": "Explicit CPU and memory configuration for each FerretDB replica. When left empty, the preset defined in `resourcesPreset` is applied.",
             "default": {}
         },
         "resourcesPreset": {
             "type": "string",
-            "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-            "default": "nano"
+            "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+            "default": "nano",
+            "enum": [
+                "none",
+                "nano",
+                "micro",
+                "small",
+                "medium",
+                "large",
+                "xlarge",
+                "2xlarge"
+            ]
         }
     }
-}
+}

packages/apps/ferretdb/values.yaml

@@ -2,7 +2,7 @@
 
 ## @param external Enable external access from outside the cluster
 ## @param size Persistent Volume size
-## @param replicas Number of Postgres replicas
+## @param replicas Number of replicas
 ## @param storageClass StorageClass used to store the data
 ##
 external: false
@@ -11,8 +11,8 @@ replicas: 2
 storageClass: ""
 
 ## Configuration for the quorum-based synchronous replication
-## @param quorum.minSyncReplicas Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed.
-## @param quorum.maxSyncReplicas Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the number of instances).
+## @param quorum.minSyncReplicas Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed
+## @param quorum.maxSyncReplicas Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the total number of replicas)
 quorum:
   minSyncReplicas: 0
   maxSyncReplicas: 0
@@ -31,7 +31,7 @@ users: {}
 
 ## @section Backup parameters
 
-## @param backup.enabled Enable pereiodic backups
+## @param backup.enabled Enable periodic backups
 ## @param backup.s3Region The AWS S3 region where backups are stored
 ## @param backup.s3Bucket The S3 bucket used for storing backups
 ## @param backup.schedule Cron schedule for automated backups
@@ -49,15 +49,11 @@ backup:
   s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
   resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
 
-## @param resources Resources
+## @param resources Explicit CPU and memory configuration for each FerretDB replica. When left empty, the preset defined in `resourcesPreset` is applied.
 resources: {}
  # resources:
- #   limits:
- #     cpu: 4000m
- #     memory: 4Gi
- #   requests:
- #     cpu: 100m
- #     memory: 512Mi
- 
-## @param resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
+ #   cpu: 4000m
+ #   memory: 4Gi
+
+## @param resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
 resourcesPreset: "nano"

packages/apps/http-cache/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.5.1
+version: 0.6.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/http-cache/Makefile

@@ -23,6 +23,8 @@ image-nginx:
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.haproxy.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json
+	yq -i -o json --indent 4 '.properties.nginx.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json
 
 update:
 	tag=$$(git ls-remote --tags --sort="v:refname" https://github.com/chrislim2888/IP2Location-C-Library | awk -F'[/^]' 'END{print $$3}') && \

packages/apps/http-cache/README.md

@@ -1,8 +1,9 @@
-# Managed Nginx Caching Service
+# Managed Nginx-based HTTP Cache Service
 
-The Nginx Caching Service is designed to optimize web traffic and enhance web application performance. This service combines custom-built Nginx instances with HAproxy for efficient caching and load balancing.
+The Nginx-based HTTP caching service is designed to optimize web traffic and enhance web application performance.
+This service combines custom-built Nginx instances with HAProxy for efficient caching and load balancing.
 
-## Deployment infromation
+## Deployment information
 
 The Nginx instances include the following modules and features:
 
@@ -53,27 +54,67 @@ The deployment architecture is illustrated in the diagram below:
 
 ## Known issues
 
-VTS module shows wrong upstream resonse time
-- https://github.com/vozlt/nginx-module-vts/issues/198
+- VTS module shows wrong upstream response time, [github.com/vozlt/nginx-module-vts#198](https://github.com/vozlt/nginx-module-vts/issues/198)
 
 ## Parameters
 
 ### Common parameters
 
-| Name                      | Description                                                                                                                                                                                                       | Value   |
-| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
-| `external`                | Enable external access from outside the cluster                                                                                                                                                                   | `false` |
-| `size`                    | Persistent Volume size                                                                                                                                                                                            | `10Gi`  |
-| `storageClass`            | StorageClass used to store the data                                                                                                                                                                               | `""`    |
-| `haproxy.replicas`        | Number of HAProxy replicas                                                                                                                                                                                        | `2`     |
-| `nginx.replicas`          | Number of Nginx replicas                                                                                                                                                                                          | `2`     |
-| `haproxy.resources`       | Resources                                                                                                                                                                                                         | `{}`    |
-| `haproxy.resourcesPreset` | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano`  |
-| `nginx.resources`         | Resources                                                                                                                                                                                                         | `{}`    |
-| `nginx.resourcesPreset`   | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano`  |
+| Name                      | Description                                                                                                                          | Value   |
+| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ------- |
+| `external`                | Enable external access from outside the cluster                                                                                      | `false` |
+| `size`                    | Persistent Volume size                                                                                                               | `10Gi`  |
+| `storageClass`            | StorageClass used to store the data                                                                                                  | `""`    |
+| `haproxy.replicas`        | Number of HAProxy replicas                                                                                                           | `2`     |
+| `nginx.replicas`          | Number of Nginx replicas                                                                                                             | `2`     |
+| `haproxy.resources`       | Explicit CPU and memory configuration for each HAProxy replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`    |
+| `haproxy.resourcesPreset` | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.    | `nano`  |
+| `nginx.resources`         | Explicit CPU and memory configuration for each nginx replica. When left empty, the preset defined in `resourcesPreset` is applied.   | `{}`    |
+| `nginx.resourcesPreset`   | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.    | `nano`  |
 
 ### Configuration parameters
 
 | Name        | Description             | Value |
 | ----------- | ----------------------- | ----- |
 | `endpoints` | Endpoints configuration | `[]`  |
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |
+
+
+### endpoints
+
+`endpoints` is a flat list of IP addresses:
+
+```yaml
+endpoints:
+  - 10.100.3.1:80
+  - 10.100.3.11:80
+  - 10.100.3.2:80
+  - 10.100.3.12:80
+  - 10.100.3.3:80
+  - 10.100.3.13:80
+```

packages/apps/http-cache/images/nginx-cache.tag

@@ -1 +1 @@
-ghcr.io/cozystack/cozystack/nginx-cache:0.5.1@sha256:50ac1581e3100bd6c477a71161cb455a341ffaf9e5e2f6086802e4e25271e8af
+ghcr.io/cozystack/cozystack/nginx-cache:0.6.0@sha256:50ac1581e3100bd6c477a71161cb455a341ffaf9e5e2f6086802e4e25271e8af

packages/apps/http-cache/templates/haproxy/deployment.yaml

@@ -33,11 +33,7 @@ spec:
       containers:
       - image: haproxy:latest
         name: haproxy
-        {{- if .Values.haproxy.resources }}
-        resources: {{- include "cozy-lib.resources.sanitize" (list .Values.haproxy.resources $) | nindent 10 }}
-        {{- else if ne .Values.haproxy.resourcesPreset "none" }}
-        resources: {{- include "cozy-lib.resources.preset" (list .Values.haproxy.resourcesPreset $) | nindent 10 }}
-        {{- end }}
+        resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.haproxy.resourcesPreset .Values.haproxy.resources $) | nindent 10 }}
         ports:
         - containerPort: 8080
           name: http

packages/apps/http-cache/templates/nginx/deployment.yaml

@@ -52,11 +52,7 @@ spec:
       shareProcessNamespace: true
       containers:
       - name: nginx
-        {{- if $.Values.nginx.resources }}
-        resources: {{- include "cozy-lib.resources.sanitize" (list $.Values.nginx.resources $) | nindent 10 }}
-        {{- else if ne $.Values.nginx.resourcesPreset "none" }}
-        resources: {{- include "cozy-lib.resources.preset" (list $.Values.nginx.resourcesPreset $) | nindent 10 }}
-        {{- end }}
+        resources: {{- include "cozy-lib.resources.defaultingSanitize" (list $.Values.nginx.resourcesPreset $.Values.nginx.resources $) | nindent 10 }}
         image: "{{ $.Files.Get "images/nginx-cache.tag" | trim }}"
         readinessProbe:
           httpGet:

packages/apps/http-cache/values.schema.json

@@ -27,13 +27,23 @@
                 },
                 "resources": {
                     "type": "object",
-                    "description": "Resources",
+                    "description": "Explicit CPU and memory configuration for each HAProxy replica. When left empty, the preset defined in `resourcesPreset` is applied.",
                     "default": {}
                 },
                 "resourcesPreset": {
                     "type": "string",
-                    "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-                    "default": "nano"
+                    "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+                    "default": "nano",
+                    "enum": [
+                        "none",
+                        "nano",
+                        "micro",
+                        "small",
+                        "medium",
+                        "large",
+                        "xlarge",
+                        "2xlarge"
+                    ]
                 }
             }
         },
@@ -47,13 +57,23 @@
                 },
                 "resources": {
                     "type": "object",
-                    "description": "Resources",
+                    "description": "Explicit CPU and memory configuration for each nginx replica. When left empty, the preset defined in `resourcesPreset` is applied.",
                     "default": {}
                 },
                 "resourcesPreset": {
                     "type": "string",
-                    "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-                    "default": "nano"
+                    "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+                    "default": "nano",
+                    "enum": [
+                        "none",
+                        "nano",
+                        "micro",
+                        "small",
+                        "medium",
+                        "large",
+                        "xlarge",
+                        "2xlarge"
+                    ]
                 }
             }
         },
@@ -64,4 +84,4 @@
             "items": {}
         }
     }
-}
+}

packages/apps/http-cache/values.yaml

@@ -12,31 +12,23 @@ size: 10Gi
 storageClass: ""
 haproxy:
   replicas: 2
-  ## @param haproxy.resources Resources
+  ## @param haproxy.resources Explicit CPU and memory configuration for each HAProxy replica. When left empty, the preset defined in `resourcesPreset` is applied.
   resources: {}
   # resources:
-  #   limits:
-  #     cpu: 4000m
-  #     memory: 4Gi
-  #   requests:
-  #     cpu: 100m
-  #     memory: 512Mi
-  
-  ## @param haproxy.resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
+  #   cpu: 4000m
+  #   memory: 4Gi
+
+  ## @param haproxy.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
   resourcesPreset: "nano"
 nginx:
   replicas: 2
-  ## @param nginx.resources Resources
+  ## @param nginx.resources Explicit CPU and memory configuration for each nginx replica. When left empty, the preset defined in `resourcesPreset` is applied.
   resources: {}
   # resources:
-  #   limits:
-  #     cpu: 4000m
-  #     memory: 4Gi
-  #   requests:
-  #     cpu: 100m
-  #     memory: 512Mi
-  
-  ## @param nginx.resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
+  #   cpu: 4000m
+  #   memory: 4Gi
+
+  ## @param nginx.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
   resourcesPreset: "nano"
 
 ## @section Configuration parameters

packages/apps/kafka/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.7.0
+version: 0.8.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/kafka/Makefile

@@ -2,3 +2,5 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.kafka.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json
+	yq -i -o json --indent 4 '.properties.zookeeper.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json

packages/apps/kafka/README.md

@@ -4,22 +4,67 @@
 
 ### Common parameters
 
-| Name                        | Description                                                                                                                                                                                                       | Value   |
-| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
-| `external`                  | Enable external access from outside the cluster                                                                                                                                                                   | `false` |
-| `kafka.size`                | Persistent Volume size for Kafka                                                                                                                                                                                  | `10Gi`  |
-| `kafka.replicas`            | Number of Kafka replicas                                                                                                                                                                                          | `3`     |
-| `kafka.storageClass`        | StorageClass used to store the Kafka data                                                                                                                                                                         | `""`    |
-| `zookeeper.size`            | Persistent Volume size for ZooKeeper                                                                                                                                                                              | `5Gi`   |
-| `zookeeper.replicas`        | Number of ZooKeeper replicas                                                                                                                                                                                      | `3`     |
-| `zookeeper.storageClass`    | StorageClass used to store the ZooKeeper data                                                                                                                                                                     | `""`    |
-| `kafka.resources`           | Resources                                                                                                                                                                                                         | `{}`    |
-| `kafka.resourcesPreset`     | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `small` |
-| `zookeeper.resources`       | Resources                                                                                                                                                                                                         | `{}`    |
-| `zookeeper.resourcesPreset` | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `micro` |
+| Name                        | Description                                                                                                                            | Value   |
+| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `external`                  | Enable external access from outside the cluster                                                                                        | `false` |
+| `kafka.size`                | Persistent Volume size for Kafka                                                                                                       | `10Gi`  |
+| `kafka.replicas`            | Number of Kafka replicas                                                                                                               | `3`     |
+| `kafka.storageClass`        | StorageClass used to store the Kafka data                                                                                              | `""`    |
+| `zookeeper.size`            | Persistent Volume size for ZooKeeper                                                                                                   | `5Gi`   |
+| `zookeeper.replicas`        | Number of ZooKeeper replicas                                                                                                           | `3`     |
+| `zookeeper.storageClass`    | StorageClass used to store the ZooKeeper data                                                                                          | `""`    |
+| `kafka.resources`           | Explicit CPU and memory configuration for each Kafka replica. When left empty, the preset defined in `resourcesPreset` is applied.     | `{}`    |
+| `kafka.resourcesPreset`     | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.      | `small` |
+| `zookeeper.resources`       | Explicit CPU and memory configuration for each Zookeeper replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`    |
+| `zookeeper.resourcesPreset` | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.      | `small` |
 
 ### Configuration parameters
 
 | Name     | Description          | Value |
 | -------- | -------------------- | ----- |
 | `topics` | Topics configuration | `[]`  |
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |
+
+### topics
+
+```yaml
+topics:
+  - name: Results
+    partitions: 1
+    replicas: 3
+    config:
+      min.insync.replicas: 2
+  - name: Orders
+    config:
+      cleanup.policy: compact
+      segment.ms: 3600000
+      max.compaction.lag.ms: 5400000
+      min.insync.replicas: 2
+    partitions: 1
+    replicas: 3
+```

packages/apps/kafka/templates/kafka.yaml

@@ -8,11 +8,7 @@ metadata:
 spec:
   kafka:
     replicas: {{ .Values.kafka.replicas }}
-    {{- if .Values.kafka.resources }}
-    resources: {{- include "cozy-lib.resources.sanitize" (list .Values.kafka.resources $) | nindent 6 }}
-    {{- else if ne .Values.kafka.resourcesPreset "none" }}
-    resources: {{- include "cozy-lib.resources.preset" (list .Values.kafka.resourcesPreset $) | nindent 6 }}
-    {{- end }}
+    resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.kafka.resourcesPreset .Values.kafka.resources $) | nindent 6 }}
     listeners:
       - name: plain
         port: 9092
@@ -70,11 +66,7 @@ spec:
           key: kafka-metrics-config.yml
   zookeeper:
     replicas: {{ .Values.zookeeper.replicas }}
-    {{- if .Values.zookeeper.resources }}
-    resources: {{- include "cozy-lib.resources.sanitize" (list .Values.zookeeper.resources $) | nindent 6 }}
-    {{- else if ne .Values.zookeeper.resourcesPreset "none" }}
-    resources: {{- include "cozy-lib.resources.preset" (list .Values.zookeeper.resourcesPreset $) | nindent 6 }}
-    {{- end }}
+    resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.zookeeper.resourcesPreset .Values.zookeeper.resources $) | nindent 6 }}
     storage:
       type: persistent-claim
       {{- with .Values.zookeeper.size }}

packages/apps/kafka/values.schema.json

@@ -27,13 +27,23 @@
                 },
                 "resources": {
                     "type": "object",
-                    "description": "Resources",
+                    "description": "Explicit CPU and memory configuration for each Kafka replica. When left empty, the preset defined in `resourcesPreset` is applied.",
                     "default": {}
                 },
                 "resourcesPreset": {
                     "type": "string",
-                    "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-                    "default": "small"
+                    "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+                    "default": "small",
+                    "enum": [
+                        "none",
+                        "nano",
+                        "micro",
+                        "small",
+                        "medium",
+                        "large",
+                        "xlarge",
+                        "2xlarge"
+                    ]
                 }
             }
         },
@@ -57,13 +67,23 @@
                 },
                 "resources": {
                     "type": "object",
-                    "description": "Resources",
+                    "description": "Explicit CPU and memory configuration for each Zookeeper replica. When left empty, the preset defined in `resourcesPreset` is applied.",
                     "default": {}
                 },
                 "resourcesPreset": {
                     "type": "string",
-                    "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-                    "default": "micro"
+                    "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+                    "default": "small",
+                    "enum": [
+                        "none",
+                        "nano",
+                        "micro",
+                        "small",
+                        "medium",
+                        "large",
+                        "xlarge",
+                        "2xlarge"
+                    ]
                 }
             }
         },
@@ -74,4 +94,4 @@
             "items": {}
         }
     }
-}
+}

packages/apps/kafka/values.yaml

@@ -14,35 +14,25 @@ kafka:
   size: 10Gi
   replicas: 3
   storageClass: ""
-  ## @param kafka.resources Resources
+  ## @param kafka.resources Explicit CPU and memory configuration for each Kafka replica. When left empty, the preset defined in `resourcesPreset` is applied.
   resources: {}
   # resources:
-  #   limits:
-  #     cpu: 4000m
-  #     memory: 4Gi
-  #   requests:
-  #     cpu: 100m
-  #     memory: 512Mi
-  
-  ## @param kafka.resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
+  #   cpu: 4000m
+  #   memory: 4Gi
+  ## @param kafka.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
   resourcesPreset: "small"
 
 zookeeper:
   size: 5Gi
   replicas: 3
   storageClass: ""
-  ## @param zookeeper.resources Resources
+  ## @param zookeeper.resources Explicit CPU and memory configuration for each Zookeeper replica. When left empty, the preset defined in `resourcesPreset` is applied.
   resources: {}
   # resources:
-  #   limits:
-  #     cpu: 4000m
-  #     memory: 4Gi
-  #   requests:
-  #     cpu: 100m
-  #     memory: 512Mi
-  
-  ## @param zookeeper.resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
-  resourcesPreset: "micro"
+  #   cpu: 4000m
+  #   memory: 4Gi
+  ## @param zookeeper.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
+  resourcesPreset: "small"
 
 ## @section Configuration parameters
 

packages/apps/kubernetes/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.24.1
+version: 0.25.2
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/kubernetes/Makefile

@@ -6,6 +6,7 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -o json -i '.properties.addons.properties.ingressNginx.properties.exposeMethod.enum = ["Proxied","LoadBalancer"]' values.schema.json
 	yq -o json -i '.properties.controlPlane.properties.apiServer.properties.resourcesPreset.enum = ["none","nano","micro","small","medium","large","xlarge","2xlarge"]' values.schema.json
 	yq -o json -i '.properties.controlPlane.properties.controllerManager.properties.resourcesPreset.enum = ["none","nano","micro","small","medium","large","xlarge","2xlarge"]' values.schema.json
 	yq -o json -i '.properties.controlPlane.properties.scheduler.properties.resourcesPreset.enum = ["none","nano","micro","small","medium","large","xlarge","2xlarge"]' values.schema.json
@@ -63,6 +64,8 @@ image-kubevirt-csi-driver:
 		--load=$(LOAD)
 	echo "$(REGISTRY)/kubevirt-csi-driver:$(call settag,$(KUBERNETES_PKG_TAG))@$$(yq e '."containerimage.digest"' images/kubevirt-csi-driver.json -o json -r)" \
 		> images/kubevirt-csi-driver.tag
+	IMAGE=$$(cat images/kubevirt-csi-driver.tag) \
+		yq -i '.csiDriver.image = strenv(IMAGE)' ../../system/kubevirt-csi-node/values.yaml
 	rm -f images/kubevirt-csi-driver.json
 
 

packages/apps/kubernetes/README.md

@@ -81,63 +81,74 @@ See the reference for components utilized in this service:
 
 ### Common Parameters
 
-| Name                                | Description                                                                                                       | Value        |
-| ----------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------ |
-| `host`                              | Hostname used to access the Kubernetes cluster externally. Defaults to `<cluster-name>.<tenant-host>` when empty. | `""`         |
-| `controlPlane.replicas`             | Number of replicas for Kubernetes control-plane components.                                                       | `2`          |
-| `storageClass`                      | StorageClass used to store user data.                                                                             | `replicated` |
-| `useCustomSecretForPatchContainerd` | if true, for patch containerd will be used secret: {{ .Release.Name }}-patch-containerd                           | `false`      |
-| `nodeGroups`                        | nodeGroups configuration                                                                                          | `{}`         |
+| Name                    | Description                                                                                                       | Value        |
+| ----------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------ |
+| `host`                  | Hostname used to access the Kubernetes cluster externally. Defaults to `<cluster-name>.<tenant-host>` when empty. | `""`         |
+| `controlPlane.replicas` | Number of replicas for Kubernetes control-plane components.                                                       | `2`          |
+| `storageClass`          | StorageClass used to store user data.                                                                             | `replicated` |
+| `nodeGroups`            | nodeGroups configuration                                                                                          | `{}`         |
 
 ### Cluster Addons
 
-| Name                                          | Description                                                                                                                                                                       | Value   |
-| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
-| `addons.certManager.enabled`                  | Enable cert-manager, which automatically creates and manages SSL/TLS certificates.                                                                                                | `false` |
-| `addons.certManager.valuesOverride`           | Custom values to override                                                                                                                                                         | `{}`    |
-| `addons.cilium.valuesOverride`                | Custom values to override                                                                                                                                                         | `{}`    |
-| `addons.gatewayAPI.enabled`                   | Enable the Gateway API                                                                                                                                                            | `false` |
-| `addons.ingressNginx.enabled`                 | Enable the Ingress-NGINX controller (requires nodes labeled with the 'ingress-nginx' role).                                                                                       | `false` |
-| `addons.ingressNginx.valuesOverride`          | Custom values to override                                                                                                                                                         | `{}`    |
-| `addons.ingressNginx.hosts`                   | List of domain names that the parent cluster should route to this tenant cluster.                                                                                                 | `[]`    |
-| `addons.gpuOperator.enabled`                  | Enable the GPU-operator                                                                                                                                                           | `false` |
-| `addons.gpuOperator.valuesOverride`           | Custom values to override                                                                                                                                                         | `{}`    |
-| `addons.fluxcd.enabled`                       | Enable FluxCD                                                                                                                                                                     | `false` |
-| `addons.fluxcd.valuesOverride`                | Custom values to override                                                                                                                                                         | `{}`    |
-| `addons.monitoringAgents.enabled`             | Enable monitoring agents (Fluent Bit and VMAgents) to send logs and metrics. If tenant monitoring is enabled, data is sent to tenant storage; otherwise, it goes to root storage. | `false` |
-| `addons.monitoringAgents.valuesOverride`      | Custom values to override                                                                                                                                                         | `{}`    |
-| `addons.verticalPodAutoscaler.valuesOverride` | Custom values to override                                                                                                                                                         | `{}`    |
+| Name                                          | Description                                                                                                                                                                       | Value     |
+| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- |
+| `addons.certManager.enabled`                  | Enable cert-manager, which automatically creates and manages SSL/TLS certificates.                                                                                                | `false`   |
+| `addons.certManager.valuesOverride`           | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.cilium.valuesOverride`                | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.gatewayAPI.enabled`                   | Enable the Gateway API                                                                                                                                                            | `false`   |
+| `addons.ingressNginx.enabled`                 | Enable the Ingress-NGINX controller (requires nodes labeled with the 'ingress-nginx' role).                                                                                       | `false`   |
+| `addons.ingressNginx.valuesOverride`          | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.ingressNginx.exposeMethod`            | Method to expose the Ingress-NGINX controller. (allowed values: Proxied, LoadBalancer)                                                                                            | `Proxied` |
+| `addons.ingressNginx.hosts`                   | List of domain names that the parent cluster should route to this tenant cluster. Taken into account only when `exposeMethod` is set to `Proxied`.                                | `[]`      |
+| `addons.gpuOperator.enabled`                  | Enable the GPU-operator                                                                                                                                                           | `false`   |
+| `addons.gpuOperator.valuesOverride`           | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.fluxcd.enabled`                       | Enable FluxCD                                                                                                                                                                     | `false`   |
+| `addons.fluxcd.valuesOverride`                | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.monitoringAgents.enabled`             | Enable monitoring agents (Fluent Bit and VMAgents) to send logs and metrics. If tenant monitoring is enabled, data is sent to tenant storage; otherwise, it goes to root storage. | `false`   |
+| `addons.monitoringAgents.valuesOverride`      | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.verticalPodAutoscaler.valuesOverride` | Custom values to override                                                                                                                                                         | `{}`      |
+| `addons.velero.enabled`                       | Enable velero for backup and restore k8s cluster.                                                                                                                                 | `false`   |
+| `addons.velero.valuesOverride`                | Custom values to override                                                                                                                                                         | `{}`      |
 
 ### Kubernetes Control Plane Configuration
 
-| Name                                               | Description                                                                  | Value    |
-| -------------------------------------------------- | ---------------------------------------------------------------------------- | -------- |
-| `controlPlane.apiServer.resources`                 | Explicit CPU/memory resource requests and limits for the API server.         | `{}`     |
-| `controlPlane.apiServer.resourcesPreset`           | Use a common resources preset when `resources` is not set explicitly.        | `medium` |
-| `controlPlane.controllerManager.resources`         | Explicit CPU/memory resource requests and limits for the controller manager. | `{}`     |
-| `controlPlane.controllerManager.resourcesPreset`   | Use a common resources preset when `resources` is not set explicitly.        | `micro`  |
-| `controlPlane.scheduler.resources`                 | Explicit CPU/memory resource requests and limits for the scheduler.          | `{}`     |
-| `controlPlane.scheduler.resourcesPreset`           | Use a common resources preset when `resources` is not set explicitly.        | `micro`  |
-| `controlPlane.konnectivity.server.resources`       | Explicit CPU/memory resource requests and limits for the Konnectivity.       | `{}`     |
-| `controlPlane.konnectivity.server.resourcesPreset` | Use a common resources preset when `resources` is not set explicitly.        | `micro`  |
+| Name                                               | Description                                                                                                                            | Value    |
+| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -------- |
+| `controlPlane.apiServer.resources`                 | Explicit CPU and memory configuration for the API Server. When left empty, the preset defined in `resourcesPreset` is applied.         | `{}`     |
+| `controlPlane.apiServer.resourcesPreset`           | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.      | `medium` |
+| `controlPlane.controllerManager.resources`         | Explicit CPU and memory configuration for the Controller Manager. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`     |
+| `controlPlane.controllerManager.resourcesPreset`   | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.      | `micro`  |
+| `controlPlane.scheduler.resources`                 | Explicit CPU and memory configuration for the Scheduler. When left empty, the preset defined in `resourcesPreset` is applied.          | `{}`     |
+| `controlPlane.scheduler.resourcesPreset`           | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.      | `micro`  |
+| `controlPlane.konnectivity.server.resources`       | Explicit CPU and memory configuration for Konnectivity. When left empty, the preset defined in `resourcesPreset` is applied.           | `{}`     |
+| `controlPlane.konnectivity.server.resourcesPreset` | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.      | `micro`  |
 
-In production environments, it's recommended to set `resources` explicitly.
-Example of `controlPlane.*.resources`:
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
 
 ```yaml
 resources:
-  limits:
-    cpu: 4000m
-    memory: 4Gi
-  requests:
-    cpu: 100m
-    memory: 512Mi
+  cpu: 4000m
+  memory: 4Gi
 ```
 
-Allowed values for `controlPlane.*.resourcesPreset` are `none`, `nano`, `micro`, `small`, `medium`, `large`, `xlarge`, `2xlarge`.
-This value is ignored if the corresponding `resources` value is set. 
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
 
-## Resources Reference
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |
 
 ### instanceType Resources
 
@@ -301,4 +312,3 @@ Specific characteristics of this series are:
   workload.
 - *vCPU-To-Memory Ratio (1:4)* - A vCPU-to-Memory ratio of 1:4 starting from
   the medium size.
-

packages/apps/kubernetes/images/cluster-autoscaler.tag

@@ -1 +1 @@
-ghcr.io/cozystack/cozystack/cluster-autoscaler:0.24.0@sha256:3a8170433e1632e5cc2b6d9db34d0605e8e6c63c158282c38450415e700e932e
+ghcr.io/cozystack/cozystack/cluster-autoscaler:0.25.2@sha256:3a8170433e1632e5cc2b6d9db34d0605e8e6c63c158282c38450415e700e932e

packages/apps/kubernetes/images/kubevirt-cloud-provider.tag

@@ -1 +1 @@
-ghcr.io/cozystack/cozystack/kubevirt-cloud-provider:0.24.0@sha256:b478952fab735f85c3ba15835012b1de8af5578b33a8a2670eaf532ffc17681e
+ghcr.io/cozystack/cozystack/kubevirt-cloud-provider:0.25.2@sha256:e522960064290747a67502d4e8927c591bdb290bad1f0bae88a02758ebfd380f

packages/apps/kubernetes/images/kubevirt-csi-driver.tag

@@ -1 +1 @@
-ghcr.io/cozystack/cozystack/kubevirt-csi-driver:0.24.0@sha256:4d3728b2050d4e0adb00b9f4abbb4a020b29e1a39f24ca1447806fc81f110fa6
+ghcr.io/cozystack/cozystack/kubevirt-csi-driver:0.25.2@sha256:761e7235ff9cb7f6f223f00954943e6a5af32ed6624ee592a8610122f96febb0

packages/apps/kubernetes/images/kubevirt-csi-driver/Dockerfile

@@ -3,7 +3,7 @@ ARG builder_image=docker.io/library/golang:1.22.5
 FROM ${builder_image} AS builder
 RUN git clone https://github.com/kubevirt/csi-driver /src/kubevirt-csi-driver \
  && cd /src/kubevirt-csi-driver \
- && git checkout 35836e0c8b68d9916d29a838ea60cdd3fc6199cf
+ && git checkout a8d6605bc9997bcfda3fb9f1f82ba6445b4984cc
 
 ARG TARGETOS
 ARG TARGETARCH
@@ -11,6 +11,7 @@ ENV GOOS=$TARGETOS
 ENV GOARCH=$TARGETARCH
 
 WORKDIR /src/kubevirt-csi-driver
+
 RUN make build
 
 FROM quay.io/centos/centos:stream9

packages/apps/kubernetes/templates/cluster.yaml

@@ -120,23 +120,11 @@ metadata:
     kamaji.clastix.io/kubeconfig-secret-key: "super-admin.svc"
 spec:
   apiServer:
-    {{- if .Values.controlPlane.apiServer.resources }}
-    resources: {{- include "cozy-lib.resources.sanitize" (list .Values.controlPlane.apiServer.resources $) | nindent 6 }}
-    {{- else if ne .Values.controlPlane.apiServer.resourcesPreset "none" }}
-    resources: {{- include "cozy-lib.resources.preset" (list .Values.controlPlane.apiServer.resourcesPreset $) | nindent 6 }}
-    {{- end }}
+    resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.controlPlane.apiServer.resourcesPreset .Values.controlPlane.apiServer.resources $) | nindent 6 }}
   controllerManager:
-    {{- if .Values.controlPlane.controllerManager.resources }}
-    resources: {{- include "cozy-lib.resources.sanitize" (list .Values.controlPlane.controllerManager.resources $) | nindent 6 }}
-    {{- else if ne .Values.controlPlane.controllerManager.resourcesPreset "none" }}
-    resources: {{- include "cozy-lib.resources.preset" (list .Values.controlPlane.controllerManager.resourcesPreset $) | nindent 6 }}
-    {{- end }}
+    resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.controlPlane.controllerManager.resourcesPreset .Values.controlPlane.controllerManager.resources $) | nindent 6 }}
   scheduler:
-    {{- if .Values.controlPlane.scheduler.resources }}
-    resources: {{- include "cozy-lib.resources.sanitize" (list .Values.controlPlane.scheduler.resources $) | nindent 6 }}
-    {{- else if ne .Values.controlPlane.scheduler.resourcesPreset "none" }}
-    resources: {{- include "cozy-lib.resources.preset" (list .Values.controlPlane.scheduler.resourcesPreset $) | nindent 6 }}
-    {{- end }}
+    resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.controlPlane.scheduler.resourcesPreset .Values.controlPlane.scheduler.resources $) | nindent 6 }}
   dataStoreName: "{{ $etcd }}"
   addons:
     coreDNS:
@@ -145,11 +133,7 @@ spec:
     konnectivity:
       server:
         port: 8132
-        {{- if .Values.controlPlane.konnectivity.server.resources }}
-        resources: {{- include "cozy-lib.resources.sanitize" (list .Values.controlPlane.konnectivity.server.resources $) | nindent 10 }}
-        {{- else if ne .Values.controlPlane.konnectivity.server.resourcesPreset "none" }}
-        resources: {{- include "cozy-lib.resources.preset" (list .Values.controlPlane.konnectivity.server.resourcesPreset $) | nindent 10 }}
-        {{- end }}
+        resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.controlPlane.konnectivity.server.resourcesPreset .Values.controlPlane.konnectivity.server.resources $) | nindent 10 }}
   kubelet:
     cgroupfs: systemd
     preferredAddressTypes:

packages/apps/kubernetes/templates/copy-patch-containerd.yaml

@@ -1,4 +1,3 @@
-{{- if not .Values.useCustomSecretForPatchContainerd }}
 {{- $sourceSecret := lookup "v1" "Secret" "cozy-system" "patch-containerd" }}
 {{- if $sourceSecret }}
 apiVersion: v1
@@ -12,4 +11,3 @@ data:
   {{ printf "%s: %s" $key ($value | quote) | indent 2 }}
 {{- end }}
 {{- end }}
-{{- end }}

packages/apps/kubernetes/templates/csi/infra-cluster-service-account.yaml

@@ -13,11 +13,17 @@ rules:
   resources: ["datavolumes"]
   verbs: ["get", "create", "delete"]
 - apiGroups: ["kubevirt.io"]
-  resources: ["virtualmachineinstances"]
+  resources: ["virtualmachineinstances", "virtualmachines"]
   verbs: ["list", "get"]
 - apiGroups: ["subresources.kubevirt.io"]
-  resources: ["virtualmachineinstances/addvolume", "virtualmachineinstances/removevolume"]
+  resources: ["virtualmachines/addvolume", "virtualmachines/removevolume"]
   verbs: ["update"]
+- apiGroups: ["snapshot.storage.k8s.io"]
+  resources: ["volumesnapshots"]
+  verbs: ["get", "create", "delete"]
+- apiGroups: [""]
+  resources: ["persistentvolumeclaims"]
+  verbs: ["get", "patch"]
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding

packages/apps/kubernetes/templates/helmreleases/cert-manager-crds.yaml

@@ -1,3 +1,4 @@
+{{- if .Values.addons.certManager.enabled }}
 apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
@@ -54,3 +55,4 @@ stringData:
   values: |
     {{- toYaml .Values.addons.certManager.valuesOverride | nindent 4 }}
 {{- end }}
+{{- end }}

packages/apps/kubernetes/templates/helmreleases/delete.yaml

@@ -40,6 +40,7 @@ spec:
                 {{ .Release.Name }}-fluxcd-operator
                 {{ .Release.Name }}-fluxcd
                 {{ .Release.Name }}-gpu-operator
+                {{ .Release.Name }}-velero
                 -p '{"spec": {"suspend": true}}'
                 --type=merge --field-manager=flux-client-side-apply || true
 ---
@@ -79,6 +80,8 @@ rules:
       - {{ .Release.Name }}-fluxcd-operator
       - {{ .Release.Name }}-fluxcd
       - {{ .Release.Name }}-gpu-operator
+      - {{ .Release.Name }}-velero
+
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding

packages/apps/kubernetes/templates/helmreleases/ingress-nginx.yaml

@@ -3,9 +3,11 @@ ingress-nginx:
   fullnameOverride: ingress-nginx
   controller:
     kind: DaemonSet
+    {{- if eq .Values.addons.ingressNginx.exposeMethod "Proxied" }}
     hostNetwork: true
     service:
       enabled: false
+    {{- end }}
     {{- if not .Values.addons.certManager.enabled }}
     admissionWebhooks:
       certManager:

packages/apps/kubernetes/templates/helmreleases/velero.yaml

@@ -0,0 +1,46 @@
+{{- if .Values.addons.velero.enabled }}
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: {{ .Release.Name }}-velero
+  labels:
+    cozystack.io/repository: system
+    cozystack.io/target-cluster-name: {{ .Release.Name }}
+spec:
+  interval: 5m
+  releaseName: velero
+  chart:
+    spec:
+      chart: cozy-velero
+      reconcileStrategy: Revision
+      sourceRef:
+        kind: HelmRepository
+        name: cozystack-system
+        namespace: cozy-system
+      version: '>= 0.0.0-0'
+  kubeConfig:
+    secretRef:
+      name: {{ .Release.Name }}-admin-kubeconfig
+      key: super-admin.svc
+  targetNamespace: cozy-velero
+  storageNamespace: cozy-velero
+  install:
+    createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
+  {{- with .Values.addons.velero.valuesOverride }}
+  values:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+
+  dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
+  - name: {{ .Release.Name }}
+    namespace: {{ .Release.Namespace }}
+  {{- end }}
+  - name: {{ .Release.Name }}-cilium
+    namespace: {{ .Release.Namespace }}
+{{- end }}

packages/apps/kubernetes/templates/ingress.yaml

@@ -1,6 +1,6 @@
 {{- $myNS := lookup "v1" "Namespace" "" .Release.Namespace }}
 {{- $ingress := index $myNS.metadata.annotations "namespace.cozystack.io/ingress" }}
-{{- if .Values.addons.ingressNginx.hosts }}
+{{- if and (eq .Values.addons.ingressNginx.exposeMethod "Proxied") .Values.addons.ingressNginx.hosts }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress

packages/apps/kubernetes/values.schema.json

@@ -20,12 +20,12 @@
           "properties": {
             "resources": {
               "type": "object",
-              "description": "Explicit CPU/memory resource requests and limits for the API server.",
+              "description": "Explicit CPU and memory configuration for the API Server. When left empty, the preset defined in `resourcesPreset` is applied.",
               "default": {}
             },
             "resourcesPreset": {
               "type": "string",
-              "description": "Use a common resources preset when `resources` is not set explicitly.",
+              "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
               "default": "medium",
               "enum": [
                 "none",
@@ -45,12 +45,12 @@
           "properties": {
             "resources": {
               "type": "object",
-              "description": "Explicit CPU/memory resource requests and limits for the controller manager.",
+              "description": "Explicit CPU and memory configuration for the Controller Manager. When left empty, the preset defined in `resourcesPreset` is applied.",
               "default": {}
             },
             "resourcesPreset": {
               "type": "string",
-              "description": "Use a common resources preset when `resources` is not set explicitly.",
+              "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
               "default": "micro",
               "enum": [
                 "none",
@@ -70,12 +70,12 @@
           "properties": {
             "resources": {
               "type": "object",
-              "description": "Explicit CPU/memory resource requests and limits for the scheduler.",
+              "description": "Explicit CPU and memory configuration for the Scheduler. When left empty, the preset defined in `resourcesPreset` is applied.",
               "default": {}
             },
             "resourcesPreset": {
               "type": "string",
-              "description": "Use a common resources preset when `resources` is not set explicitly.",
+              "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
               "default": "micro",
               "enum": [
                 "none",
@@ -98,12 +98,12 @@
               "properties": {
                 "resources": {
                   "type": "object",
-                  "description": "Explicit CPU/memory resource requests and limits for the Konnectivity.",
+                  "description": "Explicit CPU and memory configuration for Konnectivity. When left empty, the preset defined in `resourcesPreset` is applied.",
                   "default": {}
                 },
                 "resourcesPreset": {
                   "type": "string",
-                  "description": "Use a common resources preset when `resources` is not set explicitly.",
+                  "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
                   "default": "micro",
                   "enum": [
                     "none",
@@ -127,11 +127,6 @@
       "description": "StorageClass used to store user data.",
       "default": "replicated"
     },
-    "useCustomSecretForPatchContainerd": {
-      "type": "boolean",
-      "description": "if true, for patch containerd will be used secret: {{ .Release.Name }}-patch-containerd",
-      "default": false
-    },
     "addons": {
       "type": "object",
       "properties": {
@@ -183,9 +178,18 @@
               "description": "Custom values to override",
               "default": {}
             },
+            "exposeMethod": {
+              "type": "string",
+              "description": "Method to expose the Ingress-NGINX controller. (allowed values: Proxied, LoadBalancer)",
+              "default": "Proxied",
+              "enum": [
+                "Proxied",
+                "LoadBalancer"
+              ]
+            },
             "hosts": {
               "type": "array",
-              "description": "List of domain names that the parent cluster should route to this tenant cluster.",
+              "description": "List of domain names that the parent cluster should route to this tenant cluster. Taken into account only when `exposeMethod` is set to `Proxied`.",
               "default": [],
               "items": {}
             }
@@ -245,6 +249,21 @@
               "default": {}
             }
           }
+        },
+        "velero": {
+          "type": "object",
+          "properties": {
+            "enabled": {
+              "type": "boolean",
+              "description": "Enable velero for backup and restore k8s cluster.",
+              "default": false
+            },
+            "valuesOverride": {
+              "type": "object",
+              "description": "Custom values to override",
+              "default": {}
+            }
+          }
         }
       }
     }

packages/apps/kubernetes/values.yaml

@@ -3,11 +3,9 @@
 ## @param host Hostname used to access the Kubernetes cluster externally. Defaults to `<cluster-name>.<tenant-host>` when empty.
 ## @param controlPlane.replicas Number of replicas for Kubernetes control-plane components.
 ## @param storageClass StorageClass used to store user data.
-## @param useCustomSecretForPatchContainerd if true, for patch containerd will be used secret: {{ .Release.Name }}-patch-containerd
 ##
 host: ""
 storageClass: replicated
-useCustomSecretForPatchContainerd: false
 
 ## @param nodeGroups [object] nodeGroups configuration
 ##
@@ -63,12 +61,14 @@ addons:
     ## @param addons.ingressNginx.valuesOverride Custom values to override
     ##
     enabled: false
-    ## @param addons.ingressNginx.hosts List of domain names that the parent cluster should route to this tenant cluster.
+    ## @param addons.ingressNginx.exposeMethod Method to expose the Ingress-NGINX controller. (allowed values: Proxied, LoadBalancer)
+    ## @param addons.ingressNginx.hosts List of domain names that the parent cluster should route to this tenant cluster. Taken into account only when `exposeMethod` is set to `Proxied`.
     ## e.g:
     ## hosts:
     ## - example.org
     ## - foo.example.net
     ##
+    exposeMethod: Proxied
     hosts: []
     valuesOverride: {}
 
@@ -105,6 +105,15 @@ addons:
     ##
     valuesOverride: {}
 
+  ## Velero
+  ##
+  velero:
+    ## @param addons.velero.enabled Enable velero for backup and restore k8s cluster.
+    ## @param addons.velero.valuesOverride Custom values to override
+    ##
+    enabled: false
+    valuesOverride: {}
+
 ## @section Kubernetes Control Plane Configuration
 ##
 
@@ -112,35 +121,31 @@ controlPlane:
   replicas: 2
 
   apiServer:
-    ## @param controlPlane.apiServer.resources Explicit CPU/memory resource requests and limits for the API server.
-    ## @param controlPlane.apiServer.resourcesPreset Use a common resources preset when `resources` is not set explicitly.
+    ## @param controlPlane.apiServer.resources Explicit CPU and memory configuration for the API Server. When left empty, the preset defined in `resourcesPreset` is applied.
+    ## @param controlPlane.apiServer.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
     ## e.g:
     ## resources:
-    ##   limits:
-    ##     cpu: 4000m
-    ##     memory: 4Gi
-    ##   requests:
-    ##     cpu: 100m
-    ##     memory: 512Mi
+    ##   cpu: 4000m
+    ##   memory: 4Gi
     ##
     resourcesPreset: "medium"
     resources: {}
 
   controllerManager:
-    ## @param controlPlane.controllerManager.resources Explicit CPU/memory resource requests and limits for the controller manager.
-    ## @param controlPlane.controllerManager.resourcesPreset Use a common resources preset when `resources` is not set explicitly.
+    ## @param controlPlane.controllerManager.resources Explicit CPU and memory configuration for the Controller Manager. When left empty, the preset defined in `resourcesPreset` is applied.
+    ## @param controlPlane.controllerManager.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
     resourcesPreset: "micro"
     resources: {}
 
   scheduler:
-    ## @param controlPlane.scheduler.resources Explicit CPU/memory resource requests and limits for the scheduler.
-    ## @param controlPlane.scheduler.resourcesPreset Use a common resources preset when `resources` is not set explicitly.
+    ## @param controlPlane.scheduler.resources Explicit CPU and memory configuration for the Scheduler. When left empty, the preset defined in `resourcesPreset` is applied.
+    ## @param controlPlane.scheduler.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
     resourcesPreset: "micro"
     resources: {}
 
   konnectivity:
     server:
-      ## @param controlPlane.konnectivity.server.resources Explicit CPU/memory resource requests and limits for the Konnectivity.
-      ## @param controlPlane.konnectivity.server.resourcesPreset Use a common resources preset when `resources` is not set explicitly.
+      ## @param controlPlane.konnectivity.server.resources Explicit CPU and memory configuration for Konnectivity. When left empty, the preset defined in `resourcesPreset` is applied.
+      ## @param controlPlane.konnectivity.server.resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
       resourcesPreset: "micro"
       resources: {}

packages/apps/mysql/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.8.0
+version: 0.9.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/mysql/Makefile

@@ -5,6 +5,7 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json
 
 image:
 	docker buildx build images/mariadb-backup \

packages/apps/mysql/README.md

@@ -1,6 +1,7 @@
 ## Managed MariaDB Service
 
-The Managed MariaDB Service offers a powerful and widely used relational database solution. This service allows you to create and manage a replicated MariaDB cluster seamlessly.
+The Managed MariaDB Service offers a powerful and widely used relational database solution.
+This service allows you to create and manage a replicated MariaDB cluster seamlessly.
 
 ## Deployment Details
 
@@ -46,7 +47,7 @@ restic -r s3:s3.example.org/mariadb-backups/database_name restore latest --targe
 ```
 
 more details:
-- https://itnext.io/restic-effective-backup-from-stdin-4bc1e8f083c1
+- https://blog.aenix.io/restic-effective-backup-from-stdin-4bc1e8f083c1
 
 ### Known issues
 
@@ -83,16 +84,66 @@ more details:
 
 ### Backup parameters
 
-| Name                     | Description                                                                                                                                                                                                       | Value                                                  |
-| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
-| `backup.enabled`         | Enable pereiodic backups                                                                                                                                                                                          | `false`                                                |
-| `backup.s3Region`        | The AWS S3 region where backups are stored                                                                                                                                                                        | `us-east-1`                                            |
-| `backup.s3Bucket`        | The S3 bucket used for storing backups                                                                                                                                                                            | `s3.example.org/postgres-backups`                      |
-| `backup.schedule`        | Cron schedule for automated backups                                                                                                                                                                               | `0 2 * * *`                                            |
-| `backup.cleanupStrategy` | The strategy for cleaning up old backups                                                                                                                                                                          | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
-| `backup.s3AccessKey`     | The access key for S3, used for authentication                                                                                                                                                                    | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
-| `backup.s3SecretKey`     | The secret key for S3, used for authentication                                                                                                                                                                    | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
-| `backup.resticPassword`  | The password for Restic backup encryption                                                                                                                                                                         | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
-| `resources`              | Resources                                                                                                                                                                                                         | `{}`                                                   |
-| `resourcesPreset`        | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano`                                                 |
+| Name                     | Description                                                                                                                          | Value                                                  |
+| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------ |
+| `backup.enabled`         | Enable periodic backups                                                                                                              | `false`                                                |
+| `backup.s3Region`        | The AWS S3 region where backups are stored                                                                                           | `us-east-1`                                            |
+| `backup.s3Bucket`        | The S3 bucket used for storing backups                                                                                               | `s3.example.org/postgres-backups`                      |
+| `backup.schedule`        | Cron schedule for automated backups                                                                                                  | `0 2 * * *`                                            |
+| `backup.cleanupStrategy` | The strategy for cleaning up old backups                                                                                             | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
+| `backup.s3AccessKey`     | The access key for S3, used for authentication                                                                                       | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
+| `backup.s3SecretKey`     | The secret key for S3, used for authentication                                                                                       | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
+| `backup.resticPassword`  | The password for Restic backup encryption                                                                                            | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
+| `resources`              | Explicit CPU and memory configuration for each MariaDB replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`                                                   |
+| `resourcesPreset`        | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.    | `nano`                                                 |
 
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |
+
+### users
+
+```yaml
+users:
+  user1:
+    maxUserConnections: 1000
+    password: hackme
+  user2:
+    maxUserConnections: 1000
+    password: hackme
+```
+
+
+### databases
+
+```yaml
+databases:
+  myapp1:
+    roles:
+      admin:
+      - user1
+      readonly:
+      - user2
+```

packages/apps/mysql/images/mariadb-backup.tag

@@ -1 +1 @@
-ghcr.io/cozystack/cozystack/mariadb-backup:0.8.0@sha256:cfd1c37d8ad24e10681d82d6e6ce8a641b4602c1b0ffa8516ae15b4958bb12d4
+ghcr.io/cozystack/cozystack/mariadb-backup:0.9.0@sha256:cfd1c37d8ad24e10681d82d6e6ce8a641b4602c1b0ffa8516ae15b4958bb12d4

packages/apps/mysql/templates/mariadb.yaml

@@ -61,7 +61,9 @@ spec:
     metadata:
       labels:
         app.kubernetes.io/instance: {{ $.Release.Name }}
-
+    {{- if and .Values.external (eq (int .Values.replicas) 1) }}
+    type: LoadBalancer
+    {{- end }}
   storage:
     size: {{ .Values.size }}
     resizeInUseVolumes: true
@@ -70,7 +72,7 @@ spec:
     storageClassName: {{ . }}
     {{- end }}
 
-  {{- if .Values.external }}
+  {{- if and .Values.external (gt (int .Values.replicas) 1) }}
   primaryService:
     type: LoadBalancer
   {{- end }}
@@ -78,8 +80,4 @@ spec:
   #secondaryService:
   #  type: LoadBalancer
 
-  {{- if .Values.resources }}
-  resources: {{- include "cozy-lib.resources.sanitize" (list .Values.resources $) | nindent 4 }}
-  {{- else if ne .Values.resourcesPreset "none" }}
-  resources: {{- include "cozy-lib.resources.preset" (list .Values.resourcesPreset $) | nindent 4 }}
-  {{- end }}
+  resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 4 }}

packages/apps/mysql/values.schema.json

@@ -27,7 +27,7 @@
             "properties": {
                 "enabled": {
                     "type": "boolean",
-                    "description": "Enable pereiodic backups",
+                    "description": "Enable periodic backups",
                     "default": false
                 },
                 "s3Region": {
@@ -69,13 +69,23 @@
         },
         "resources": {
             "type": "object",
-            "description": "Resources",
+            "description": "Explicit CPU and memory configuration for each MariaDB replica. When left empty, the preset defined in `resourcesPreset` is applied.",
             "default": {}
         },
         "resourcesPreset": {
             "type": "string",
-            "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-            "default": "nano"
+            "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+            "default": "nano",
+            "enum": [
+                "none",
+                "nano",
+                "micro",
+                "small",
+                "medium",
+                "large",
+                "xlarge",
+                "2xlarge"
+            ]
         }
     }
-}
+}

packages/apps/mysql/values.yaml

@@ -37,7 +37,7 @@ databases: {}
 
 ## @section Backup parameters
 
-## @param backup.enabled Enable pereiodic backups
+## @param backup.enabled Enable periodic backups
 ## @param backup.s3Region The AWS S3 region where backups are stored
 ## @param backup.s3Bucket The S3 bucket used for storing backups
 ## @param backup.schedule Cron schedule for automated backups
@@ -55,15 +55,11 @@ backup:
   s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
   resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
 
-## @param resources Resources
+## @param resources Explicit CPU and memory configuration for each MariaDB replica. When left empty, the preset defined in `resourcesPreset` is applied.
 resources: {}
  # resources:
- #   limits:
- #     cpu: 4000m
- #     memory: 4Gi
- #   requests:
- #     cpu: 100m
- #     memory: 512Mi
+ #   cpu: 4000m
+ #   memory: 4Gi
  
-## @param resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
+## @param resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
 resourcesPreset: "nano"

packages/apps/nats/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.7.0
+version: 0.8.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/nats/Makefile

@@ -2,3 +2,4 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json

packages/apps/nats/README.md

@@ -1,18 +1,48 @@
 # Managed NATS Service
 
+NATS is an open-source, simple, secure, and high performance messaging system.
+It provides a data layer for cloud native applications, IoT messaging, and microservices architectures.
+
 ## Parameters
 
 ### Common parameters
 
-| Name                | Description                                                                                                                                                                                                       | Value   |
-| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
-| `external`          | Enable external access from outside the cluster                                                                                                                                                                   | `false` |
-| `replicas`          | Persistent Volume size for NATS                                                                                                                                                                                   | `2`     |
-| `storageClass`      | StorageClass used to store the data                                                                                                                                                                               | `""`    |
-| `users`             | Users configuration                                                                                                                                                                                               | `{}`    |
-| `jetstream.size`    | Jetstream persistent storage size                                                                                                                                                                                 | `10Gi`  |
-| `jetstream.enabled` | Enable or disable Jetstream                                                                                                                                                                                       | `true`  |
-| `config.merge`      | Additional configuration to merge into NATS config                                                                                                                                                                | `{}`    |
-| `config.resolver`   | Additional configuration to merge into NATS config                                                                                                                                                                | `{}`    |
-| `resources`         | Resources                                                                                                                                                                                                         | `{}`    |
-| `resourcesPreset`   | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano`  |
+| Name                | Description                                                                                                                       | Value   |
+| ------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `external`          | Enable external access from outside the cluster                                                                                   | `false` |
+| `replicas`          | Persistent Volume size for NATS                                                                                                   | `2`     |
+| `storageClass`      | StorageClass used to store the data                                                                                               | `""`    |
+| `users`             | Users configuration                                                                                                               | `{}`    |
+| `jetstream.size`    | Jetstream persistent storage size                                                                                                 | `10Gi`  |
+| `jetstream.enabled` | Enable or disable Jetstream                                                                                                       | `true`  |
+| `config.merge`      | Additional configuration to merge into NATS config                                                                                | `{}`    |
+| `config.resolver`   | Additional configuration to merge into NATS config                                                                                | `{}`    |
+| `resources`         | Explicit CPU and memory configuration for each NATS replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`    |
+| `resourcesPreset`   | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge. | `nano`  |
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |
+

packages/apps/nats/templates/nats.yaml

@@ -46,11 +46,7 @@ spec:
             containers:
               - name: nats
                 image: nats:2.10.17-alpine
-                {{- if .Values.resources }}
-                resources: {{- include "cozy-lib.resources.sanitize" (list .Values.resources $) | nindent 22 }}
-                {{- else if ne .Values.resourcesPreset "none" }}
-                resources: {{- include "cozy-lib.resources.preset" (list .Values.resourcesPreset $) | nindent 22 }}
-                {{- end }}
+                resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 22 }}
       fullnameOverride: {{ .Release.Name }}
       config:
         cluster:

packages/apps/nats/values.schema.json

@@ -49,13 +49,23 @@
         },
         "resources": {
             "type": "object",
-            "description": "Resources",
+            "description": "Explicit CPU and memory configuration for each NATS replica. When left empty, the preset defined in `resourcesPreset` is applied.",
             "default": {}
         },
         "resourcesPreset": {
             "type": "string",
-            "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-            "default": "nano"
+            "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+            "default": "nano",
+            "enum": [
+                "none",
+                "nano",
+                "micro",
+                "small",
+                "medium",
+                "large",
+                "xlarge",
+                "2xlarge"
+            ]
         }
     }
-}
+}

packages/apps/nats/values.yaml

@@ -62,15 +62,11 @@ config:
   ## Example see: https://github.com/nats-io/k8s/blob/main/helm/charts/nats/values.yaml#L247
   resolver: {}
 
-## @param resources Resources
+## @param resources Explicit CPU and memory configuration for each NATS replica. When left empty, the preset defined in `resourcesPreset` is applied.
 resources: {}
  # resources:
- #   limits:
- #     cpu: 4000m
- #     memory: 4Gi
- #   requests:
- #     cpu: 100m
- #     memory: 512Mi
+ #   cpu: 4000m
+ #   memory: 4Gi
  
-## @param resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
+## @param resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
 resourcesPreset: "nano"

packages/apps/postgres/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.14.0
+version: 0.17.1
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/postgres/Makefile

@@ -1,24 +1,5 @@
-POSTGRES_BACKUP_TAG = $(shell awk '$$1 == "version:" {print $$2}' Chart.yaml)
-
-include ../../../scripts/common-envs.mk
 include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
-
-image:
-	docker buildx build images/postgres-backup \
-		--provenance false \
-		--builder=$(BUILDER) \
-		--platform=$(PLATFORM) \
-		--tag $(REGISTRY)/postgres-backup:$(call settag,$(POSTGRES_BACKUP_TAG)) \
-		--cache-from type=registry,ref=$(REGISTRY)/postgres-backup:latest \
-		--cache-to type=inline \
-		--metadata-file images/postgres-backup.json \
-		--push=$(PUSH) \
-		--label "org.opencontainers.image.source=https://github.com/cozystack/cozystack" \
-		--load=$(LOAD)
-	echo "$(REGISTRY)/postgres-backup:$(call settag,$(POSTGRES_BACKUP_TAG))@$$(yq e '."containerimage.digest"' images/postgres-backup.json -o json -r)" \
-		> images/postgres-backup.tag
-	cp images/postgres-backup.tag ../ferretdb/images/
-	rm -f images/postgres-backup.json
+	yq -i -o json --indent 4 '.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json

packages/apps/postgres/README.md

@@ -1,6 +1,8 @@
 # Managed PostgreSQL Service
 
-PostgreSQL is currently the leading choice among relational databases, known for its robust features and performance. Our Managed PostgreSQL Service takes advantage of platform-side implementation to provide a self-healing replicated cluster. This cluster is efficiently managed using the highly acclaimed CloudNativePG operator, which has gained popularity within the community.
+PostgreSQL is currently the leading choice among relational databases, known for its robust features and performance.
+The Managed PostgreSQL Service takes advantage of platform-side implementation to provide a self-healing replicated cluster.
+This cluster is efficiently managed using the highly acclaimed CloudNativePG operator, which has gained popularity within the community.
 
 ## Deployment Details
 
@@ -9,32 +11,57 @@ This managed service is controlled by the CloudNativePG operator, ensuring effic
 - Docs: <https://cloudnative-pg.io/docs/>
 - Github: <https://github.com/cloudnative-pg/cloudnative-pg>
 
-## HowTos
+## Operations
 
-### How to switch master/slave replica
+### How to enable backups
+
+To back up a PostgreSQL application, an external S3-compatible storage is required.
+
+To start regular backups, update the application, setting `backup.enabled` to `true`, and fill in the path and credentials to an  `backup.*`:
+
+```yaml
+## @param backup.enabled Enable regular backups
+## @param backup.schedule Cron schedule for automated backups
+## @param backup.retentionPolicy Retention policy
+## @param backup.destinationPath Path to store the backup (i.e. s3://bucket/path/to/folder)
+## @param backup.endpointURL S3 Endpoint used to upload data to the cloud
+## @param backup.s3AccessKey Access key for S3, used for authentication
+## @param backup.s3SecretKey Secret key for S3, used for authentication
+backup:
+  enabled: false
+  retentionPolicy: 30d
+  destinationPath: s3://bucket/path/to/folder/
+  endpointURL: http://minio-gateway-service:9000
+  schedule: "0 2 * * * *"
+  s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
+  s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
+```
+
+### How to recover a backup
+
+CloudNativePG supports point-in-time-recovery.
+Recovering a backup is done by creating a new database instance and restoring the data in it.
+
+Create a new PostgreSQL application with a different name, but identical configuration.
+Set `bootstrap.enabled` to `true` and fill in the name of the database instance to recover from and the recovery time:
+
+```yaml
+## @param bootstrap.enabled Restore database cluster from a backup
+## @param bootstrap.recoveryTime Timestamp (PITR) up to which recovery will proceed, expressed in RFC 3339 format. If left empty, will restore latest
+## @param bootstrap.oldName Name of database cluster before deleting
+##
+bootstrap:
+  enabled: false
+  recoveryTime: ""  # leave empty for latest or exact timestamp; example: 2020-11-26 15:22:00.00000+00
+  oldName: "<previous-postgres-instance>"
+```
+
+### How to switch primary/secondary replica
 
 See:
 
 - <https://cloudnative-pg.io/documentation/1.15/rolling_update/#manual-updates-supervised>
 
-### How to restore backup
-
-find snapshot:
-
-```bash
-restic -r s3:s3.example.org/postgres-backups/database_name snapshots
-```
-
-restore:
-
-```bash
-restic -r s3:s3.example.org/postgres-backups/database_name restore latest --target /tmp/
-```
-
-more details:
-
-- <https://itnext.io/restic-effective-backup-from-stdin-4bc1e8f083c1>
-
 ## Parameters
 
 ### Common parameters
@@ -58,15 +85,82 @@ more details:
 
 ### Backup parameters
 
-| Name                     | Description                                                                                                                                                                                                       | Value                                                  |
-| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
-| `backup.enabled`         | Enable pereiodic backups                                                                                                                                                                                          | `false`                                                |
-| `backup.s3Region`        | The AWS S3 region where backups are stored                                                                                                                                                                        | `us-east-1`                                            |
-| `backup.s3Bucket`        | The S3 bucket used for storing backups                                                                                                                                                                            | `s3.example.org/postgres-backups`                      |
-| `backup.schedule`        | Cron schedule for automated backups                                                                                                                                                                               | `0 2 * * *`                                            |
-| `backup.cleanupStrategy` | The strategy for cleaning up old backups                                                                                                                                                                          | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
-| `backup.s3AccessKey`     | The access key for S3, used for authentication                                                                                                                                                                    | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
-| `backup.s3SecretKey`     | The secret key for S3, used for authentication                                                                                                                                                                    | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
-| `backup.resticPassword`  | The password for Restic backup encryption                                                                                                                                                                         | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
-| `resources`              | Resources                                                                                                                                                                                                         | `{}`                                                   |
-| `resourcesPreset`        | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano`                                                 |
+| Name                     | Description                                                | Value                               |
+| ------------------------ | ---------------------------------------------------------- | ----------------------------------- |
+| `backup.enabled`         | Enable regular backups                                     | `false`                             |
+| `backup.schedule`        | Cron schedule for automated backups                        | `0 2 * * * *`                       |
+| `backup.retentionPolicy` | Retention policy                                           | `30d`                               |
+| `backup.destinationPath` | Path to store the backup (i.e. s3://bucket/path/to/folder) | `s3://bucket/path/to/folder/`       |
+| `backup.endpointURL`     | S3 Endpoint used to upload data to the cloud               | `http://minio-gateway-service:9000` |
+| `backup.s3AccessKey`     | Access key for S3, used for authentication                 | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`  |
+| `backup.s3SecretKey`     | Secret key for S3, used for authentication                 | `ju3eum4dekeich9ahM1te8waeGai0oog`  |
+
+### Bootstrap parameters
+
+| Name                     | Description                                                                                                                             | Value   |
+| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `bootstrap.enabled`      | Restore database cluster from a backup                                                                                                  | `false` |
+| `bootstrap.recoveryTime` | Timestamp (PITR) up to which recovery will proceed, expressed in RFC 3339 format. If left empty, will restore latest                    | `""`    |
+| `bootstrap.oldName`      | Name of database cluster before deleting                                                                                                | `""`    |
+| `resources`              | Explicit CPU and memory configuration for each PostgreSQL replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`    |
+| `resourcesPreset`        | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.       | `micro` |
+
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `250m` | `128Mi` |
+| `micro`     | `500m` | `256Mi` |
+| `small`     | `1`    | `512Mi` |
+| `medium`    | `1`    | `1Gi`   |
+| `large`     | `2`    | `2Gi`   |
+| `xlarge`    | `4`    | `4Gi`   |
+| `2xlarge`   | `8`    | `8Gi`   |
+
+### users
+
+```yaml
+users:
+  user1:
+    password: strongpassword
+  user2:
+    password: hackme
+  airflow:
+    password: qwerty123
+  debezium:
+    replication: true
+```
+
+### databases
+
+```yaml
+databases:          
+  myapp:            
+    roles:          
+      admin:        
+      - user1       
+      - debezium    
+      readonly:     
+      - user2       
+  airflow:          
+    roles:          
+      admin:        
+      - airflow     
+    extensions:     
+    - hstore        
+```

packages/apps/postgres/images/postgres-backup.tag

@@ -1 +0,0 @@
-ghcr.io/cozystack/cozystack/postgres-backup:0.14.0@sha256:10179ed56457460d95cd5708db2a00130901255fa30c4dd76c65d2ef5622b61f

packages/apps/postgres/images/postgres-backup/Dockerfile

@@ -1,2 +0,0 @@
-FROM alpine:3.20
-RUN apk add --no-cache postgresql16-client uuidgen restic

packages/apps/postgres/templates/backup-cronjob.yaml

@@ -1,99 +0,0 @@
-{{- if .Values.backup.enabled }}
-{{ $image := .Files.Get "images/backup.json" | fromJson }}
-
-apiVersion: batch/v1
-kind: CronJob
-metadata:
-  name: {{ .Release.Name }}-backup
-spec:
-  schedule: "{{ .Values.backup.schedule }}"
-  concurrencyPolicy: Forbid
-  successfulJobsHistoryLimit: 3
-  failedJobsHistoryLimit: 3
-  jobTemplate:
-    spec:
-      backoffLimit: 2
-      template:
-        metadata:
-          annotations:
-            checksum/config: {{ include (print $.Template.BasePath "/backup-script.yaml") . | sha256sum }}
-            checksum/secret: {{ include (print $.Template.BasePath "/backup-secret.yaml") . | sha256sum }}
-        spec:
-          imagePullSecrets:
-          - name: {{ .Release.Name }}-regsecret
-          restartPolicy: OnFailure
-          containers:
-          - name: pgdump
-            image: "{{ $.Files.Get "images/postgres-backup.tag" | trim }}"
-            command:
-            - /bin/sh
-            - /scripts/backup.sh
-            env:
-            - name: REPO_PREFIX
-              value: {{ required "s3Bucket is not specified!" .Values.backup.s3Bucket | quote }}
-            - name: CLEANUP_STRATEGY
-              value: {{ required "cleanupStrategy is not specified!" .Values.backup.cleanupStrategy | quote }}
-            - name: PGUSER
-              valueFrom:
-                secretKeyRef:
-                  name: {{ .Release.Name }}-superuser
-                  key: username
-            - name: PGPASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: {{ .Release.Name }}-superuser
-                  key: password
-            - name: PGHOST
-              value: {{ .Release.Name }}-rw
-            - name: PGPORT
-              value: "5432"
-            - name: PGDATABASE
-              value: postgres
-            - name: AWS_ACCESS_KEY_ID
-              valueFrom:
-                secretKeyRef:
-                  name: {{ .Release.Name }}-backup
-                  key: s3AccessKey
-            - name: AWS_SECRET_ACCESS_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: {{ .Release.Name }}-backup
-                  key: s3SecretKey
-            - name: AWS_DEFAULT_REGION
-              value: {{ .Values.backup.s3Region }}
-            - name: RESTIC_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: {{ .Release.Name }}-backup
-                  key: resticPassword
-            volumeMounts:
-            - mountPath: /scripts
-              name: scripts
-            - mountPath: /tmp
-              name: tmp
-            - mountPath: /.cache
-              name: cache
-            securityContext:
-              allowPrivilegeEscalation: false
-              capabilities:
-                drop:
-                - ALL
-              privileged: false
-              readOnlyRootFilesystem: true
-              runAsNonRoot: true
-            {{- include "postgresjobs.resources" . | nindent 12 }}
-          volumes:
-          - name: scripts
-            secret:
-              secretName: {{ .Release.Name }}-backup-script
-          - name: tmp
-            emptyDir: {}
-          - name: cache
-            emptyDir: {}
-          securityContext:
-            runAsNonRoot: true
-            runAsUser: 9000
-            runAsGroup: 9000
-            seccompProfile:
-              type: RuntimeDefault
-{{- end }}

packages/apps/postgres/templates/backup-script.yaml

@@ -1,50 +0,0 @@
-{{- if .Values.backup.enabled }}
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: {{ .Release.Name }}-backup-script
-stringData:
-  backup.sh: |
-    #!/bin/sh
-    set -e
-    set -o pipefail
-
-    JOB_ID="job-$(uuidgen|cut -f1 -d-)"
-    DB_LIST=$(psql -Atq -c 'SELECT datname FROM pg_catalog.pg_database;' | grep -v '^\(postgres\|app\|template.*\)$')
-    echo DB_LIST=$(echo "$DB_LIST" | shuf) # shuffle list
-    echo "Job ID: $JOB_ID"
-    echo "Target repo: $REPO_PREFIX"
-    echo "Cleanup strategy: $CLEANUP_STRATEGY"
-    echo "Start backup for:"
-    echo "$DB_LIST"
-    echo
-    echo "Backup started at `date +%Y-%m-%d\ %H:%M:%S`"
-    for db in $DB_LIST; do
-      (
-        set -x
-        restic -r "s3:${REPO_PREFIX}/$db" cat config >/dev/null 2>&1 || \
-          restic -r "s3:${REPO_PREFIX}/$db" init --repository-version 2
-        restic -r "s3:${REPO_PREFIX}/$db" unlock --remove-all >/dev/null 2>&1 || true # no locks, k8s takes care of it
-        pg_dump -Z0 -Ft -d "$db" | \
-          restic -r "s3:${REPO_PREFIX}/$db" backup --tag "$JOB_ID" --stdin --stdin-filename dump.tar
-        restic -r "s3:${REPO_PREFIX}/$db" tag --tag "$JOB_ID" --set "completed"
-      )
-    done
-    echo "Backup finished at `date +%Y-%m-%d\ %H:%M:%S`"
-
-    echo
-    echo "Run cleanup:"
-    echo
-
-    echo "Cleanup started at `date +%Y-%m-%d\ %H:%M:%S`"
-    for db in $DB_LIST; do
-      (
-        set -x
-        restic forget -r "s3:${REPO_PREFIX}/$db" --group-by=tags --keep-tag "completed" # keep completed snapshots only
-        restic forget -r "s3:${REPO_PREFIX}/$db" --group-by=tags $CLEANUP_STRATEGY
-        restic prune -r "s3:${REPO_PREFIX}/$db"
-      )
-    done
-    echo "Cleanup finished at `date +%Y-%m-%d\ %H:%M:%S`"
-{{- end }}

packages/apps/postgres/templates/backup-secret.yaml

@@ -1,11 +1,10 @@
-{{- if .Values.backup.enabled }}
+{{- if or .Values.backup.enabled .Values.bootstrap.enabled }}
 ---
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ .Release.Name }}-backup
+  name: {{ .Release.Name }}-s3-creds
 stringData:
-  s3AccessKey: {{ required "s3AccessKey is not specified!" .Values.backup.s3AccessKey }}
-  s3SecretKey: {{ required "s3SecretKey is not specified!" .Values.backup.s3SecretKey }}
-  resticPassword: {{ required "resticPassword is not specified!" .Values.backup.resticPassword }}
+  AWS_ACCESS_KEY_ID: {{ required "s3AccessKey is not specified!" .Values.backup.s3AccessKey | quote }}
+  AWS_SECRET_ACCESS_KEY: {{ required "s3SecretKey is not specified!" .Values.backup.s3SecretKey | quote }}
 {{- end }}

packages/apps/postgres/templates/backup.yaml

@@ -0,0 +1,12 @@
+{{- if .Values.backup.enabled }}
+---
+apiVersion: postgresql.cnpg.io/v1
+kind: ScheduledBackup
+metadata:
+  name: {{ .Release.Name }}
+spec:
+  schedule: {{ .Values.backup.schedule | quote }}
+  backupOwnerReference: self
+  cluster:
+    name: {{ .Release.Name }}
+{{- end }}

packages/apps/postgres/templates/db.yaml

@@ -5,11 +5,44 @@ metadata:
   name: {{ .Release.Name }}
 spec:
   instances: {{ .Values.replicas }}
-  {{- if .Values.resources }}
-  resources: {{- include "cozy-lib.resources.sanitize" (list .Values.resources $) | nindent 4 }}
-  {{- else if ne .Values.resourcesPreset "none" }}
-  resources: {{- include "cozy-lib.resources.preset" (list .Values.resourcesPreset $) | nindent 4 }}
+  {{- if .Values.backup.enabled }}
+  backup:
+    barmanObjectStore:
+      destinationPath: {{ .Values.backup.destinationPath }}
+      endpointURL: {{ .Values.backup.endpointURL }}
+      s3Credentials:
+        accessKeyId:
+          name: {{ .Release.Name }}-s3-creds
+          key: AWS_ACCESS_KEY_ID
+        secretAccessKey:
+          name: {{ .Release.Name }}-s3-creds
+          key: AWS_SECRET_ACCESS_KEY
+    retentionPolicy: {{ .Values.backup.retentionPolicy }}
   {{- end }}
+  
+  {{- if .Values.bootstrap.enabled }}
+  bootstrap:
+    recovery:
+      source: {{ .Values.bootstrap.oldName }}
+      {{- if .Values.bootstrap.recoveryTime }}
+      recoveryTarget:
+        targetTime: {{ .Values.bootstrap.recoveryTime }}
+      {{- end }}
+  externalClusters:
+    - name: {{ .Values.bootstrap.oldName }}
+      barmanObjectStore:
+        destinationPath: {{ .Values.backup.destinationPath }}
+        endpointURL: {{ .Values.backup.endpointURL }}
+        s3Credentials:
+          accessKeyId:
+            name: {{ .Release.Name }}-s3-creds
+            key: AWS_ACCESS_KEY_ID
+          secretAccessKey:
+            name: {{ .Release.Name }}-s3-creds
+            key: AWS_SECRET_ACCESS_KEY
+  {{- end }}
+
+  resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 4 }}
 
   enableSuperuserAccess: true
   {{- $configMap := lookup "v1" "ConfigMap" "cozy-system" "cozystack-scheduling" }}

packages/apps/postgres/templates/init-script.yaml

@@ -38,15 +38,17 @@ stringData:
     until pg_isready ; do sleep 5; done
 
     echo "== create users"
-    {{- if .Values.users }}
+    {{- if and .Values.users (not (hasKey .Values.users "postgres")) }}
     psql -v ON_ERROR_STOP=1 <<\EOT
     {{- range $user, $u := .Values.users }}
-    SELECT 'CREATE ROLE {{ $user }} LOGIN INHERIT;'
+    SELECT 'CREATE ROLE "{{ $user }}" LOGIN INHERIT;'
     WHERE NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = '{{ $user }}')\gexec
-    ALTER ROLE {{ $user }} WITH PASSWORD '{{ index $passwords $user }}' LOGIN INHERIT {{ ternary "REPLICATION" "NOREPLICATION" (default false $u.replication) }};
-    COMMENT ON ROLE {{ $user }} IS 'user managed by helm';
+    ALTER ROLE "{{ $user }}" WITH PASSWORD '{{ index $passwords $user }}' LOGIN INHERIT {{ ternary "REPLICATION" "NOREPLICATION" (default false $u.replication) }};
+    COMMENT ON ROLE "{{ $user }}" IS 'user managed by helm';
     {{- end }}
     EOT
+    {{- else if and .Values.users (hasKey .Values.users "postgres") }}
+    {{- fail "`users.postgres` is forbidden by policy. Use a different username." }}
     {{- end }}
 
     echo "== delete users"
@@ -68,15 +70,15 @@ stringData:
     {{- if .Values.databases }}
     psql -v ON_ERROR_STOP=1 --echo-all <<\EOT
     {{- range $database, $d := .Values.databases }}
-    SELECT 'CREATE DATABASE {{ $database }}'
+    SELECT 'CREATE DATABASE "{{ $database }}"'
     WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = '{{ $database }}')\gexec
-    COMMENT ON DATABASE {{ $database }} IS 'database managed by helm';
-    SELECT 'CREATE ROLE {{ $database }}_admin NOINHERIT;'
+    COMMENT ON DATABASE "{{ $database }}" IS 'database managed by helm';
+    SELECT 'CREATE ROLE "{{ $database }}_admin" NOINHERIT;'
     WHERE NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = '{{ $database }}_admin')\gexec
-    COMMENT ON ROLE {{ $database }}_admin IS 'role managed by helm';
-    SELECT 'CREATE ROLE {{ $database }}_readonly NOINHERIT;'
+    COMMENT ON ROLE "{{ $database }}_admin" IS 'role managed by helm';
+    SELECT 'CREATE ROLE "{{ $database }}_readonly" NOINHERIT;'
     WHERE NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = '{{ $database }}_readonly')\gexec
-    COMMENT ON ROLE {{ $database }}_readonly IS 'role managed by helm';
+    COMMENT ON ROLE "{{ $database }}_readonly" IS 'role managed by helm';
     {{- end }}
     EOT
     {{- end }}
@@ -84,8 +86,8 @@ stringData:
     echo "== grant privileges on databases to roles"
     {{- range $database, $d := .Values.databases }}
     psql -v ON_ERROR_STOP=1 --echo-all -d "{{ $database }}" <<\EOT
-    ALTER DATABASE {{ $database }} OWNER TO {{ $database }}_admin;
-    GRANT CONNECT ON DATABASE {{ $database }} TO {{ $database }}_readonly;
+    ALTER DATABASE "{{ $database }}" OWNER TO "{{ $database }}_admin";
+    GRANT CONNECT ON DATABASE "{{ $database }}" TO "{{ $database }}_readonly";
 
     DO $$
     DECLARE
@@ -165,14 +167,14 @@ stringData:
     {{- range $database, $d := .Values.databases }}
     {{- range $user, $u := $.Values.users }}
     {{- if has $user $d.roles.admin }}
-    GRANT {{ $database }}_admin TO {{ $user }};
+    GRANT "{{ $database }}_admin" TO "{{ $user }}";
     {{- else }}
-    REVOKE {{ $database }}_admin FROM {{ $user }};
+    REVOKE "{{ $database }}_admin" FROM "{{ $user }}";
     {{- end }}
     {{- if has $user $d.roles.readonly }}
-    GRANT {{ $database }}_readonly TO {{ $user }};
+    GRANT "{{ $database }}_readonly" TO "{{ $user }}";
     {{- else }}
-    REVOKE {{ $database }}_readonly FROM {{ $user }};
+    REVOKE "{{ $database }}_readonly" FROM "{{ $user }}";
     {{- end }}
     {{- end }}
     {{- end }}

packages/apps/postgres/values.schema.json

@@ -62,55 +62,80 @@
             "properties": {
                 "enabled": {
                     "type": "boolean",
-                    "description": "Enable pereiodic backups",
+                    "description": "Enable regular backups",
                     "default": false
                 },
-                "s3Region": {
-                    "type": "string",
-                    "description": "The AWS S3 region where backups are stored",
-                    "default": "us-east-1"
-                },
-                "s3Bucket": {
-                    "type": "string",
-                    "description": "The S3 bucket used for storing backups",
-                    "default": "s3.example.org/postgres-backups"
-                },
                 "schedule": {
                     "type": "string",
                     "description": "Cron schedule for automated backups",
-                    "default": "0 2 * * *"
+                    "default": "0 2 * * * *"
                 },
-                "cleanupStrategy": {
+                "retentionPolicy": {
                     "type": "string",
-                    "description": "The strategy for cleaning up old backups",
-                    "default": "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+                    "description": "Retention policy",
+                    "default": "30d"
+                },
+                "destinationPath": {
+                    "type": "string",
+                    "description": "Path to store the backup (i.e. s3://bucket/path/to/folder)",
+                    "default": "s3://bucket/path/to/folder/"
+                },
+                "endpointURL": {
+                    "type": "string",
+                    "description": "S3 Endpoint used to upload data to the cloud",
+                    "default": "http://minio-gateway-service:9000"
                 },
                 "s3AccessKey": {
                     "type": "string",
-                    "description": "The access key for S3, used for authentication",
+                    "description": "Access key for S3, used for authentication",
                     "default": "oobaiRus9pah8PhohL1ThaeTa4UVa7gu"
                 },
                 "s3SecretKey": {
                     "type": "string",
-                    "description": "The secret key for S3, used for authentication",
+                    "description": "Secret key for S3, used for authentication",
                     "default": "ju3eum4dekeich9ahM1te8waeGai0oog"
+                }
+            }
+        },
+        "bootstrap": {
+            "type": "object",
+            "properties": {
+                "enabled": {
+                    "type": "boolean",
+                    "description": "Restore database cluster from a backup",
+                    "default": false
                 },
-                "resticPassword": {
+                "recoveryTime": {
                     "type": "string",
-                    "description": "The password for Restic backup encryption",
-                    "default": "ChaXoveekoh6eigh4siesheeda2quai0"
+                    "description": "Timestamp (PITR) up to which recovery will proceed, expressed in RFC 3339 format. If left empty, will restore latest",
+                    "default": ""
+                },
+                "oldName": {
+                    "type": "string",
+                    "description": "Name of database cluster before deleting",
+                    "default": ""
                 }
             }
         },
         "resources": {
             "type": "object",
-            "description": "Resources",
+            "description": "Explicit CPU and memory configuration for each PostgreSQL replica. When left empty, the preset defined in `resourcesPreset` is applied.",
             "default": {}
         },
         "resourcesPreset": {
             "type": "string",
-            "description": "Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).",
-            "default": "nano"
+            "description": "Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.",
+            "default": "micro",
+            "enum": [
+                "none",
+                "nano",
+                "micro",
+                "small",
+                "medium",
+                "large",
+                "xlarge",
+                "2xlarge"
+            ]
         }
     }
-}
+}

packages/apps/postgres/values.yaml

@@ -59,33 +59,39 @@ databases: {}
 
 ## @section Backup parameters
 
-## @param backup.enabled Enable pereiodic backups
-## @param backup.s3Region The AWS S3 region where backups are stored
-## @param backup.s3Bucket The S3 bucket used for storing backups
+## @param backup.enabled Enable regular backups
 ## @param backup.schedule Cron schedule for automated backups
-## @param backup.cleanupStrategy The strategy for cleaning up old backups
-## @param backup.s3AccessKey The access key for S3, used for authentication
-## @param backup.s3SecretKey The secret key for S3, used for authentication
-## @param backup.resticPassword The password for Restic backup encryption
+## @param backup.retentionPolicy Retention policy
+## @param backup.destinationPath Path to store the backup (i.e. s3://bucket/path/to/folder)
+## @param backup.endpointURL S3 Endpoint used to upload data to the cloud
+## @param backup.s3AccessKey Access key for S3, used for authentication
+## @param backup.s3SecretKey Secret key for S3, used for authentication
 backup:
   enabled: false
-  s3Region: us-east-1
-  s3Bucket: s3.example.org/postgres-backups
-  schedule: "0 2 * * *"
-  cleanupStrategy: "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+  retentionPolicy: 30d
+  destinationPath: s3://bucket/path/to/folder/
+  endpointURL: http://minio-gateway-service:9000
+  schedule: "0 2 * * * *"
   s3AccessKey: oobaiRus9pah8PhohL1ThaeTa4UVa7gu
   s3SecretKey: ju3eum4dekeich9ahM1te8waeGai0oog
-  resticPassword: ChaXoveekoh6eigh4siesheeda2quai0
 
-## @param resources Resources
+## @section Bootstrap parameters
+
+## @param bootstrap.enabled Restore database cluster from a backup
+## @param bootstrap.recoveryTime Timestamp (PITR) up to which recovery will proceed, expressed in RFC 3339 format. If left empty, will restore latest
+## @param bootstrap.oldName Name of database cluster before deleting
+##
+bootstrap:
+  enabled: false
+  # example: 2020-11-26 15:22:00.00000+00
+  recoveryTime: ""
+  oldName: ""
+
+## @param resources Explicit CPU and memory configuration for each PostgreSQL replica. When left empty, the preset defined in `resourcesPreset` is applied.
 resources: {}
  # resources:
- #   limits:
- #     cpu: 4000m
- #     memory: 4Gi
- #   requests:
- #     cpu: 100m
- #     memory: 512Mi
+ #   cpu: 4000m
+ #   memory: 4Gi
  
-## @param resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production).
-resourcesPreset: "nano"
+## @param resourcesPreset Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.
+resourcesPreset: "micro"

packages/apps/rabbitmq/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.7.0
+version: 0.8.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/rabbitmq/Makefile

@@ -2,3 +2,4 @@ include ../../../scripts/package.mk
 
 generate:
 	readme-generator -v values.yaml -s values.schema.json -r README.md
+	yq -i -o json --indent 4 '.properties.resourcesPreset.enum = ["none", "nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"]' values.schema.json

packages/apps/rabbitmq/README.md

@@ -22,9 +22,36 @@ The service utilizes official RabbitMQ operator. This ensures the reliability an
 
 ### Configuration parameters
 
-| Name              | Description                                                                                                                                                                                                       | Value  |
-| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
-| `users`           | Users configuration                                                                                                                                                                                               | `{}`   |
-| `vhosts`          | Virtual Hosts configuration                                                                                                                                                                                       | `{}`   |
-| `resources`       | Resources                                                                                                                                                                                                         | `{}`   |
-| `resourcesPreset` | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if resources is set (resources is recommended for production). | `nano` |
+| Name              | Description                                                                                                                           | Value  |
+| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------ |
+| `users`           | Users configuration                                                                                                                   | `{}`   |
+| `vhosts`          | Virtual Hosts configuration                                                                                                           | `{}`   |
+| `resources`       | Explicit CPU and memory configuration for each RabbitMQ replica. When left empty, the preset defined in `resourcesPreset` is applied. | `{}`   |
+| `resourcesPreset` | Default sizing preset used when `resources` is omitted. Allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge.     | `nano` |
+
+## Parameter examples and reference
+
+### resources and resourcesPreset
+
+`resources` sets explicit CPU and memory configurations for each replica.
+When left empty, the preset defined in `resourcesPreset` is applied.
+
+```yaml
+resources:
+  cpu: 4000m
+  memory: 4Gi
+```
+
+`resourcesPreset` sets named CPU and memory configurations for each replica.
+This setting is ignored if the corresponding `resources` value is set.
+
+| Preset name | CPU    | memory  |
+|-------------|--------|---------|
+| `nano`      | `100m` | `128Mi` |
+| `micro`     | `250m` | `256Mi` |
+| `small`     | `500m` | `512Mi` |
+| `medium`    | `500m` | `1Gi`   |
+| `large`     | `1`    | `2Gi`   |
+| `xlarge`    | `2`    | `4Gi`   |
+| `2xlarge`   | `4`    | `8Gi`   |
+