hotfix: handle missing flux-operator release during upgrade (#198)

image to test:

```
ghcr.io/aenix-io/cozystack/cozystack:v0.8.0@sha256:48e9f676f4eca5f7036648a56767c31beb0aca8fdc6d6798bd65de74886ed1ef
```


this PR should fix a problem of upgrading from older cozystack version

```
make: Leaving directory '/cozystack/packages/core/platform'
deployment.apps/source-controller condition met
deployment.apps/helm-controller condition met
Error from server (NotFound): helmreleases.helm.toolkit.fluxcd.io "fluxcd" not found
NAME                                        CREATED AT
helmreleases.helm.toolkit.fluxcd.io         2024-05-29T11:00:16Z
helmrepositories.source.toolkit.fluxcd.io   2024-05-29T11:00:17Z
make: Entering directory '/cozystack/packages/system/fluxcd-operator'
kubectl patch hr -n cozy-fluxcd fluxcd-operator -p '{"spec": {"suspend": true}}' --type=merge --field-manager=flux-client-side-apply
Error from server (NotFound): helmreleases.helm.toolkit.fluxcd.io "fluxcd-operator" not found
make: *** [../../../scripts/package-system.mk:20: suspend] Error 1
make: Leaving directory '/cozystack/packages/system/fluxcd-operator'
time="2024-07-04T12:50:05Z" level=fatal msg="failed to run" err="exit status 2"
```

.gitignore

@@ -1 +1,3 @@
 _out
+.git
+.idea

Makefile

@@ -3,6 +3,8 @@
 build:
 	make -C packages/apps/http-cache image
 	make -C packages/apps/kubernetes image
+	make -C packages/system/cilium image
+	make -C packages/system/kubeovn image
 	make -C packages/system/dashboard image
 	make -C packages/core/installer image
 	make manifests
@@ -20,4 +22,4 @@ repos:
 	make -C packages/extra repo
 
 assets:
-	make -C packages/core/talos/ assets
+	make -C packages/core/installer/ assets

README.md

@@ -10,7 +10,7 @@
 
 # Cozystack
 
-**Cozystack** is an open-source **PaaS platform** for cloud providers.
+**Cozystack** is a free PaaS platform and framework for building clouds.
 
 With Cozystack, you can transform your bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
 
@@ -33,7 +33,7 @@ You can use Cozystack as Kubernetes distribution for Bare Metal
 
 ## Documentation
 
-The documentation is located on official [cozystack.io](cozystack.io) website.
+The documentation is located on official [cozystack.io](https://cozystack.io) website.
 
 Read [Get Started](https://cozystack.io/docs/get-started/) section for a quick start.
 
@@ -44,6 +44,8 @@ If you encounter any difficulties, start with the [troubleshooting guide](https:
 Versioning adheres to the [Semantic Versioning](http://semver.org/) principles.  
 A full list of the available releases is available in the GitHub repository's [Release](https://github.com/aenix-io/cozystack/releases) section.
 
+- [Roadmap](https://github.com/orgs/aenix-io/projects/2)
+
 ## Contributions
 
 Contributions are highly appreciated and very welcomed!

hack/e2e.sh

@@ -0,0 +1,318 @@
+#!/bin/bash
+if [ "$COZYSTACK_INSTALLER_YAML" = "" ]; then
+  echo 'COZYSTACK_INSTALLER_YAML variable is not set!' >&2
+  echo 'please set it with following command:' >&2
+  echo >&2
+  echo 'export COZYSTACK_INSTALLER_YAML=$(helm template -n cozy-system installer packages/core/installer)' >&2
+  echo >&2
+  exit 1
+fi
+
+if [ "$(cat /proc/sys/net/ipv4/ip_forward)" != 1 ]; then
+  echo "IPv4 forwarding is not enabled!" >&2
+  echo 'please enable forwarding with the following command:' >&2
+  echo >&2
+  echo 'echo 1 > /proc/sys/net/ipv4/ip_forward' >&2
+  echo >&2
+  exit 1
+fi
+
+set -x
+set -e
+
+kill `cat srv1/qemu.pid srv2/qemu.pid srv3/qemu.pid` || true
+
+ip link del cozy-br0 || true
+ip link add cozy-br0 type bridge
+ip link set cozy-br0 up
+ip addr add 192.168.123.1/24 dev cozy-br0
+
+# Enable forward & masquerading
+echo 1 > /proc/sys/net/ipv4/ip_forward
+iptables -t nat -A POSTROUTING -s 192.168.123.0/24 -j MASQUERADE
+
+rm -rf srv1 srv2 srv3
+mkdir -p srv1 srv2 srv3
+
+# Prepare cloud-init
+for i in 1 2 3; do
+  echo "local-hostname: srv$i" > "srv$i/meta-data"
+  echo '#cloud-config' > "srv$i/user-data"
+  cat > "srv$i/network-config" <<EOT
+version: 2
+ethernets:
+  eth0:
+    dhcp4: false
+    addresses:
+      - "192.168.123.1$i/26"
+    gateway4: "192.168.123.1"
+    nameservers:
+      search: [cluster.local]
+      addresses: [8.8.8.8]
+EOT
+
+  ( cd srv$i && genisoimage \
+      -output seed.img \
+      -volid cidata -rational-rock -joliet \
+      user-data meta-data network-config
+  )
+done
+
+# Prepare system drive
+if [ ! -f nocloud-amd64.raw ]; then
+  wget https://github.com/aenix-io/cozystack/releases/latest/download/nocloud-amd64.raw.xz -O nocloud-amd64.raw.xz
+  rm -f nocloud-amd64.raw
+  xz --decompress nocloud-amd64.raw.xz
+fi
+for i in 1 2 3; do
+  cp nocloud-amd64.raw srv$i/system.img
+  qemu-img resize srv$i/system.img 20G
+done
+
+# Prepare data drives
+for i in 1 2 3; do
+  qemu-img create srv$i/data.img 100G
+done
+
+# Prepare networking
+for i in 1 2 3; do
+  ip link del cozy-srv$i || true
+  ip tuntap add dev cozy-srv$i mode tap
+  ip link set cozy-srv$i up
+  ip link set cozy-srv$i master cozy-br0
+done
+
+# Start VMs
+for i in 1 2 3; do
+  qemu-system-x86_64 -machine type=pc,accel=kvm -cpu host -smp 4 -m 8192 \
+    -device virtio-net,netdev=net0,mac=52:54:00:12:34:5$i -netdev tap,id=net0,ifname=cozy-srv$i,script=no,downscript=no \
+    -drive file=srv$i/system.img,if=virtio,format=raw \
+    -drive file=srv$i/seed.img,if=virtio,format=raw \
+    -drive file=srv$i/data.img,if=virtio,format=raw \
+    -display none -daemonize -pidfile srv$i/qemu.pid
+done
+
+sleep 5
+
+# Wait for VM to start up
+timeout 60 sh -c 'until nc -nzv 192.168.123.11 50000 && nc -nzv 192.168.123.12 50000 && nc -nzv 192.168.123.13 50000; do sleep 1; done'
+
+cat > patch.yaml <<\EOT
+machine:
+  kubelet:
+    nodeIP:
+      validSubnets:
+      - 192.168.123.0/24
+    extraConfig:
+      maxPods: 512
+  kernel:
+    modules:
+    - name: openvswitch
+    - name: drbd
+      parameters:
+        - usermode_helper=disabled
+    - name: zfs
+    - name: spl
+  install:
+    image: ghcr.io/aenix-io/cozystack/talos:v1.7.1
+  files:
+  - content: |
+      [plugins]
+        [plugins."io.containerd.grpc.v1.cri"]
+          device_ownership_from_security_context = true      
+    path: /etc/cri/conf.d/20-customization.part
+    op: create
+
+cluster:
+  network:
+    cni:
+      name: none
+    dnsDomain: cozy.local
+    podSubnets:
+    - 10.244.0.0/16
+    serviceSubnets:
+    - 10.96.0.0/16
+EOT
+
+cat > patch-controlplane.yaml <<\EOT
+machine:
+  network:
+    interfaces:
+    - interface: eth0
+      vip:
+        ip: 192.168.123.10
+cluster:
+  allowSchedulingOnControlPlanes: true
+  controllerManager:
+    extraArgs:
+      bind-address: 0.0.0.0
+  scheduler:
+    extraArgs:
+      bind-address: 0.0.0.0
+  apiServer:
+    certSANs:
+    - 127.0.0.1
+  proxy:
+    disabled: true
+  discovery:
+    enabled: false
+  etcd:
+    advertisedSubnets:
+    - 192.168.123.0/24
+EOT
+
+# Gen configuration
+if [ ! -f secrets.yaml ]; then
+  talosctl gen secrets
+fi
+
+rm -f controlplane.yaml worker.yaml talosconfig kubeconfig
+talosctl gen config --with-secrets secrets.yaml cozystack https://192.168.123.10:6443 --config-patch=@patch.yaml --config-patch-control-plane @patch-controlplane.yaml
+export TALOSCONFIG=$PWD/talosconfig
+
+# Apply configuration
+talosctl apply -f controlplane.yaml -n 192.168.123.11 -e 192.168.123.11 -i
+talosctl apply -f controlplane.yaml -n 192.168.123.12 -e 192.168.123.12 -i
+talosctl apply -f controlplane.yaml -n 192.168.123.13 -e 192.168.123.13 -i
+
+# Wait for VM to be configured
+timeout 60 sh -c 'until nc -nzv 192.168.123.11 50000 && nc -nzv 192.168.123.12 50000 && nc -nzv 192.168.123.13 50000; do sleep 1; done'
+
+# Bootstrap
+talosctl bootstrap -n 192.168.123.11 -e 192.168.123.11
+
+# Wait for etcd
+timeout 120 sh -c 'while talosctl etcd members -n 192.168.123.11,192.168.123.12,192.168.123.13 -e 192.168.123.10 2>&1 | grep "rpc error"; do sleep 1; done'
+
+rm -f kubeconfig
+talosctl kubeconfig kubeconfig -e 192.168.123.10 -n 192.168.123.10
+export KUBECONFIG=$PWD/kubeconfig
+
+# Wait for kubernetes nodes appear
+timeout 60 sh -c 'until [ $(kubectl get node -o name | wc -l) = 3 ]; do sleep 1; done'
+kubectl create ns cozy-system
+kubectl create -f - <<\EOT
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cozystack
+  namespace: cozy-system
+data:
+  bundle-name: "paas-full"
+  ipv4-pod-cidr: "10.244.0.0/16"
+  ipv4-pod-gateway: "10.244.0.1"
+  ipv4-svc-cidr: "10.96.0.0/16"
+  ipv4-join-cidr: "100.64.0.0/16"
+EOT
+
+#
+echo "$COZYSTACK_INSTALLER_YAML" | kubectl apply -f -
+
+# wait for cozystack pod to start
+kubectl wait deploy  --timeout=1m --for=condition=available -n cozy-system cozystack
+
+# wait for helmreleases appear
+timeout 60 sh -c 'until kubectl get hr -A | grep cozy; do sleep 1; done'
+
+sleep 5
+
+kubectl get hr -A | awk 'NR>1 {print "kubectl wait --timeout=15m --for=condition=ready -n " $1 " hr/" $2 " &"} END{print "wait"}' | sh -x
+# Wait for linstor controller
+kubectl wait deploy --timeout=5m --for=condition=available -n cozy-linstor linstor-controller
+
+# Wait for all linstor nodes become Online
+timeout 60 sh -c 'until [ $(kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor node list | grep -c Online) = 3 ]; do sleep 1; done'
+
+kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor ps cdp zfs srv1 /dev/vdc --pool-name data --storage-pool data
+kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor ps cdp zfs srv2 /dev/vdc --pool-name data --storage-pool data
+kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor ps cdp zfs srv3 /dev/vdc --pool-name data --storage-pool data
+
+kubectl create -f- <<EOT
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: local
+  annotations:
+    storageclass.kubernetes.io/is-default-class: "true"
+provisioner: linstor.csi.linbit.com
+parameters:
+  linstor.csi.linbit.com/storagePool: "data"
+  linstor.csi.linbit.com/layerList: "storage"
+  linstor.csi.linbit.com/allowRemoteVolumeAccess: "false"
+volumeBindingMode: WaitForFirstConsumer
+allowVolumeExpansion: true
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: replicated
+provisioner: linstor.csi.linbit.com
+parameters:
+  linstor.csi.linbit.com/storagePool: "data"
+  linstor.csi.linbit.com/autoPlace: "3"
+  linstor.csi.linbit.com/layerList: "drbd storage"
+  linstor.csi.linbit.com/allowRemoteVolumeAccess: "true"
+  property.linstor.csi.linbit.com/DrbdOptions/auto-quorum: suspend-io
+  property.linstor.csi.linbit.com/DrbdOptions/Resource/on-no-data-accessible: suspend-io
+  property.linstor.csi.linbit.com/DrbdOptions/Resource/on-suspended-primary-outdated: force-secondary
+  property.linstor.csi.linbit.com/DrbdOptions/Net/rr-conflict: retry-connect
+volumeBindingMode: WaitForFirstConsumer
+allowVolumeExpansion: true
+EOT
+kubectl create -f- <<EOT
+---
+apiVersion: metallb.io/v1beta1
+kind: L2Advertisement
+metadata:
+  name: cozystack
+  namespace: cozy-metallb
+spec:
+  ipAddressPools:
+  - cozystack
+---
+apiVersion: metallb.io/v1beta1
+kind: IPAddressPool
+metadata:
+  name: cozystack
+  namespace: cozy-metallb
+spec:
+  addresses:
+  - 192.168.123.200-192.168.123.250
+  autoAssign: true
+  avoidBuggyIPs: false
+EOT
+
+kubectl patch -n tenant-root hr/tenant-root --type=merge -p '{"spec":{ "values":{
+  "host": "example.org",
+  "ingress": true,
+  "monitoring": true,
+  "etcd": true
+}}}'
+
+# Wait for HelmRelease be created
+timeout 60 sh -c 'until kubectl get hr -n tenant-root etcd ingress monitoring tenant-root; do sleep 1; done'
+
+# Wait for HelmReleases be installed
+kubectl wait --timeout=2m --for=condition=ready -n tenant-root hr etcd ingress monitoring tenant-root
+
+# Wait for nginx-ingress-controller
+timeout 60 sh -c 'until kubectl get deploy -n tenant-root root-ingress-controller; do sleep 1; done'
+kubectl wait --timeout=5m --for=condition=available -n tenant-root deploy root-ingress-controller
+
+# Wait for etcd
+kubectl wait --timeout=5m --for=jsonpath=.status.readyReplicas=3 -n tenant-root sts etcd
+
+# Wait for Victoria metrics
+kubectl wait --timeout=5m --for=condition=available deploy -n tenant-root vmalert-vmalert vminsert-longterm vminsert-shortterm
+kubectl wait --timeout=5m --for=jsonpath=.status.readyReplicas=2 -n tenant-root sts vmalertmanager-alertmanager vmselect-longterm vmselect-shortterm vmstorage-longterm vmstorage-shortterm
+
+# Wait for grafana
+kubectl wait --timeout=5m --for=condition=ready -n tenant-root clusters.postgresql.cnpg.io grafana-db
+kubectl wait --timeout=5m --for=condition=available -n tenant-root deploy grafana-deployment 
+
+# Get IP of nginx-ingress
+ip=$(kubectl get svc -n tenant-root root-ingress-controller -o jsonpath='{.status.loadBalancer.ingress..ip}')
+
+# Check Grafana
+curl -sS -k "https://$ip" -H 'Host: grafana.example.org' | grep Found

hack/gen_versions_map.sh

@@ -20,9 +20,28 @@ miss_map=$(echo "$new_map" | awk 'NR==FNR { new_map[$1 " " $2] = $3; next } { if
 resolved_miss_map=$(
   echo "$miss_map" | while read chart version commit; do
     if [ "$commit" = HEAD ]; then
-      line=$(git show HEAD:"./$chart/Chart.yaml" | awk '/^version:/ {print NR; exit}')
-      change_commit=$(git --no-pager blame -L"$line",+1 HEAD -- "$chart/Chart.yaml" | awk '{print $1}')
-      commit=$(git describe --always "$change_commit~1")
+      line=$(awk '/^version:/ {print NR; exit}' "./$chart/Chart.yaml")
+      change_commit=$(git --no-pager blame -L"$line",+1 -- "$chart/Chart.yaml" | awk '{print $1}')
+       
+      if [ "$change_commit" = "00000000" ]; then
+        # Not commited yet, use previus commit
+        line=$(git show HEAD:"./$chart/Chart.yaml" | awk '/^version:/ {print NR; exit}')
+        commit=$(git --no-pager blame -L"$line",+1 HEAD -- "$chart/Chart.yaml" | awk '{print $1}')
+        if [ $(echo $commit | cut -c1) = "^" ]; then
+          # Previus commit not exists
+          commit=$(echo $commit | cut -c2-)
+        fi
+      else
+        # Commited, but version_map wasn't updated
+        line=$(git show HEAD:"./$chart/Chart.yaml" | awk '/^version:/ {print NR; exit}')
+        change_commit=$(git --no-pager blame -L"$line",+1 HEAD -- "$chart/Chart.yaml" | awk '{print $1}')
+        if [ $(echo $change_commit | cut -c1) = "^" ]; then
+          # Previus commit not exists
+          commit=$(echo $change_commit | cut -c2-)
+        else
+          commit=$(git describe --always "$change_commit~1")
+        fi
+      fi
     fi
     echo "$chart $version $commit"
   done

hack/prepare_release.sh

@@ -1,19 +0,0 @@
-#!/bin/sh
-set -e
-
-if [ -e $1 ]; then
-  echo "Please pass version in the first argument"
-  echo "Example: $0 v0.0.2"
-  exit 1
-fi
-
-version=$1
-talos_version=$(awk '/^version:/ {print $2}' packages/core/installer/images/talos/profiles/installer.yaml)
-
-set -x
-
-sed -i "/^TAG / s|=.*|= ${version}|" \
-  packages/apps/http-cache/Makefile \
-  packages/apps/kubernetes/Makefile \
-  packages/core/installer/Makefile \
-  packages/system/dashboard/Makefile

manifests/cozystack-installer.yaml

@@ -15,13 +15,6 @@ metadata:
   namespace: cozy-system
 ---
 # Source: cozy-installer/templates/cozystack.yaml
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: cozystack
-  namespace: cozy-system
----
-# Source: cozy-installer/templates/cozystack.yaml
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
@@ -61,6 +54,11 @@ spec:
   selector:
     matchLabels:
       app: cozystack
+  strategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxSurge: 0
+      maxUnavailable: 1
   template:
     metadata:
       labels:
@@ -70,7 +68,7 @@ spec:
       serviceAccountName: cozystack
       containers:
       - name: cozystack
-        image: "ghcr.io/aenix-io/cozystack/cozystack:v0.1.0"
+        image: "ghcr.io/aenix-io/cozystack/cozystack:v0.8.0"
         env:
         - name: KUBERNETES_SERVICE_HOST
           value: localhost
@@ -89,7 +87,7 @@ spec:
             fieldRef:
               fieldPath: metadata.name
       - name: darkhttpd
-        image: "ghcr.io/aenix-io/cozystack/cozystack:v0.1.0"
+        image: "ghcr.io/aenix-io/cozystack/cozystack:v0.8.0"
         command:
         - /usr/bin/darkhttpd
         - /cozystack/assets
@@ -102,3 +100,6 @@ spec:
       - key: "node.kubernetes.io/not-ready"
         operator: "Exists"
         effect: "NoSchedule"
+      - key: "node.cilium.io/agent-not-ready"
+        operator: "Exists"
+        effect: "NoSchedule"

packages/apps/Makefile

@@ -7,11 +7,11 @@ repo:
 	awk '$$3 != "HEAD" {print "mkdir -p $(TMP)/" $$1 "-" $$2}' versions_map | sh -ex
 	awk '$$3 != "HEAD" {print "git archive " $$3 " " $$1 " | tar -xf- --strip-components=1 -C $(TMP)/" $$1 "-" $$2 }' versions_map | sh -ex
 	helm package -d "$(OUT)" $$(find . $(TMP) -mindepth 2 -maxdepth 2 -name Chart.yaml | awk 'sub("/Chart.yaml", "")' | sort -V)
-	cd "$(OUT)" && helm repo index .
+	cd "$(OUT)" && helm repo index . --url http://cozystack.cozy-system.svc/repos/apps
 	rm -rf "$(TMP)"
 
 fix-chartnames:
-	find . -name Chart.yaml -maxdepth 2 | awk -F/ '{print $$2}' | while read i; do sed -i "s/^name: .*/name: $$i/" "$$i/Chart.yaml"; done
+	find . -maxdepth 2 -name Chart.yaml  | awk -F/ '{print $$2}' | while read i; do sed -i "s/^name: .*/name: $$i/" "$$i/Chart.yaml"; done
 
 gen-versions-map: fix-chartnames
 	../../hack/gen_versions_map.sh

packages/apps/clickhouse/Chart.yaml

@@ -0,0 +1,25 @@
+apiVersion: v2
+name: clickhouse
+description: Managed ClickHouse service
+icon: https://cdn.worldvectorlogo.com/logos/clickhouse.svg
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.2.1
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "24.3.0"

packages/apps/clickhouse/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/clickhouse/README.md

@@ -0,0 +1,17 @@
+# Managed Clickhouse Service
+
+## Parameters
+
+### Common parameters
+
+| Name       | Description                   | Value  |
+| ---------- | ----------------------------- | ------ |
+| `size`     | Persistent Volume size        | `10Gi` |
+| `shards`   | Number of Clickhouse replicas | `1`    |
+| `replicas` | Number of Clickhouse shards   | `2`    |
+
+### Configuration parameters
+
+| Name    | Description         | Value |
+| ------- | ------------------- | ----- |
+| `users` | Users configuration | `{}`  |

packages/apps/clickhouse/templates/clickhouse.yaml

@@ -0,0 +1,37 @@
+apiVersion: "clickhouse.altinity.com/v1"
+kind: "ClickHouseInstallation"
+metadata:
+  name: "{{ .Release.Name }}"
+spec:
+  {{- with .Values.size }}
+  defaults:
+    templates:
+      dataVolumeClaimTemplate: data-volume-template
+  {{- end }}
+  configuration:
+    {{- with .Values.users }}
+    users:
+      {{- range $name, $u := . }}
+      {{ $name }}/password_sha256_hex: {{ sha256sum $u.password }}
+      {{ $name }}/profile: {{ ternary "readonly" "default" (index $u "readonly" | default false) }}
+      {{ $name }}/networks/ip: ["::/0"]
+      {{- end }}
+    {{- end }}
+    profiles:
+      readonly/readonly: "1"
+    clusters:
+      - name: "clickhouse"
+        layout:
+          shardsCount: {{ .Values.shards }}
+          replicasCount: {{ .Values.replicas }}
+  {{- with .Values.size }}
+  templates:
+    volumeClaimTemplates:
+      - name: data-volume-template
+        spec:
+          accessModes:
+            - ReadWriteOnce
+          resources:
+            requests:
+              storage: {{ . }}
+  {{- end }}

packages/apps/clickhouse/values.schema.json

@@ -0,0 +1,21 @@
+{
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "size": {
+            "type": "string",
+            "description": "Persistent Volume size",
+            "default": "10Gi"
+        },
+        "shards": {
+            "type": "number",
+            "description": "Number of Clickhouse replicas",
+            "default": 1
+        },
+        "replicas": {
+            "type": "number",
+            "description": "Number of Clickhouse shards",
+            "default": 2
+        }
+    }
+}

packages/apps/clickhouse/values.yaml

@@ -0,0 +1,22 @@
+## @section Common parameters
+
+## @param size Persistent Volume size
+## @param shards Number of Clickhouse replicas
+## @param replicas Number of Clickhouse shards
+##
+size: 10Gi
+shards: 1
+replicas: 2
+
+## @section Configuration parameters
+
+## @param users [object] Users configuration
+## Example:
+## users:
+##   user1:
+##     password: strongpassword
+##   user2:
+##     readonly: true
+##     password: hackme
+##
+users: {}

packages/apps/http-cache/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.2.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "1.25.3"

packages/apps/http-cache/Makefile

@@ -1,22 +1,23 @@
-PUSH := 1
-LOAD := 0
-REGISTRY := ghcr.io/aenix-io/cozystack
 NGINX_CACHE_TAG = v0.1.0
-TAG := v0.1.0
+
+include ../../../scripts/common-envs.mk
 
 image: image-nginx
 
 image-nginx:
 	docker buildx build --platform linux/amd64 --build-arg ARCH=amd64 images/nginx-cache \
 		--provenance false \
-		--tag $(REGISTRY)/nginx-cache:$(NGINX_CACHE_TAG) \
-		--tag $(REGISTRY)/nginx-cache:$(NGINX_CACHE_TAG)-$(TAG) \
-		--cache-from type=registry,ref=$(REGISTRY)/nginx-cache:$(NGINX_CACHE_TAG) \
+		--tag $(REGISTRY)/nginx-cache:$(call settag,$(NGINX_CACHE_TAG)) \
+		--tag $(REGISTRY)/nginx-cache:$(call settag,$(NGINX_CACHE_TAG)-$(TAG)) \
+		--cache-from type=registry,ref=$(REGISTRY)/nginx-cache:latest \
 		--cache-to type=inline \
 		--metadata-file images/nginx-cache.json \
 		--push=$(PUSH) \
 		--load=$(LOAD)
-	echo "$(REGISTRY)/nginx-cache:$(NGINX_CACHE_TAG)" > images/nginx-cache.tag
+	echo "$(REGISTRY)/nginx-cache:$(call settag,$(NGINX_CACHE_TAG))" > images/nginx-cache.tag
+
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md
 
 update:
 	tag=$$(git ls-remote --tags --sort="v:refname" https://github.com/chrislim2888/IP2Location-C-Library | awk -F'[/^]' 'END{print $$3}') && \

packages/apps/http-cache/README.md

@@ -55,3 +55,20 @@ The deployment architecture is illustrated in the diagram below:
 
 VTS module shows wrong upstream resonse time
 - https://github.com/vozlt/nginx-module-vts/issues/198
+
+## Parameters
+
+### Common parameters
+
+| Name               | Description                                     | Value   |
+| ------------------ | ----------------------------------------------- | ------- |
+| `external`         | Enable external access from outside the cluster | `false` |
+| `size`             | Persistent Volume size                          | `10Gi`  |
+| `haproxy.replicas` | Number of HAProxy replicas                      | `2`     |
+| `nginx.replicas`   | Number of Nginx replicas                        | `2`     |
+
+### Configuration parameters
+
+| Name        | Description             | Value |
+| ----------- | ----------------------- | ----- |
+| `endpoints` | Endpoints configuration | `[]`  |

packages/apps/http-cache/images/nginx-cache.json

@@ -1,4 +1,48 @@
 {
-  "containerimage.config.digest": "sha256:318fd8d0d6f6127387042f6ad150e87023d1961c7c5059dd5324188a54b0ab4e",
-  "containerimage.digest": "sha256:e3cf145238e6e45f7f13b9acaea445c94ff29f76a34ba9fa50828401a5a3cc68"
+  "buildx.build.provenance": {
+    "buildType": "https://mobyproject.org/buildkit@v1",
+    "materials": [
+      {
+        "uri": "pkg:docker/ubuntu@22.04?platform=linux%2Famd64",
+        "digest": {
+          "sha256": "340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221"
+        }
+      }
+    ],
+    "invocation": {
+      "configSource": {
+        "entryPoint": "Dockerfile"
+      },
+      "parameters": {
+        "frontend": "dockerfile.v0",
+        "args": {
+          "build-arg:ARCH": "amd64"
+        },
+        "locals": [
+          {
+            "name": "context"
+          },
+          {
+            "name": "dockerfile"
+          }
+        ]
+      },
+      "environment": {
+        "platform": "linux/amd64"
+      }
+    }
+  },
+  "buildx.build.ref": "priceless_leavitt/priceless_leavitt0/iixrpj9up3jfyyrovbx167irv",
+  "containerimage.config.digest": "sha256:a359a6c83a1861ac146c42030e513b925ec155207b77baf5f61f19c507ab3ee5",
+  "containerimage.descriptor": {
+    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
+    "digest": "sha256:57b113a4392b958612c14ed348147c221fe17d3ad6623fb7bd778e54139f45b9",
+    "size": 1094,
+    "platform": {
+      "architecture": "amd64",
+      "os": "linux"
+    }
+  },
+  "containerimage.digest": "sha256:57b113a4392b958612c14ed348147c221fe17d3ad6623fb7bd778e54139f45b9",
+  "image.name": "ghcr.io/aenix-io/cozystack/nginx-cache:v0.1.0,ghcr.io/aenix-io/cozystack/nginx-cache:v0.1.0-v0.8.0"
 }

packages/apps/http-cache/templates/haproxy/configmap.yaml

@@ -74,7 +74,7 @@ data:
         option redispatch 1
         default-server observe layer7 error-limit 10 on-error mark-down
 
-        {{- range $i, $e := until (int $.Values.replicas) }}
+        {{- range $i, $e := until (int $.Values.nginx.replicas) }}
         server cache{{ $i }} {{ $.Release.Name }}-nginx-cache-{{ $i }}:80 check
         {{- end }}
         {{- range $i, $e := $.Values.endpoints }} 

packages/apps/http-cache/templates/haproxy/deployment.yaml

@@ -7,7 +7,7 @@ metadata:
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/managed-by: {{ .Release.Service }}
 spec:
-  replicas: 2
+  replicas: {{ .Values.haproxy.replicas }}
   selector:
     matchLabels:
       app: {{ .Release.Name }}-haproxy

packages/apps/http-cache/templates/nginx/deployment.yaml

@@ -11,7 +11,7 @@ spec:
   selector:
     matchLabels:
       app: {{ $.Release.Name }}-nginx-cache
-{{- range $i := until 3 }}
+{{- range $i := until (int $.Values.nginx.replicas) }}
 ---
 apiVersion: apps/v1
 kind: Deployment

packages/apps/http-cache/values.schema.json

@@ -0,0 +1,42 @@
+{
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "size": {
+            "type": "string",
+            "description": "Persistent Volume size",
+            "default": "10Gi"
+        },
+        "haproxy": {
+            "type": "object",
+            "properties": {
+                "replicas": {
+                    "type": "number",
+                    "description": "Number of HAProxy replicas",
+                    "default": 2
+                }
+            }
+        },
+        "nginx": {
+            "type": "object",
+            "properties": {
+                "replicas": {
+                    "type": "number",
+                    "description": "Number of Nginx replicas",
+                    "default": 2
+                }
+            }
+        },
+        "endpoints": {
+            "type": "array",
+            "description": "Endpoints configuration",
+            "default": [],
+            "items": {}
+        }
+    }
+}

packages/apps/http-cache/values.yaml

@@ -1,9 +1,28 @@
+
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param size Persistent Volume size
+## @param haproxy.replicas Number of HAProxy replicas
+## @param nginx.replicas Number of Nginx replicas
+##
 external: false
 size: 10Gi
-endpoints:
-  - 10.100.3.1:80
-  - 10.100.3.11:80
-  - 10.100.3.2:80
-  - 10.100.3.12:80
-  - 10.100.3.3:80
-  - 10.100.3.13:80
+haproxy:
+  replicas: 2
+nginx:
+  replicas: 2
+
+## @section Configuration parameters
+
+## @param endpoints Endpoints configuration
+## Example:
+## endpoints:
+##   - 10.100.3.1:80
+##   - 10.100.3.11:80
+##   - 10.100.3.2:80
+##   - 10.100.3.12:80
+##   - 10.100.3.3:80
+##   - 10.100.3.13:80
+##
+endpoints: []

packages/apps/kafka/Chart.yaml

@@ -0,0 +1,25 @@
+apiVersion: v2
+name: kafka
+description: Managed Kafka service
+icon: https://upload.wikimedia.org/wikipedia/commons/0/05/Apache_kafka.svg
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.2.1
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "3.7.0"

packages/apps/kafka/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/kafka/README.md

@@ -0,0 +1,19 @@
+# Managed Kafka Service
+
+## Parameters
+
+### Common parameters
+
+| Name                 | Description                                     | Value   |
+| -------------------- | ----------------------------------------------- | ------- |
+| `external`           | Enable external access from outside the cluster | `false` |
+| `kafka.size`         | Persistent Volume size for Kafka                | `10Gi`  |
+| `kafka.replicas`     | Number of Kafka replicas                        | `3`     |
+| `zookeeper.size`     | Persistent Volume size for ZooKeeper            | `5Gi`   |
+| `zookeeper.replicas` | Number of ZooKeeper replicas                    | `3`     |
+
+### Configuration parameters
+
+| Name     | Description          | Value |
+| -------- | -------------------- | ----- |
+| `topics` | Topics configuration | `[]`  |

packages/apps/kafka/templates/kafka.yaml

@@ -0,0 +1,53 @@
+apiVersion: kafka.strimzi.io/v1beta2
+kind: Kafka
+metadata:
+  name: {{ .Release.Name }}
+  labels:
+    app.kubernetes.io/instance: {{ .Release.Name }}
+    app.kubernetes.io/managed-by: {{ .Release.Service }}
+spec:
+  kafka:
+    replicas: {{ .Values.kafka.replicas }}
+    listeners:
+      - name: plain
+        port: 9092
+        type: internal
+        tls: false
+      - name: tls
+        port: 9093
+        type: internal
+        tls: true
+      - name: external
+        port: 9094
+        {{- if .Values.external }}
+        type: loadbalancer
+        {{- else }}
+        type: internal
+        {{- end }}
+        tls: false
+    config:
+      offsets.topic.replication.factor: 3
+      transaction.state.log.replication.factor: 3
+      transaction.state.log.min.isr: 2
+      default.replication.factor: 3
+      min.insync.replicas: 2
+    storage:
+      type: jbod
+      volumes:
+      - id: 0
+        type: persistent-claim
+        {{- with .Values.kafka.size }}
+        size: {{ . }}
+        {{- end }}
+        deleteClaim: true
+  zookeeper:
+    replicas: {{ .Values.zookeeper.replicas }}
+    storage:
+      type: persistent-claim
+      {{- with .Values.zookeeper.size }}
+      size: {{ . }}
+      {{- end }}
+      deleteClaim: false
+  entityOperator:
+    topicOperator: {}
+    userOperator: {}

packages/apps/kafka/templates/topics.yaml

@@ -0,0 +1,21 @@
+{{- range $topic := .Values.topics }}
+---
+apiVersion: kafka.strimzi.io/v1beta2
+kind: KafkaTopic
+metadata:
+  name: "{{ $.Release.Name }}-{{ kebabcase $topic.name }}"
+  labels:
+    strimzi.io/cluster: "{{ $.Release.Name }}"
+spec:
+  topicName: "{{ $topic.name }}"
+  {{- with $topic.partitions }}
+  partitions: "{{ . }}"
+  {{- end }}
+  {{- with $topic.replicas }}
+  replicas: "{{ . }}"
+  {{- end }}
+  {{- with $topic.config }}
+  config:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+{{- end }}

packages/apps/kafka/values.schema.json

@@ -0,0 +1,47 @@
+{
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "kafka": {
+            "type": "object",
+            "properties": {
+                "size": {
+                    "type": "string",
+                    "description": "Persistent Volume size for Kafka",
+                    "default": "10Gi"
+                },
+                "replicas": {
+                    "type": "number",
+                    "description": "Number of Kafka replicas",
+                    "default": 3
+                }
+            }
+        },
+        "zookeeper": {
+            "type": "object",
+            "properties": {
+                "size": {
+                    "type": "string",
+                    "description": "Persistent Volume size for ZooKeeper",
+                    "default": "5Gi"
+                },
+                "replicas": {
+                    "type": "number",
+                    "description": "Number of ZooKeeper replicas",
+                    "default": 3
+                }
+            }
+        },
+        "topics": {
+            "type": "array",
+            "description": "Topics configuration",
+            "default": [],
+            "items": {}
+        }
+    }
+}

packages/apps/kafka/values.yaml

@@ -0,0 +1,37 @@
+
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param kafka.size Persistent Volume size for Kafka
+## @param kafka.replicas Number of Kafka replicas
+## @param zookeeper.size Persistent Volume size for ZooKeeper
+## @param zookeeper.replicas Number of ZooKeeper replicas
+##
+external: false
+kafka:
+  size: 10Gi
+  replicas: 3
+zookeeper:
+  size: 5Gi
+  replicas: 3
+
+## @section Configuration parameters
+
+## @param topics Topics configuration
+## Example:
+## topics:
+##   - name: Results
+##     partitions: 1
+##     replicas: 3
+##     config:
+##       min.insync.replicas: 2
+##   - name: Orders
+##     config:
+##       cleanup.policy: compact
+##       segment.ms: 3600000
+##       max.compaction.lag.ms: 5400000
+##       min.insync.replicas: 2
+##     partitions: 1
+##     replicas: 3
+##
+topics: []

packages/apps/kubernetes/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.5.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "1.19.4"

packages/apps/kubernetes/Makefile

@@ -1,19 +1,20 @@
-PUSH := 1
-LOAD := 0
-REGISTRY := ghcr.io/aenix-io/cozystack
-TAG := v0.1.0
 UBUNTU_CONTAINER_DISK_TAG = v1.29.1
 
+include ../../../scripts/common-envs.mk
+
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md
+
 image: image-ubuntu-container-disk
 
 image-ubuntu-container-disk:
 	docker buildx build --platform linux/amd64 --build-arg ARCH=amd64 images/ubuntu-container-disk \
 		--provenance false \
-		--tag $(REGISTRY)/ubuntu-container-disk:$(UBUNTU_CONTAINER_DISK_TAG) \
-		--tag $(REGISTRY)/ubuntu-container-disk:$(UBUNTU_CONTAINER_DISK_TAG)-$(TAG) \
-		--cache-from type=registry,ref=$(REGISTRY)/ubuntu-container-disk:$(UBUNTU_CONTAINER_DISK_TAG) \
+		--tag $(REGISTRY)/ubuntu-container-disk:$(call settag,$(UBUNTU_CONTAINER_DISK_TAG)) \
+		--tag $(REGISTRY)/ubuntu-container-disk:$(call settag,$(UBUNTU_CONTAINER_DISK_TAG)-$(TAG)) \
+		--cache-from type=registry,ref=$(REGISTRY)/ubuntu-container-disk:latest \
 		--cache-to type=inline \
 		--metadata-file images/ubuntu-container-disk.json \
 		--push=$(PUSH) \
 		--load=$(LOAD)
-	echo "$(REGISTRY)/ubuntu-container-disk:$(UBUNTU_CONTAINER_DISK_TAG)" > images/ubuntu-container-disk.tag
+	echo "$(REGISTRY)/ubuntu-container-disk:$(call settag,$(UBUNTU_CONTAINER_DISK_TAG))" > images/ubuntu-container-disk.tag

packages/apps/kubernetes/README.md

@@ -26,3 +26,23 @@ How to access to deployed cluster:
 ```
 kubectl get secret -n <namespace> kubernetes-<clusterName>-admin-kubeconfig -o go-template='{{ printf "%s\n" (index .data "super-admin.conf" | base64decode) }}' > test
 ```
+
+## Parameters
+
+### Common parameters
+
+| Name                    | Description                                                                                                                            | Value |
+| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----- |
+| `host`                  | The hostname used to access the Kubernetes cluster externally (defaults to using the cluster name as a subdomain for the tenant host). | `""`  |
+| `controlPlane.replicas` | Number of replicas for Kubernetes contorl-plane components                                                                             | `2`   |
+| `nodeGroups`            | nodeGroups configuration                                                                                                               | `{}`  |
+
+### Cluster Addons
+
+| Name                          | Description                                                                        | Value   |
+| ----------------------------- | ---------------------------------------------------------------------------------- | ------- |
+| `addons.certManager.enabled`  | Enables the cert-manager                                                           | `false` |
+| `addons.ingressNginx.enabled` | Enable Ingress-NGINX controller (expect nodes with 'ingress-nginx' role)           | `false` |
+| `addons.ingressNginx.hosts`   | List of domain names that should be passed through to the cluster by upper cluster | `[]`    |
+| `addons.fluxcd.enabled`       | Enables Flux CD                                                                    | `false` |
+

packages/apps/kubernetes/images/ubuntu-container-disk.json

@@ -1,4 +1,48 @@
 {
-  "containerimage.config.digest": "sha256:ee8968be63c7c45621ec45f3687211e0875acb24e8d9784e8d2ebcbf46a3538c",
-  "containerimage.digest": "sha256:16c3c07e74212585786dc1f1ae31d3ab90a575014806193e8e37d1d7751cb084"
+  "buildx.build.provenance": {
+    "buildType": "https://mobyproject.org/buildkit@v1",
+    "materials": [
+      {
+        "uri": "pkg:docker/ubuntu@22.04?platform=linux%2Famd64",
+        "digest": {
+          "sha256": "340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221"
+        }
+      }
+    ],
+    "invocation": {
+      "configSource": {
+        "entryPoint": "Dockerfile"
+      },
+      "parameters": {
+        "frontend": "dockerfile.v0",
+        "args": {
+          "build-arg:ARCH": "amd64"
+        },
+        "locals": [
+          {
+            "name": "context"
+          },
+          {
+            "name": "dockerfile"
+          }
+        ]
+      },
+      "environment": {
+        "platform": "linux/amd64"
+      }
+    }
+  },
+  "buildx.build.ref": "priceless_leavitt/priceless_leavitt0/yscjdwk0a8zfgvypn9gfzoeqj",
+  "containerimage.config.digest": "sha256:62f92e19bf0610f85515bef28db8465650a25346472f52360736ad3a49ce7529",
+  "containerimage.descriptor": {
+    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
+    "digest": "sha256:f1176049da8651f438b2035c5a2cb6fd253ae868c4640e56785c91364069e0d8",
+    "size": 506,
+    "platform": {
+      "architecture": "amd64",
+      "os": "linux"
+    }
+  },
+  "containerimage.digest": "sha256:f1176049da8651f438b2035c5a2cb6fd253ae868c4640e56785c91364069e0d8",
+  "image.name": "ghcr.io/aenix-io/cozystack/ubuntu-container-disk:v1.29.1,ghcr.io/aenix-io/cozystack/ubuntu-container-disk:v1.29.1-v0.8.0"
 }

packages/apps/kubernetes/templates/cluster-autoscaler/deployment.yaml

@@ -15,6 +15,12 @@ spec:
       labels:
         app: {{ .Release.Name }}-cluster-autoscaler
     spec:
+      tolerations:
+        - key: CriticalAddonsOnly
+          operator: Exists
+        - key: node-role.kubernetes.io/control-plane
+          operator: Exists
+          effect: "NoSchedule"
       containers:
       - image: ghcr.io/kvaps/test:cluster-autoscaller
         name: cluster-autoscaler

packages/apps/kubernetes/templates/cluster.yaml

@@ -39,7 +39,9 @@ metadata:
 spec:
   dataStoreName: "{{ $etcd }}"
   addons:
-    coreDNS: {}
+    coreDNS:
+      dnsServiceIPs:
+      - 10.95.0.10
     konnectivity: {}
   kubelet:
     cgroupfs: systemd
@@ -55,7 +57,7 @@ spec:
       className: "{{ $ingress }}"
   deployment:
   replicas: 2
-  version: 1.29.0
+  version: 1.29.4
 ---
 apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
 kind: KubevirtCluster
@@ -64,21 +66,37 @@ metadata:
     cluster.x-k8s.io/managed-by: kamaji
   name: {{ .Release.Name }}
   namespace: {{ .Release.Namespace }}
+{{- range $groupName, $group := .Values.nodeGroups }}
 ---
 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
 kind: KubeadmConfigTemplate
 metadata:
-  name: {{ .Release.Name }}-md-0
-  namespace: {{ .Release.Namespace }}
+  name: {{ $.Release.Name }}-{{ $groupName }}
+  namespace: {{ $.Release.Namespace }}
 spec:
   template:
     spec:
+      diskSetup:
+        filesystems:
+        - device: /dev/vdb
+          filesystem: xfs
+          label: containerd
+          partition: "none"
+        - device: /dev/vdc
+          filesystem: xfs
+          label: kubelet
+          partition: "none"
+      mounts:
+      - ["LABEL=containerd", "/var/lib/containerd"]
+      - ["LABEL=kubelet", "/var/lib/kubelet"]
+      preKubeadmCommands:
+      - sed -i 's|root:x:|root::|' /etc/passwd
       joinConfiguration:
         nodeRegistration:
           kubeletExtraArgs: {}
         discovery:
           bootstrapToken:
-            apiServerEndpoint: {{ .Release.Name }}.{{ .Release.Namespace }}.svc:6443
+            apiServerEndpoint: {{ $.Release.Name }}.{{ $.Release.Namespace }}.svc:6443
       initConfiguration:
         skipPhases:
         - addon/kube-proxy
@@ -86,8 +104,8 @@ spec:
 apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
 kind: KubevirtMachineTemplate
 metadata:
-  name: {{ .Release.Name }}-md-0
-  namespace: {{ .Release.Namespace }}
+  name: {{ $.Release.Name }}-{{ $groupName }}
+  namespace: {{ $.Release.Namespace }}
 spec:
   template:
     spec:
@@ -95,7 +113,11 @@ spec:
         checkStrategy: ssh
       virtualMachineTemplate:
         metadata:
-          namespace: {{ .Release.Namespace }}
+          namespace: {{ $.Release.Namespace }}
+          labels:
+            {{- range $group.roles }}
+            node-role.kubernetes.io/{{ . }}: ""
+            {{- end }}
         spec:
           runStrategy: Always
           template:
@@ -103,48 +125,67 @@ spec:
               domain:
                 cpu:
                   threads: 1
-                  cores: 2
+                  cores: {{ $group.resources.cpu }}
                   sockets: 1
                 devices:
                   disks:
-                  - disk:
+                  - name: system
+                    disk:
                       bus: virtio
-                    name: containervolume
+                      pciAddress: 0000:07:00.0
+                  - name: containerd
+                    disk:
+                      bus: virtio
+                      pciAddress: 0000:08:00.0
+                  - name: kubelet
+                    disk:
+                      bus: virtio
+                      pciAddress: 0000:09:00.0
                   networkInterfaceMultiqueue: true
                 memory:
-                  guest: 1024Mi
+                  guest: {{ $group.resources.memory }}
               evictionStrategy: External
               volumes:
-              - containerDisk:
+              - name: system
+                containerDisk:
                   image: "{{ $.Files.Get "images/ubuntu-container-disk.tag" | trim }}@{{ index ($.Files.Get "images/ubuntu-container-disk.json" | fromJson) "containerimage.digest" }}"
-                name: containervolume
+              - name: containerd
+                emptyDisk:
+                  capacity: 20Gi
+              - name: kubelet
+                emptyDisk:
+                  capacity: 20Gi
 ---
 apiVersion: cluster.x-k8s.io/v1beta1
 kind: MachineDeployment
 metadata:
-  name: {{ .Release.Name }}-md-0
-  namespace: {{ .Release.Namespace }}
+  name: {{ $.Release.Name }}-{{ $groupName }}
+  namespace: {{ $.Release.Namespace }}
   annotations:
-    cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "2"
-    cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "0"
-    capacity.cluster-autoscaler.kubernetes.io/memory: "1024Mi"
-    capacity.cluster-autoscaler.kubernetes.io/cpu: "2"
+    cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "{{ $group.minReplicas }}"
+    cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "{{ $group.maxReplicas }}"
+    capacity.cluster-autoscaler.kubernetes.io/memory: "{{ $group.resources.memory }}"
+    capacity.cluster-autoscaler.kubernetes.io/cpu: "{{ $group.resources.cpu }}"
 spec:
-  clusterName: {{ .Release.Name }}
-  selector:
-    matchLabels: null
+  clusterName: {{ $.Release.Name }}
   template:
+    metadata:
+      labels:
+        {{- range $group.roles }}
+        node-role.kubernetes.io/{{ . }}: ""
+        {{- end }}
     spec:
       bootstrap:
         configRef:
           apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
           kind: KubeadmConfigTemplate
-          name: {{ .Release.Name }}-md-0
+          name: {{ $.Release.Name }}-{{ $groupName }}
           namespace: default
-      clusterName: {{ .Release.Name }}
+      clusterName: {{ $.Release.Name }}
       infrastructureRef:
         apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
         kind: KubevirtMachineTemplate
-        name: {{ .Release.Name }}-md-0
+        name: {{ $.Release.Name }}-{{ $groupName }}
         namespace: default
-      version: v1.23.10
+      version: v1.29.4
+{{- end }}

packages/apps/kubernetes/templates/csi/deploy.yaml

@@ -16,12 +16,10 @@ spec:
     spec:
       serviceAccountName: {{ .Release.Name }}-kcsi
       priorityClassName: system-cluster-critical
-      nodeSelector:
-        node-role.kubernetes.io/control-plane: ""
       tolerations:
         - key: CriticalAddonsOnly
           operator: Exists
-        - key: node-role.kubernetes.io/master
+        - key: node-role.kubernetes.io/control-plane
           operator: Exists
           effect: "NoSchedule"
       containers:

packages/apps/kubernetes/templates/helmreleases/cert-manager.yaml

@@ -0,0 +1,39 @@
+{{- if .Values.addons.certManager.enabled }}
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: {{ .Release.Name }}-cert-manager
+  labels:
+    cozystack.io/repository: system
+    coztstack.io/target-cluster-name: {{ .Release.Name }}
+spec:
+  interval: 5m
+  releaseName: cert-manager
+  chart:
+    spec:
+      chart: cozy-cert-manager
+      reconcileStrategy: Revision
+      sourceRef:
+        kind: HelmRepository
+        name: cozystack-system
+        namespace: cozy-system
+  kubeConfig:
+    secretRef:
+      name: {{ .Release.Name }}-kubeconfig
+  targetNamespace: cozy-cert-manager
+  storageNamespace: cozy-cert-manager
+  install:
+    createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
+  dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
+  - name: {{ .Release.Name }}
+    namespace: {{ .Release.Namespace }}
+  {{- end }}
+  - name: {{ .Release.Name }}-cilium
+    namespace: {{ .Release.Namespace }}
+{{- end }}

packages/apps/kubernetes/templates/helmreleases/cilium.yaml

@@ -1,4 +1,4 @@
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
+apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
   name: {{ .Release.Name }}-cilium
@@ -6,7 +6,7 @@ metadata:
     cozystack.io/repository: system
     coztstack.io/target-cluster-name: {{ .Release.Name }}
 spec:
-  interval: 1m
+  interval: 5m
   releaseName: cilium
   chart:
     spec:
@@ -23,10 +23,17 @@ spec:
   storageNamespace: cozy-cilium
   install:
     createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
   values:
     cilium:
       tunnel: disabled
-      autoDirectNodeRoutes: true
+      autoDirectNodeRoutes: false
+      bpf:
+        masquerade: true
       cgroup:
         autoMount:
           enabled: true
@@ -38,9 +45,11 @@ spec:
         chainingMode: ~
         customConf: false
         configMap: ""
-      routingMode: native
+      routingMode: tunnel
       enableIPv4Masquerade: true
-      ipv4NativeRoutingCIDR: "10.244.0.0/16"
+      ipv4NativeRoutingCIDR: ""
   dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
   - name: {{ .Release.Name }}
     namespace: {{ .Release.Namespace }}
+  {{- end }}

packages/apps/kubernetes/templates/helmreleases/csi.yaml

@@ -1,4 +1,4 @@
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
+apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
   name: {{ .Release.Name }}-csi
@@ -6,7 +6,7 @@ metadata:
     cozystack.io/repository: system
     coztstack.io/target-cluster-name: {{ .Release.Name }}
 spec:
-  interval: 1m
+  interval: 5m
   releaseName: csi
   chart:
     spec:
@@ -23,6 +23,13 @@ spec:
   storageNamespace: cozy-csi
   install:
     createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
   dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
   - name: {{ .Release.Name }}
     namespace: {{ .Release.Namespace }}
+  {{- end }}

packages/apps/kubernetes/templates/helmreleases/delete.yaml

@@ -12,19 +12,31 @@ spec:
     spec:
       serviceAccountName: {{ .Release.Name }}-flux-teardown
       restartPolicy: Never
+      tolerations:
+        - key: CriticalAddonsOnly
+          operator: Exists
+        - key: node-role.kubernetes.io/control-plane
+          operator: Exists
+          effect: "NoSchedule"
       containers:
         - name: kubectl
           image: docker.io/clastix/kubectl:v1.29.1
           command:
-            - kubectl
-            - --namespace={{ .Release.Namespace }}
-            - patch
-            - helmrelease
-            - {{ .Release.Name }}-cilium
-            - {{ .Release.Name }}-csi
-            - -p
-            - '{"spec": {"suspend": true}}'
-            - --type=merge
+            - /bin/sh
+            - -c
+            - |
+                kubectl
+                --namespace={{ .Release.Namespace }}
+                patch
+                helmrelease
+                {{ .Release.Name }}-cilium
+                {{ .Release.Name }}-csi
+                {{ .Release.Name }}-cert-manager
+                {{ .Release.Name }}-ingress-nginx
+                {{ .Release.Name }}-fluxcd-operator
+                {{ .Release.Name }}-fluxcd
+                -p '{"spec": {"suspend": true}}'
+                --type=merge --field-manager=flux-client-side-apply || true
 ---
 apiVersion: v1
 kind: ServiceAccount
@@ -54,6 +66,10 @@ rules:
     resourceNames:
       - {{ .Release.Name }}-cilium
       - {{ .Release.Name }}-csi
+      - {{ .Release.Name }}-cert-manager
+      - {{ .Release.Name }}-ingress-nginx
+      - {{ .Release.Name }}-fluxcd-operator
+      - {{ .Release.Name }}-fluxcd
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding

packages/apps/kubernetes/templates/helmreleases/fluxcd.yaml

@@ -0,0 +1,84 @@
+{{- if .Values.addons.fluxcd.enabled }}
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: {{ .Release.Name }}-fluxcd-operator
+  labels:
+    cozystack.io/repository: system
+    coztstack.io/target-cluster-name: {{ .Release.Name }}
+spec:
+  interval: 5m
+  releaseName: fluxcd-operator
+  chart:
+    spec:
+      chart: cozy-fluxcd-operator
+      reconcileStrategy: Revision
+      sourceRef:
+        kind: HelmRepository
+        name: cozystack-system
+        namespace: cozy-system
+  kubeConfig:
+    secretRef:
+      name: {{ .Release.Name }}-kubeconfig
+  targetNamespace: cozy-fluxcd
+  storageNamespace: cozy-fluxcd
+  install:
+    createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
+  values:
+    flux-operator:
+      fullnameOverride: flux-operator
+      tolerations: []
+      hostNetwork: false
+  dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
+  - name: {{ .Release.Name }}
+    namespace: {{ .Release.Namespace }}
+  {{- end }}
+  - name: {{ .Release.Name }}-cilium
+    namespace: {{ .Release.Namespace }}
+---
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: {{ .Release.Name }}-fluxcd
+  labels:
+    cozystack.io/repository: system
+    coztstack.io/target-cluster-name: {{ .Release.Name }}
+spec:
+  interval: 5m
+  releaseName: fluxcd
+  chart:
+    spec:
+      chart: cozy-fluxcd
+      reconcileStrategy: Revision
+      sourceRef:
+        kind: HelmRepository
+        name: cozystack-system
+        namespace: cozy-system
+  kubeConfig:
+    secretRef:
+      name: {{ .Release.Name }}-kubeconfig
+  targetNamespace: cozy-fluxcd
+  storageNamespace: cozy-fluxcd
+  install:
+    createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
+  dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
+  - name: {{ .Release.Name }}
+    namespace: {{ .Release.Namespace }}
+  {{- end }}
+  - name: {{ .Release.Name }}-cilium
+    namespace: {{ .Release.Namespace }}
+  - name: {{ .Release.Name }}-fluxcd-operator
+    namespace: {{ .Release.Namespace }}
+{{- end }}

packages/apps/kubernetes/templates/helmreleases/ingress-nginx.yaml

@@ -0,0 +1,49 @@
+{{- if .Values.addons.ingressNginx.enabled }}
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: {{ .Release.Name }}-ingress-nginx
+  labels:
+    cozystack.io/repository: system
+    coztstack.io/target-cluster-name: {{ .Release.Name }}
+spec:
+  interval: 5m
+  releaseName: ingress-nginx
+  chart:
+    spec:
+      chart: cozy-ingress-nginx
+      reconcileStrategy: Revision
+      sourceRef:
+        kind: HelmRepository
+        name: cozystack-system
+        namespace: cozy-system
+  kubeConfig:
+    secretRef:
+      name: {{ .Release.Name }}-kubeconfig
+  targetNamespace: cozy-ingress-nginx
+  storageNamespace: cozy-ingress-nginx
+  install:
+    createNamespace: true
+    remediation:
+      retries: -1
+  upgrade:
+    remediation:
+      retries: -1
+  values:
+    ingress-nginx:
+      fullnameOverride: ingress-nginx
+      controller:
+        kind: DaemonSet
+        hostNetwork: true
+        service:
+          enabled: false
+      nodeSelector:
+        node-role.kubernetes.io/ingress-nginx: ""
+  dependsOn:
+  {{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
+  - name: {{ .Release.Name }}
+    namespace: {{ .Release.Namespace }}
+  {{- end }}
+  - name: {{ .Release.Name }}-cilium
+    namespace: {{ .Release.Namespace }}
+{{- end }}

packages/apps/kubernetes/templates/ingress.yaml

@@ -0,0 +1,45 @@
+{{- $myNS := lookup "v1" "Namespace" "" .Release.Namespace }}
+{{- $ingress := index $myNS.metadata.annotations "namespace.cozystack.io/ingress" }}
+{{- if .Values.addons.ingressNginx.hosts }}
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: {{ .Release.Name }}-ingress-nginx
+  annotations:
+    nginx.ingress.kubernetes.io/ssl-redirect: "false"
+    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
+spec:
+  ingressClassName: "{{ $ingress }}"
+  rules:
+  {{- range .Values.addons.ingressNginx.hosts }}
+  - host: {{ . | quote }}
+    http:
+      paths:
+      - path: /
+        pathType: ImplementationSpecific
+        backend:
+          service:
+            name: {{ $.Release.Name }}-ingress-nginx
+            port:
+              number: 443
+  {{- end }}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ .Release.Name }}-ingress-nginx
+spec:
+  ports:
+  - appProtocol: http
+    name: http
+    port: 80
+    targetPort: 80
+  - appProtocol: https
+    name: https
+    port: 443
+    targetPort: 443
+  selector:
+    cluster.x-k8s.io/cluster-name: {{ .Release.Name }}
+    node-role.kubernetes.io/ingress-nginx: ""
+{{- end }}

packages/apps/kubernetes/templates/kccm/manager.yaml

@@ -14,6 +14,12 @@ spec:
       labels:
         k8s-app: {{ .Release.Name }}-kccm
     spec:
+      tolerations:
+        - key: CriticalAddonsOnly
+          operator: Exists
+        - key: node-role.kubernetes.io/control-plane
+          operator: Exists
+          effect: "NoSchedule"
       containers:
       - name: kubevirt-cloud-controller-manager
         args:
@@ -44,6 +50,4 @@ spec:
       - secret:
           secretName: {{ .Release.Name }}-admin-kubeconfig
         name: kubeconfig
-      tolerations:
-      - operator: Exists
       serviceAccountName: {{ .Release.Name }}-kccm

packages/apps/kubernetes/values.schema.json

@@ -1,11 +1,62 @@
 {
-  "$schema": "http://json-schema.org/schema#",
-  "type": "object",
-  "properties": {
-    "host": {
-      "type": "string",
-      "title": "Domain name for this kubernetes cluster",
-      "description": "This host will be used for all apps deployed in this tenant"
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "host": {
+            "type": "string",
+            "description": "The hostname used to access the Kubernetes cluster externally (defaults to using the cluster name as a subdomain for the tenant host).",
+            "default": ""
+        },
+        "controlPlane": {
+            "type": "object",
+            "properties": {
+                "replicas": {
+                    "type": "number",
+                    "description": "Number of replicas for Kubernetes contorl-plane components",
+                    "default": 2
+                }
+            }
+        },
+        "addons": {
+            "type": "object",
+            "properties": {
+                "certManager": {
+                    "type": "object",
+                    "properties": {
+                        "enabled": {
+                            "type": "boolean",
+                            "description": "Enables the cert-manager",
+                            "default": false
+                        }
+                    }
+                },
+                "ingressNginx": {
+                    "type": "object",
+                    "properties": {
+                        "enabled": {
+                            "type": "boolean",
+                            "description": "Enable Ingress-NGINX controller (expect nodes with 'ingress-nginx' role)",
+                            "default": false
+                        },
+                        "hosts": {
+                            "type": "array",
+                            "description": "List of domain names that should be passed through to the cluster by upper cluster",
+                            "default": [],
+                            "items": {}
+                        }
+                    }
+                },
+                "fluxcd": {
+                    "type": "object",
+                    "properties": {
+                        "enabled": {
+                            "type": "boolean",
+                            "description": "Enables Flux CD",
+                            "default": false
+                        }
+                    }
+                }
+            }
+        }
     }
-  }
-}
+}

packages/apps/kubernetes/values.yaml

@@ -1 +1,51 @@
+## @section Common parameters
+
+## @param host The hostname used to access the Kubernetes cluster externally (defaults to using the cluster name as a subdomain for the tenant host).
+## @param controlPlane.replicas Number of replicas for Kubernetes contorl-plane components
+##
 host: ""
+controlPlane:
+  replicas: 2
+
+## @param nodeGroups [object] nodeGroups configuration
+##
+nodeGroups:
+  md0:
+    minReplicas: 0
+    maxReplicas: 10
+    resources:
+      cpu: 2
+      memory: 1024Mi
+    roles:
+    - ingress-nginx
+
+## @section Cluster Addons
+##
+addons:
+
+  ## Cert-manager: automatically creates and manages SSL/TLS certificate
+  ##
+  certManager:
+    ## @param addons.certManager.enabled Enables the cert-manager
+    enabled: false
+
+  ## Ingress-NGINX Controller
+  ##
+  ingressNginx:
+    ## @param addons.ingressNginx.enabled Enable Ingress-NGINX controller (expect nodes with 'ingress-nginx' role)
+    ##
+    enabled: false
+    ## @param addons.ingressNginx.hosts List of domain names that should be passed through to the cluster by upper cluster
+    ## e.g:
+    ## hosts:
+    ## - example.org
+    ## - foo.example.net
+    ##
+    hosts: []
+
+  ## Flux CD
+  ##
+  fluxcd:
+    ## @param addons.fluxcd.enabled Enables Flux CD
+    ##
+    enabled: false

packages/apps/mysql/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.3.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "11.0.2"

packages/apps/mysql/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/mysql/README.md

@@ -62,3 +62,34 @@ more details:
   mysqldump -h <slave> -P 3306 -u<user> -p<password> --column-statistics=0 <database> <table> ~/tmp/fix-table.sql
   mysql -h <master> -P 3306 -u<user> -p<password> <database> < ~/tmp/fix-table.sql
   ```
+
+## Parameters
+
+### Common parameters
+
+| Name       | Description                                     | Value   |
+| ---------- | ----------------------------------------------- | ------- |
+| `external` | Enable external access from outside the cluster | `false` |
+| `size`     | Persistent Volume size                          | `10Gi`  |
+| `replicas` | Number of MariaDB replicas                      | `2`     |
+
+### Configuration parameters
+
+| Name        | Description             | Value |
+| ----------- | ----------------------- | ----- |
+| `users`     | Users configuration     | `{}`  |
+| `databases` | Databases configuration | `[]`  |
+
+### Backup parameters
+
+| Name                     | Description                                    | Value                                                  |
+| ------------------------ | ---------------------------------------------- | ------------------------------------------------------ |
+| `backup.enabled`         | Enable pereiodic backups                       | `false`                                                |
+| `backup.s3Region`        | The AWS S3 region where backups are stored     | `us-east-1`                                            |
+| `backup.s3Bucket`        | The S3 bucket used for storing backups         | `s3.example.org/postgres-backups`                      |
+| `backup.schedule`        | Cron schedule for automated backups            | `0 2 * * *`                                            |
+| `backup.cleanupStrategy` | The strategy for cleaning up old backups       | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
+| `backup.s3AccessKey`     | The access key for S3, used for authentication | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
+| `backup.s3SecretKey`     | The secret key for S3, used for authentication | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
+| `backup.resticPassword`  | The password for Restic backup encryption      | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
+

packages/apps/mysql/templates/db.yaml

@@ -1,7 +1,7 @@
 {{- range $name := .Values.databases }}
 {{ $dnsName := replace "_" "-" $name }}
 ---
-apiVersion: mariadb.mmontes.io/v1alpha1
+apiVersion: k8s.mariadb.com/v1alpha1
 kind: Database
 metadata:
   name: {{ $.Release.Name }}-{{ $dnsName }}

packages/apps/mysql/templates/mariadb.yaml

@@ -1,18 +1,20 @@
 ---
-apiVersion: mariadb.mmontes.io/v1alpha1
+apiVersion: k8s.mariadb.com/v1alpha1
 kind: MariaDB
 metadata:
   name: {{ .Release.Name }}
 spec:
+  {{- if (and .Values.users.root .Values.users.root.password) }}
   rootPasswordSecretKeyRef:
     name: {{ .Release.Name }}
     key: root-password
+  {{- end }}
 
   image: "mariadb:11.0.2"
 
   port: 3306
 
-  replicas: 2
+  replicas: {{ .Values.replicas }}
   affinity:
     podAntiAffinity:
       requiredDuringSchedulingIgnoredDuringExecution:
@@ -28,15 +30,18 @@ spec:
             - {{ .Release.Name }}
         topologyKey: "kubernetes.io/hostname"
 
+  {{- if gt (int .Values.replicas) 1 }}
   replication:
     enabled: true
     #primary:
     #  podIndex: 0
     #  automaticFailover: true
+  {{- end }}
 
   metrics:
+    enabled: true
     exporter:
-      image: prom/mysqld-exporter:v0.14.0
+      image: prom/mysqld-exporter:v0.15.1
       resources:
         requests:
           cpu: 50m
@@ -53,14 +58,10 @@ spec:
     name: {{ .Release.Name }}-my-cnf
     key: config
 
-  volumeClaimTemplate:
-    resources:
-      requests:
-        storage: {{ .Values.size }}
-    accessModes:
-      - ReadWriteOnce
-
-
+  storage:
+    size: {{ .Values.size }}
+    resizeInUseVolumes: true
+    waitForVolumeResize: true
 
   {{- if .Values.external }}
   primaryService:

packages/apps/mysql/templates/user.yaml

@@ -2,7 +2,7 @@
 {{ if not (eq $name "root") }}
 {{ $dnsName := replace "_" "-" $name }}
 ---
-apiVersion: mariadb.mmontes.io/v1alpha1
+apiVersion: k8s.mariadb.com/v1alpha1
 kind: User
 metadata:
   name: {{ $.Release.Name }}-{{ $dnsName }}
@@ -15,7 +15,7 @@ spec:
     key: {{ $name }}-password
   maxUserConnections: {{ $u.maxUserConnections }}
 ---
-apiVersion: mariadb.mmontes.io/v1alpha1
+apiVersion: k8s.mariadb.com/v1alpha1
 kind: Grant
 metadata:
   name: {{ $.Release.Name }}-{{ $dnsName }}

packages/apps/mysql/values.schema.json

@@ -0,0 +1,72 @@
+{
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "size": {
+            "type": "string",
+            "description": "Persistent Volume size",
+            "default": "10Gi"
+        },
+        "replicas": {
+            "type": "number",
+            "description": "Number of MariaDB replicas",
+            "default": 2
+        },
+        "databases": {
+            "type": "array",
+            "description": "Databases configuration",
+            "default": [],
+            "items": {}
+        },
+        "backup": {
+            "type": "object",
+            "properties": {
+                "enabled": {
+                    "type": "boolean",
+                    "description": "Enable pereiodic backups",
+                    "default": false
+                },
+                "s3Region": {
+                    "type": "string",
+                    "description": "The AWS S3 region where backups are stored",
+                    "default": "us-east-1"
+                },
+                "s3Bucket": {
+                    "type": "string",
+                    "description": "The S3 bucket used for storing backups",
+                    "default": "s3.example.org/postgres-backups"
+                },
+                "schedule": {
+                    "type": "string",
+                    "description": "Cron schedule for automated backups",
+                    "default": "0 2 * * *"
+                },
+                "cleanupStrategy": {
+                    "type": "string",
+                    "description": "The strategy for cleaning up old backups",
+                    "default": "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+                },
+                "s3AccessKey": {
+                    "type": "string",
+                    "description": "The access key for S3, used for authentication",
+                    "default": "oobaiRus9pah8PhohL1ThaeTa4UVa7gu"
+                },
+                "s3SecretKey": {
+                    "type": "string",
+                    "description": "The secret key for S3, used for authentication",
+                    "default": "ju3eum4dekeich9ahM1te8waeGai0oog"
+                },
+                "resticPassword": {
+                    "type": "string",
+                    "description": "The password for Restic backup encryption",
+                    "default": "ChaXoveekoh6eigh4siesheeda2quai0"
+                }
+            }
+        }
+    }
+}

packages/apps/mysql/values.yaml

@@ -1,24 +1,50 @@
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param size Persistent Volume size
+## @param replicas Number of MariaDB replicas
+##
 external: false
 size: 10Gi
+replicas: 2
 
-users:
-  root:
-    password: strongpassword
-  user1:
-    privileges: ['ALL']
-    maxUserConnections: 1000
-    password: hackme
-  user2:
-    privileges: ['SELECT']
-    maxUserConnections: 1000
-    password: hackme
+## @section Configuration parameters
 
-databases:
-- wordpress1
-- wordpress2
-- wordpress3
-- wordpress4
+## @param users [object] Users configuration
+## Example:
+## users:
+##   root:
+##     password: strongpassword
+##   user1:
+##     privileges: ['ALL']
+##     maxUserConnections: 1000
+##     password: hackme
+##   user2:
+##     privileges: ['SELECT']
+##     maxUserConnections: 1000
+##     password: hackme
+##
+users: {}
 
+## @param databases Databases configuration
+## Example:
+## databases:
+## - wordpress1
+## - wordpress2
+## - wordpress3
+## - wordpress4
+databases: []
+
+## @section Backup parameters
+
+## @param backup.enabled Enable pereiodic backups
+## @param backup.s3Region The AWS S3 region where backups are stored
+## @param backup.s3Bucket The S3 bucket used for storing backups
+## @param backup.schedule Cron schedule for automated backups
+## @param backup.cleanupStrategy The strategy for cleaning up old backups
+## @param backup.s3AccessKey The access key for S3, used for authentication
+## @param backup.s3SecretKey The secret key for S3, used for authentication
+## @param backup.resticPassword The password for Restic backup encryption
 backup:
   enabled: false
   s3Region: us-east-1

packages/apps/postgres/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.3.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "16.2"

packages/apps/postgres/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/postgres/README.md

@@ -30,3 +30,37 @@ restic -r s3:s3.example.org/postgres-backups/database_name restore latest --targ
 
 more details:
 - https://itnext.io/restic-effective-backup-from-stdin-4bc1e8f083c1
+
+## Parameters
+
+### Common parameters
+
+| Name                     | Description                                                                                                             | Value   |
+| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | ------- |
+| `external`               | Enable external access from outside the cluster                                                                         | `false` |
+| `size`                   | Persistent Volume size                                                                                                  | `10Gi`  |
+| `replicas`               | Number of Postgres replicas                                                                                             | `2`     |
+| `quorum.minSyncReplicas` | Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed.           | `0`     |
+| `quorum.maxSyncReplicas` | Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the number of instances). | `0`     |
+
+### Configuration parameters
+
+| Name        | Description             | Value |
+| ----------- | ----------------------- | ----- |
+| `users`     | Users configuration     | `{}`  |
+| `databases` | Databases configuration | `{}`  |
+
+### Backup parameters
+
+| Name                     | Description                                    | Value                                                  |
+| ------------------------ | ---------------------------------------------- | ------------------------------------------------------ |
+| `backup.enabled`         | Enable pereiodic backups                       | `false`                                                |
+| `backup.s3Region`        | The AWS S3 region where backups are stored     | `us-east-1`                                            |
+| `backup.s3Bucket`        | The S3 bucket used for storing backups         | `s3.example.org/postgres-backups`                      |
+| `backup.schedule`        | Cron schedule for automated backups            | `0 2 * * *`                                            |
+| `backup.cleanupStrategy` | The strategy for cleaning up old backups       | `--keep-last=3 --keep-daily=3 --keep-within-weekly=1m` |
+| `backup.s3AccessKey`     | The access key for S3, used for authentication | `oobaiRus9pah8PhohL1ThaeTa4UVa7gu`                     |
+| `backup.s3SecretKey`     | The secret key for S3, used for authentication | `ju3eum4dekeich9ahM1te8waeGai0oog`                     |
+| `backup.resticPassword`  | The password for Restic backup encryption      | `ChaXoveekoh6eigh4siesheeda2quai0`                     |
+
+

packages/apps/postgres/templates/db.yaml

@@ -4,13 +4,16 @@ kind: Cluster
 metadata:
   name: {{ .Release.Name }}
 spec:
-  instances: 2
+  instances: {{ .Values.replicas }}
   enableSuperuserAccess: true
 
   postgresql:
     parameters:
       max_wal_senders: "30"
 
+  minSyncReplicas: {{ .Values.quorum.minSyncReplicas }}
+  maxSyncReplicas: {{ .Values.quorum.maxSyncReplicas }}
+
   monitoring:
     enablePodMonitor: true
 

packages/apps/postgres/templates/init-script.yaml

@@ -100,13 +100,13 @@ stringData:
     echo "== assign roles to users"
     psql -v ON_ERROR_STOP=1 --echo-all <<\EOT
     {{- range $database, $d := .Values.databases }}
-    {{- range $user, $u := $.Values.roles }}
-    {{- if has $user $d.users.admin }}
+    {{- range $user, $u := $.Values.users }}
+    {{- if has $user $d.roles.admin }}
     GRANT {{ $database }}_admin TO {{ $user }};
     {{- else }}
     REVOKE {{ $database }}_admin FROM {{ $user }};
     {{- end }}
-    {{- if has $user $d.users.readonly }}
+    {{- if has $user $d.roles.readonly }}
     GRANT {{ $database }}_readonly TO {{ $user }};
     {{- else }}
     REVOKE {{ $database }}_readonly FROM {{ $user }};

packages/apps/postgres/values.schema.json

@@ -0,0 +1,86 @@
+{
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "size": {
+            "type": "string",
+            "description": "Persistent Volume size",
+            "default": "10Gi"
+        },
+        "replicas": {
+            "type": "number",
+            "description": "Number of Postgres replicas",
+            "default": 2
+        },
+        "quorum": {
+            "type": "object",
+            "properties": {
+                "minSyncReplicas": {
+                    "type": "number",
+                    "description": "Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed.",
+                    "default": 0
+                },
+                "maxSyncReplicas": {
+                    "type": "number",
+                    "description": "Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the number of instances).",
+                    "default": 0
+                }
+            }
+        },
+        "databases": {
+            "type": "object",
+            "description": "Databases configuration",
+            "default": {}
+        },
+        "backup": {
+            "type": "object",
+            "properties": {
+                "enabled": {
+                    "type": "boolean",
+                    "description": "Enable pereiodic backups",
+                    "default": false
+                },
+                "s3Region": {
+                    "type": "string",
+                    "description": "The AWS S3 region where backups are stored",
+                    "default": "us-east-1"
+                },
+                "s3Bucket": {
+                    "type": "string",
+                    "description": "The S3 bucket used for storing backups",
+                    "default": "s3.example.org/postgres-backups"
+                },
+                "schedule": {
+                    "type": "string",
+                    "description": "Cron schedule for automated backups",
+                    "default": "0 2 * * *"
+                },
+                "cleanupStrategy": {
+                    "type": "string",
+                    "description": "The strategy for cleaning up old backups",
+                    "default": "--keep-last=3 --keep-daily=3 --keep-within-weekly=1m"
+                },
+                "s3AccessKey": {
+                    "type": "string",
+                    "description": "The access key for S3, used for authentication",
+                    "default": "oobaiRus9pah8PhohL1ThaeTa4UVa7gu"
+                },
+                "s3SecretKey": {
+                    "type": "string",
+                    "description": "The secret key for S3, used for authentication",
+                    "default": "ju3eum4dekeich9ahM1te8waeGai0oog"
+                },
+                "resticPassword": {
+                    "type": "string",
+                    "description": "The password for Restic backup encryption",
+                    "default": "ChaXoveekoh6eigh4siesheeda2quai0"
+                }
+            }
+        }
+    }
+}

packages/apps/postgres/values.yaml

@@ -1,33 +1,64 @@
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param size Persistent Volume size
+## @param replicas Number of Postgres replicas
+##
 external: false
 size: 10Gi
+replicas: 2
 
-users:
-  user1:
-    password: strongpassword
-  user2:
-    password: hackme
-  airflow:
-    password: qwerty123
-  debezium:
-    replication: true
+## Configuration for the quorum-based synchronous replication
+## @param quorum.minSyncReplicas Minimum number of synchronous replicas that must acknowledge a transaction before it is considered committed.
+## @param quorum.maxSyncReplicas Maximum number of synchronous replicas that can acknowledge a transaction (must be lower than the number of instances).
+quorum:
+  minSyncReplicas: 0
+  maxSyncReplicas: 0
 
-databases:
+## @section Configuration parameters
 
-  myapp:
-    roles:
-      admin:
-      - user1
-      - debezium
-      readonly:
-      - user2
+## @param users [object] Users configuration
+## Example:
+## users:
+##   user1:
+##     password: strongpassword
+##   user2:
+##     password: hackme
+##   airflow:
+##     password: qwerty123
+##   debezium:
+##     replication: true
+##
+users: {}
 
-  airflow:
-    roles:
-      admin:
-      - airflow
-    extensions:
-    - hstore
+## @param databases Databases configuration
+## Example:
+## databases:
+##   myapp:
+##     roles:
+##       admin:
+##       - user1
+##       - debezium
+##       readonly:
+##       - user2
+##   airflow:
+##     roles:
+##       admin:
+##       - airflow
+##     extensions:
+##     - hstore
+databases: {}
 
+## @section Backup parameters
+
+## @param backup.enabled Enable pereiodic backups
+## @param backup.s3Region The AWS S3 region where backups are stored
+## @param backup.s3Bucket The S3 bucket used for storing backups
+## @param backup.schedule Cron schedule for automated backups
+## @param backup.cleanupStrategy The strategy for cleaning up old backups
+## @param backup.s3AccessKey The access key for S3, used for authentication
+## @param backup.s3SecretKey The secret key for S3, used for authentication
+## @param backup.resticPassword The password for Restic backup encryption
 backup:
   enabled: false
   s3Region: us-east-1

packages/apps/rabbitmq/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.2.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "3.12.2"

packages/apps/rabbitmq/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/rabbitmq/README.md

@@ -8,3 +8,12 @@ The service utilizes official RabbitMQ operator. This ensures the reliability an
 
 - Github: https://github.com/rabbitmq/cluster-operator/
 - Docs: https://www.rabbitmq.com/kubernetes/operator/operator-overview.html
+
+## Parameters
+
+### Common parameters
+
+| Name       | Description                                     | Value   |
+| ---------- | ----------------------------------------------- | ------- |
+| `external` | Enable external access from outside the cluster | `false` |
+| `replicas` | Number of RabbitMQ replicas                     | `3`     |

packages/apps/rabbitmq/templates/rabbitmq.yaml

@@ -6,7 +6,7 @@ metadata:
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/managed-by: {{ .Release.Service }}
 spec:
-  replicas: 3
+  replicas: {{ .Values.replicas }}
   {{- if .Values.external }}
   service:
     type: LoadBalancer

packages/apps/rabbitmq/values.schema.json

@@ -1,10 +1,16 @@
 {
-  "$schema": "http://json-schema.org/schema#",
-  "type": "object",
-  "properties": {
-    "external": {
-      "type": "boolean",
-      "title": "Enable external Access"
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "replicas": {
+            "type": "number",
+            "description": "Number of RabbitMQ replicas",
+            "default": 3
+        }
     }
-  }
-}
+}

packages/apps/rabbitmq/values.yaml

@@ -1 +1,7 @@
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param replicas Number of RabbitMQ replicas
+##
 external: false
+replicas: 3

packages/apps/redis/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.1
+version: 0.2.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "6.2.6"

packages/apps/redis/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/redis/README.md

@@ -8,3 +8,15 @@ Service utilizes the Spotahome Redis Operator for efficient management and orche
 
 - Docs: https://redis.io/docs/
 - GitHub: https://github.com/spotahome/redis-operator
+
+## Parameters
+
+### Common parameters
+
+| Name       | Description                                     | Value   |
+| ---------- | ----------------------------------------------- | ------- |
+| `external` | Enable external access from outside the cluster | `false` |
+| `size`     | Persistent Volume size                          | `1Gi`   |
+| `replicas` | Number of Redis replicas                        | `2`     |
+
+

packages/apps/redis/templates/redisfailover.yaml

@@ -14,7 +14,7 @@ spec:
       limits:
         memory: 100Mi
   redis:
-    replicas: 3
+    replicas: {{ .Values.replicas }}
     resources:
       requests:
         cpu: 150m

packages/apps/redis/values.schema.json

@@ -1,14 +1,21 @@
 {
-  "$schema": "http://json-schema.org/schema#",
-  "type": "object",
-  "properties": {
-    "external": {
-      "type": "boolean",
-      "title": "Enable external Access"
-    },
-    "size": {
-      "type": "string",
-      "title": "Disk Size"
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "size": {
+            "type": "string",
+            "description": "Persistent Volume size",
+            "default": "1Gi"
+        },
+        "replicas": {
+            "type": "number",
+            "description": "Number of Redis replicas",
+            "default": 2
+        }
     }
-  }
-}
+}

packages/apps/redis/values.yaml

@@ -1,2 +1,9 @@
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param size Persistent Volume size
+## @param replicas Number of Redis replicas
+##
 external: false
-size: 5Gi
+size: 1Gi
+replicas: 2

packages/apps/tcp-balancer/Chart.yaml

@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.2.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "2.9.7"

packages/apps/tcp-balancer/Makefile

@@ -0,0 +1,4 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json.tmp -r README.md
+	cat values.schema.json.tmp | jq '.properties.httpAndHttps.properties.mode.enum = ["tcp","tcp-with-proxy"]' > values.schema.json
+	rm -f values.schema.json.tmp

packages/apps/tcp-balancer/README.md

@@ -7,3 +7,23 @@ The Managed TCP Load Balancer Service simplifies the deployment and management o
 Managed TCP Load Balancer Service efficiently utilizes HAProxy for load balancing purposes. HAProxy is a well-established and reliable solution for distributing incoming TCP traffic across multiple backend servers, ensuring high availability and efficient resource utilization. This deployment choice guarantees the seamless and dependable operation of your load balancing infrastructure.
 
 - Docs: https://www.haproxy.com/documentation/
+
+## Parameters
+
+### Common parameters
+
+| Name       | Description                                     | Value   |
+| ---------- | ----------------------------------------------- | ------- |
+| `external` | Enable external access from outside the cluster | `false` |
+| `replicas` | Number of HAProxy replicas                      | `2`     |
+
+### Configuration parameters
+
+| Name                             | Description                                                   | Value   |
+| -------------------------------- | ------------------------------------------------------------- | ------- |
+| `httpAndHttps.mode`              | Mode for balancer. Allowed values: `tcp` and `tcp-with-proxy` | `tcp`   |
+| `httpAndHttps.targetPorts.http`  | HTTP port number.                                             | `80`    |
+| `httpAndHttps.targetPorts.https` | HTTPS port number.                                            | `443`   |
+| `httpAndHttps.endpoints`         | Endpoint addresses list                                       | `[]`    |
+| `whitelistHTTP`                  | Secure HTTP by enabling  client networks whitelisting         | `false` |
+| `whitelist`                      | List of client networks                                       | `[]`    |

packages/apps/tcp-balancer/templates/deployment.yaml

@@ -7,7 +7,7 @@ metadata:
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/managed-by: {{ .Release.Service }}
 spec:
-  replicas: 2
+  replicas: {{ .Values.replicas }}
   selector:
     matchLabels:
       app: {{ .Release.Name }}-haproxy

packages/apps/tcp-balancer/values.schema.json

@@ -0,0 +1,62 @@
+{
+  "title": "Chart Values",
+  "type": "object",
+  "properties": {
+    "external": {
+      "type": "boolean",
+      "description": "Enable external access from outside the cluster",
+      "default": false
+    },
+    "replicas": {
+      "type": "number",
+      "description": "Number of HAProxy replicas",
+      "default": 2
+    },
+    "httpAndHttps": {
+      "type": "object",
+      "properties": {
+        "mode": {
+          "type": "string",
+          "description": "Mode for balancer. Allowed values: `tcp` and `tcp-with-proxy`",
+          "default": "tcp",
+          "enum": [
+            "tcp",
+            "tcp-with-proxy"
+          ]
+        },
+        "targetPorts": {
+          "type": "object",
+          "properties": {
+            "http": {
+              "type": "number",
+              "description": "HTTP port number.",
+              "default": 80
+            },
+            "https": {
+              "type": "number",
+              "description": "HTTPS port number.",
+              "default": 443
+            }
+          }
+        },
+        "endpoints": {
+          "type": "array",
+          "description": "Endpoint addresses list",
+          "default": [],
+          "items": {}
+        }
+      }
+    },
+    "whitelistHTTP": {
+      "type": "boolean",
+      "description": "Secure HTTP by enabling  client networks whitelisting",
+      "default": false
+    },
+    "whitelist": {
+      "type": "array",
+      "description": "List of client networks",
+      "default": [],
+      "items": {}
+    }
+  }
+}

packages/apps/tcp-balancer/values.yaml

@@ -1,18 +1,45 @@
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param replicas Number of HAProxy replicas
+##
 external: false
+replicas: 2
+
+## @section Configuration parameters
+
+## @param httpAndHttps.mode Mode for balancer. Allowed values: `tcp` and `tcp-with-proxy`
+## @param httpAndHttps.targetPorts.http HTTP port number.
+## @param httpAndHttps.targetPorts.https HTTPS port number.
+## @param httpAndHttps.endpoints Endpoint addresses list
+## Example:
+## httpAndHttps:
+##   mode: tcp
+##   targetPorts:
+##     http: 80
+##     https: 443
+##   endpoints:
+##    - 10.100.3.1
+##    - 10.100.3.11
+##    - 10.100.3.2
+##    - 10.100.3.12
+##    - 10.100.3.3
+##    - 10.100.3.13
+
 httpAndHttps:
   mode: tcp
   targetPorts:
     http: 80
     https: 443
-  endpoints:
-   - 10.100.3.1
-   - 10.100.3.11
-   - 10.100.3.2
-   - 10.100.3.12
-   - 10.100.3.3
-   - 10.100.3.13
+  endpoints: []
 
+## @param whitelistHTTP Secure HTTP by enabling  client networks whitelisting
+## @param whitelist List of client networks
+## Example:
+## whitelistHTTP: true
+## whitelist:
+## - "1.2.3.4"
+## - "10.100.0.0/16"
+##
 whitelistHTTP: false
-whitelist:
-- "1.2.3.4"
-- "10.100.0.0/16"
+whitelist: []

packages/apps/tenant/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 name: tenant
-description: Additional tenant namespace
+description: Separated tenant namespace
 icon: https://upload.wikimedia.org/wikipedia/commons/0/04/User_icon_1.svg
 
 type: application
-version: 1.0.0
+version: 1.2.0

packages/apps/tenant/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/tenant/README.md

@@ -25,7 +25,7 @@ tenant-root (example.org)
 
 Thus, you can create `tenant-u1` with a set of services like `etcd`, `ingress`, `monitoring`. And create another tenant namespace `tenant-u2` inside of `tenant-u1`.
 
-Let's see what will happen when you run Kubernetes and Postgres under `tenant-u2` namesapce.
+Let's see what will happen when you run Kubernetes and Postgres under `tenant-u2` namespace.
 
 Since `tenant-u2` does not have its own cluster services like `etcd`, `ingress`, and `monitoring`, the applications will use the cluster services of the parent tenant.  
 This in turn means:
@@ -45,3 +45,14 @@ tenant-u1
     ├── kubernetes-cluster1
     └── postgres-db1
 ```
+
+## Parameters
+
+### Common parameters
+
+| Name         | Description                                                                                                                 | Value   |
+| ------------ | --------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `host`       | The hostname used to access tenant services (defaults to using the tenant name as a subdomain for it's parent tenant host). | `""`    |
+| `etcd`       | Deploy own Etcd cluster                                                                                                     | `false` |
+| `monitoring` | Deploy own Monitoring Stack                                                                                                 | `false` |
+| `ingress`    | Deploy own Ingress Controller                                                                                               | `false` |

packages/apps/tenant/templates/etcd.yaml

@@ -1,5 +1,5 @@
 {{- if .Values.etcd }}
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
+apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
   name: etcd

packages/apps/tenant/templates/ingress.yaml

@@ -1,5 +1,5 @@
 {{- if .Values.ingress }}
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
+apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
   name: ingress

packages/apps/tenant/templates/monitoring.yaml

@@ -1,5 +1,5 @@
 {{- if .Values.monitoring }}
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
+apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
   name: monitoring

packages/apps/tenant/values.schema.json

@@ -1,27 +1,26 @@
 {
-  "$schema": "http://json-schema.org/schema#",
-  "type": "object",
-  "properties": {
-    "host": {
-      "type": "string",
-      "form": true,
-      "title": "Domain name for this tenant",
-      "description": "This host will be used for all apps deployed in this tenant"
-    },
-    "etcd": {
-      "type": "boolean",
-      "title": "Deploy own Etcd cluster",
-      "form": true
-    },
-    "monitoring": {
-      "type": "boolean",
-      "title": "Deploy own Monitoring Stack",
-      "form": true
-    },
-    "ingress": {
-      "type": "boolean",
-      "title": "Deploy own Ingress Controller",
-      "form": true
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "host": {
+            "type": "string",
+            "description": "The hostname used to access tenant services (defaults to using the tenant name as a subdomain for it's parent tenant host).",
+            "default": ""
+        },
+        "etcd": {
+            "type": "boolean",
+            "description": "Deploy own Etcd cluster",
+            "default": false
+        },
+        "monitoring": {
+            "type": "boolean",
+            "description": "Deploy own Monitoring Stack",
+            "default": false
+        },
+        "ingress": {
+            "type": "boolean",
+            "description": "Deploy own Ingress Controller",
+            "default": false
+        }
     }
-  }
-}
+}

packages/apps/tenant/values.yaml

@@ -1,3 +1,9 @@
+## @section Common parameters
+
+## @param host The hostname used to access tenant services (defaults to using the tenant name as a subdomain for it's parent tenant host).
+## @param etcd Deploy own Etcd cluster
+## @param monitoring Deploy own Monitoring Stack
+## @param ingress Deploy own Ingress Controller
 host: ""
 etcd: false
 monitoring: false

packages/apps/versions_map

@@ -1,14 +1,37 @@
-http-cache 0.1.0 HEAD
-kubernetes 0.1.0 HEAD
-mysql 0.1.0 HEAD
-postgres 0.1.0 HEAD
-rabbitmq 0.1.0 HEAD
-redis 0.1.1 HEAD
-tcp-balancer 0.1.0 HEAD
+clickhouse 0.1.0 ca79f72
+clickhouse 0.2.0 7cd7de73
+clickhouse 0.2.1 HEAD
+http-cache 0.1.0 a956713
+http-cache 0.2.0 HEAD
+kafka 0.1.0 760f86d2
+kafka 0.2.0 a2cc83d
+kafka 0.2.1 HEAD
+kubernetes 0.1.0 f642698
+kubernetes 0.2.0 7cd7de73
+kubernetes 0.3.0 7caccec1
+kubernetes 0.4.0 6cae6ce8
+kubernetes 0.5.0 HEAD
+mysql 0.1.0 f642698
+mysql 0.2.0 8b975ff0
+mysql 0.3.0 HEAD
+postgres 0.1.0 f642698
+postgres 0.2.0 7cd7de73
+postgres 0.2.1 4a97e297
+postgres 0.3.0 HEAD
+rabbitmq 0.1.0 f642698
+rabbitmq 0.2.0 HEAD
+redis 0.1.1 f642698
+redis 0.2.0 HEAD
+tcp-balancer 0.1.0 f642698
+tcp-balancer 0.2.0 HEAD
 tenant 0.1.3 3d1b86c
 tenant 0.1.4 d200480
 tenant 0.1.5 e3ab858
-tenant 1.0.0 HEAD
+tenant 1.0.0 7cd7de7
+tenant 1.1.0 4da8ac3b
+tenant 1.2.0 HEAD
 virtual-machine 0.1.4 f2015d6
-virtual-machine 0.1.5 HEAD
-vpn 0.1.0 HEAD
+virtual-machine 0.1.5 7cd7de7
+virtual-machine 0.2.0 HEAD
+vpn 0.1.0 f642698
+vpn 0.2.0 HEAD

packages/apps/virtual-machine/Chart.yaml

@@ -17,7 +17,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.5
+version: 0.2.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

packages/apps/virtual-machine/Makefile

@@ -0,0 +1,7 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json.tmp -r README.md
+	cat values.schema.json.tmp | \
+		jq '.properties.image.enum = ["ubuntu", "cirros", "alpine", "fedora"]' | \
+		jq '.properties.resources.properties.memory["x-display"] = "slider"' \
+		> values.schema.json
+	rm -f values.schema.json.tmp

packages/apps/virtual-machine/README.md

@@ -8,3 +8,17 @@ The virtual machine is managed and hosted through KubeVirt, allowing you to harn
 
 - Docs: https://kubevirt.io/user-guide/
 - GitHub: https://github.com/kubevirt/kubevirt
+
+## Parameters
+
+### Common parameters
+
+| Name               | Description                                                                                       | Value    |
+| ------------------ | ------------------------------------------------------------------------------------------------- | -------- |
+| `external`         | Enable external access from outside the cluster                                                   | `false`  |
+| `running`          | Determines if the virtual machine should be running                                               | `true`   |
+| `password`         | The default password for the virtual machine                                                      | `hackme` |
+| `image`            | The base image for the virtual machine. Allowed values: `ubuntu`, `cirros`, `alpine` and `fedora` | `ubuntu` |
+| `disk`             | The size of the disk allocated for the virtual machine                                            | `5Gi`    |
+| `resources.cpu`    | The number of CPU cores allocated to the virtual machine                                          | `1`      |
+| `resources.memory` | The amount of memory allocated to the virtual machine                                             | `1024M`  |

packages/apps/virtual-machine/values.schema.json

@@ -1,44 +1,51 @@
 {
-  "$schema": "http://json-schema.org/schema#",
+  "title": "Chart Values",
   "type": "object",
   "properties": {
-    "running": {
-      "type": "boolean",
-      "title": "Running"
-    },
     "external": {
       "type": "boolean",
-      "title": "Enable external Access"
+      "description": "Enable external access from outside the cluster",
+      "default": false
     },
-    "image": {
-      "type": "string",
-      "form": true,
-      "enum": ["ubuntu", "cirros", "alpine", "fedora"],
-      "title": "Choose image"
+    "running": {
+      "type": "boolean",
+      "description": "Determines if the virtual machine should be running",
+      "default": true
     },
     "password": {
       "type": "string",
-      "title": "Password"
+      "description": "The default password for the virtual machine",
+      "default": "hackme"
+    },
+    "image": {
+      "type": "string",
+      "description": "The base image for the virtual machine. Allowed values: `ubuntu`, `cirros`, `alpine` and `fedora`",
+      "default": "ubuntu",
+      "enum": [
+        "ubuntu",
+        "cirros",
+        "alpine",
+        "fedora"
+      ]
     },
     "disk": {
       "type": "string",
-      "title": "Disk Size"
+      "description": "The size of the disk allocated for the virtual machine",
+      "default": "5Gi"
     },
     "resources": {
       "type": "object",
       "properties": {
+        "cpu": {
+          "type": "number",
+          "description": "The number of CPU cores allocated to the virtual machine",
+          "default": 1
+        },
         "memory": {
           "type": "string",
-          "x-display": "slider",
-          "title": "Memory",
-          "minimum": 10,
-          "maximum": 2048
-        },
-        "cpu": {
-          "type": "integer",
-          "title": "CPU",
-          "minimum": 1,
-          "maximum": 4
+          "description": "The amount of memory allocated to the virtual machine",
+          "default": "1024M",
+          "x-display": "slider"
         }
       }
     }

packages/apps/virtual-machine/values.yaml

@@ -1,3 +1,13 @@
+## @section Common parameters
+
+## @param external Enable external access from outside the cluster
+## @param running Determines if the virtual machine should be running
+## @param password The default password for the virtual machine
+## @param image The base image for the virtual machine. Allowed values: `ubuntu`, `cirros`, `alpine` and `fedora`
+## @param disk The size of the disk allocated for the virtual machine
+## @param resources.cpu The number of CPU cores allocated to the virtual machine
+## @param resources.memory The amount of memory allocated to the virtual machine
+
 external: false
 running: true
 password: hackme

packages/apps/vpn/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v2
 name: vpn
-description: Establish a connection from your computer
+description: Managed VPN service
 icon: https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Outline_VPN_icon.png/600px-Outline_VPN_icon.png
 
 # A chart can be either an 'application' or a 'library' chart.
@@ -16,10 +16,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.0
+version: 0.2.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "1.16.0"
+appVersion: "1.8.1"

packages/apps/vpn/Makefile

@@ -0,0 +1,2 @@
+generate:
+	readme-generator -v values.yaml -s values.schema.json -r README.md

packages/apps/vpn/README.md

@@ -10,3 +10,18 @@ The VPN Service is powered by the Outline Server, an advanced and user-friendly
 
 - Docs: https://shadowsocks.org/
 - Docs: https://github.com/Jigsaw-Code/outline-server/tree/master/src/shadowbox
+
+## Parameters
+
+### Common parameters
+
+| Name       | Description                                     | Value   |
+| ---------- | ----------------------------------------------- | ------- |
+| `external` | Enable external access from outside the cluster | `false` |
+| `replicas` | Number of VPN-server replicas                   | `2`     |
+
+### Configuration parameters
+
+| Name    | Description         | Value |
+| ------- | ------------------- | ----- |
+| `users` | Users configuration | `{}`  |

packages/apps/vpn/templates/deployment.yaml

@@ -4,7 +4,7 @@ kind: Deployment
 metadata:
   name: {{ .Release.Name }}-vpn
 spec:
-  replicas: 2
+  replicas: {{ .Values.replicas }}
   selector:
     matchLabels:
       app: {{ .Release.Name }}-vpn

packages/apps/vpn/values.schema.json

@@ -0,0 +1,16 @@
+{
+    "title": "Chart Values",
+    "type": "object",
+    "properties": {
+        "external": {
+            "type": "boolean",
+            "description": "Enable external access from outside the cluster",
+            "default": false
+        },
+        "replicas": {
+            "type": "number",
+            "description": "Number of VPN-server replicas",
+            "default": 2
+        }
+    }
+}

packages/apps/vpn/values.yaml

@@ -1,7 +1,18 @@
-external: false
+## @section Common parameters
 
-users:
-  user1:
-    password: hackme
-  user2:
-    password: tttt
+## @param external Enable external access from outside the cluster
+## @param replicas Number of VPN-server replicas
+##
+external: false
+replicas: 2
+
+## @section Configuration parameters
+
+## @param users [object] Users configuration
+## Example:
+## users:
+##   user1:
+##     password: hackme
+##   user2:
+##     password: tttt
+users: {}