github-personal/cozystack
1
0
Fork 0
mirror of https://github.com/outbackdingo/cozystack.git synced 2026-02-05 08:17:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Upgrade proxmox-CCM to 0.5.1 and proxmox-csi-plugin to 0.8.2

Browse Source
This commit is contained in:
Marian Koreniuk
2024-11-22 20:31:47 +01:00
parent f76bc8d823
commit 7258c56685
21 changed files with 456 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/Chart.yaml
View File

@@ -1,24 +1,24 @@
apiVersion: v2
name: proxmox-cloud-controller-manager
description: A Helm chart for Kubernetes
description: Cloud Controller Manager plugin for Proxmox
type: application
home: https://github.com/sergelogvinov/proxmox-cloud-controller-manager
icon: https://proxmox.com/templates/yoo_nano2/favicon.ico
icon: https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/charts/proxmox-cloud-controller-manager/icon.png
sources:
- https://github.com/sergelogvinov/proxmox-cloud-controller-manager
- https://github.com/sergelogvinov/proxmox-cloud-controller-manager
keywords:
- ccm
- ccm
- proxmox
- kubernetes
maintainers:
- name: sergelogvinov
url: https://github.com/sergelogvinov
- name: sergelogvinov
url: https://github.com/sergelogvinov
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.6
version: 0.2.8
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: v0.2.0
appVersion: v0.5.1

48
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/README.md
View File

@@ -1,8 +1,14 @@
# proxmox-cloud-controller-manager
![Version: 0.1.6](https://img.shields.io/badge/Version-0.1.6-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.2.0](https://img.shields.io/badge/AppVersion-v0.2.0-informational?style=flat-square)
![Version: 0.2.8](https://img.shields.io/badge/Version-0.2.8-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.5.1](https://img.shields.io/badge/AppVersion-v0.5.1-informational?style=flat-square)
A Helm chart for Kubernetes
Cloud Controller Manager plugin for Proxmox
The Cloud Controller Manager (CCM) is responsible for managing node resources in cloud-based Kubernetes environments.
Key functions of the Cloud Controller Manager:
- `Node Management`: It manages nodes by initializing new nodes when they join the cluster (e.g., during scaling up) and removing nodes when they are no longer needed (e.g., during scaling down).
- `Cloud-Specific Operations`: The CCM ensures that the cloud provider's API is integrated into the Kubernetes cluster to control and automate tasks like load balancing, storage provisioning, and node lifecycle management.
**Homepage:** <https://github.com/sergelogvinov/proxmox-cloud-controller-manager>
@@ -16,7 +22,22 @@ A Helm chart for Kubernetes
* <https://github.com/sergelogvinov/proxmox-cloud-controller-manager>
Example:
## Requirements
You need to set `--cloud-provider=external` in the kubelet argument for all nodes in the cluster.
## Proxmox permissions
```shell
# Create role CCM
pveum role add CCM -privs "VM.Audit"
# Create user and grant permissions
pveum user add kubernetes@pve
pveum aclmod / -user kubernetes@pve -role CCM
pveum user token add kubernetes@pve ccm -privsep 0
```
## Helm values example
```yaml
# proxmox-ccm.yaml
@@ -35,8 +56,13 @@ enabledControllers:
- cloud-node-lifecycle
# Deploy CCM only on control-plane nodes
nodeSelector:
node-role.kubernetes.io/control-plane: ""
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
tolerations:
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
@@ -46,7 +72,7 @@ Deploy chart:
```shell
helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
proxmox-cloud-controller-manager charts/proxmox-cloud-controller-manager
proxmox-cloud-controller-manager oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager
```
## Values
@@ -68,14 +94,16 @@ helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
| config | object | `{"clusters":[]}` | Proxmox cluster config. |
| serviceAccount | object | `{"annotations":{},"create":true,"name":""}` | Pods Service Account. ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ |
| priorityClassName | string | `"system-cluster-critical"` | CCM pods' priorityClassName. |
| initContainers | list | `[]` | Add additional init containers to the CCM pods. ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ |
| hostAliases | list | `[]` | hostAliases Deployment pod host aliases ref: https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/ |
| podAnnotations | object | `{}` | Annotations for data pods. ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ |
| podSecurityContext | object | `{"fsGroup":10258,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":10258,"runAsNonRoot":true,"runAsUser":10258}` | Pods Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
| securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}` | Container Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
| resources | object | `{"requests":{"cpu":"10m","memory":"32Mi"}}` | Resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Deployment update stategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
| useDaemonSet | bool | `false` | Deploy CCM in Daemonset mode. CCM will use hostNetwork. It allows to use CCM without CNI plugins. |
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Deployment update strategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
| nodeSelector | object | `{}` | Node labels for data pods assignment. ref: https://kubernetes.io/docs/user-guide/node-selection/ |
| tolerations | list | `[{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane","operator":"Exists"},{"effect":"NoSchedule","key":"node.cloudprovider.kubernetes.io/uninitialized","operator":"Exists"}]` | Tolerations for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ |
| affinity | object | `{}` | Affinity for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity |
----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.11.2](https://github.com/norwoodj/helm-docs/releases/v1.11.2)
| extraVolumes | list | `[]` | Additional volumes for Pods |
| extraVolumeMounts | list | `[]` | Additional volume mounts for Pods |

36
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/README.md.gotmpl
View File

@@ -6,6 +6,12 @@
{{ template "chart.description" . }}
The Cloud Controller Manager (CCM) is responsible for managing node resources in cloud-based Kubernetes environments.
Key functions of the Cloud Controller Manager:
- `Node Management`: It manages nodes by initializing new nodes when they join the cluster (e.g., during scaling up) and removing nodes when they are no longer needed (e.g., during scaling down).
- `Cloud-Specific Operations`: The CCM ensures that the cloud provider's API is integrated into the Kubernetes cluster to control and automate tasks like load balancing, storage provisioning, and node lifecycle management.
{{ template "chart.homepageLine" . }}
{{ template "chart.maintainersSection" . }}
@@ -14,7 +20,22 @@
{{ template "chart.requirementsSection" . }}
Example:
## Requirements
You need to set `--cloud-provider=external` in the kubelet argument for all nodes in the cluster.
## Proxmox permissions
```shell
# Create role CCM
pveum role add CCM -privs "VM.Audit"
# Create user and grant permissions
pveum user add kubernetes@pve
pveum aclmod / -user kubernetes@pve -role CCM
pveum user token add kubernetes@pve ccm -privsep 0
```
## Helm values example
```yaml
# proxmox-ccm.yaml
@@ -33,8 +54,13 @@ enabledControllers:
- cloud-node-lifecycle
# Deploy CCM only on control-plane nodes
nodeSelector:
node-role.kubernetes.io/control-plane: ""
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
tolerations:
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
@@ -44,9 +70,7 @@ Deploy chart:
```shell
helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
proxmox-cloud-controller-manager charts/proxmox-cloud-controller-manager
proxmox-cloud-controller-manager oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager
```
{{ template "chart.valuesSection" . }}
{{ template "helm-docs.versionFooter" . }}

10
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/ci/values.yaml
View File

@@ -1,11 +1,15 @@
image:
repository: ghcr.io/sergelogvinov/proxmox-cloud-controller-manager
pullPolicy: Always
tag: edge
nodeSelector:
node-role.kubernetes.io/control-plane: ""
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
logVerbosityLevel: 4

BIN
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 37 KiB

51
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/templates/deployment.yaml
View File

@@ -1,14 +1,23 @@
apiVersion: apps/v1
{{- if .Values.useDaemonSet }}
kind: DaemonSet
{{- else }}
kind: Deployment
{{- end }}
metadata:
name: {{ include "proxmox-cloud-controller-manager.fullname" . }}
labels:
{{- include "proxmox-cloud-controller-manager.labels" . | nindent 4 }}
namespace: {{ .Release.Namespace }}
spec:
{{- if not .Values.useDaemonSet }}
replicas: {{ .Values.replicaCount }}
strategy:
type: {{ .Values.updateStrategy.type }}
{{- else }}
updateStrategy:
type: {{ .Values.updateStrategy.type }}
{{- end }}
selector:
matchLabels:
{{- include "proxmox-cloud-controller-manager.selectorLabels" . | nindent 6 }}
@@ -35,6 +44,15 @@ spec:
serviceAccountName: {{ include "proxmox-cloud-controller-manager.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
{{- if .Values.useDaemonSet }}
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true
{{- end }}
{{- with .Values.hostAliases }}
hostAliases:
{{- toYaml . | nindent 8 }}
{{- end }}
initContainers: {{- toYaml .Values.initContainers | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
@@ -49,13 +67,18 @@ spec:
- --leader-elect-resource-name=cloud-controller-manager-proxmox
- --use-service-account-credentials
- --secure-port=10258
- --authorization-always-allow-paths=/healthz,/livez,/readyz,/metrics
{{- with .Values.extraArgs }}
{{- toYaml . | nindent 12 }}
{{- end }}
ports:
- name: metrics
containerPort: 10258
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: 10258
port: metrics
scheme: HTTPS
initialDelaySeconds: 20
periodSeconds: 30
@@ -66,18 +89,36 @@ spec:
- name: cloud-config
mountPath: /etc/proxmox
readOnly: true
{{- with .Values.extraVolumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- with .Values.affinity }}
{{- toYaml . | nindent 8 }}
{{- else }}
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
{{- include "proxmox-cloud-controller-manager.selectorLabels" . | nindent 20 }}
topologyKey: topology.kubernetes.io/zone
weight: 1
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- with .Values.tolerations }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if .Values.useDaemonSet }}
- effect: NoSchedule
key: node.kubernetes.io/not-ready
operator: Exists
{{- end }}
{{- if not .Values.useDaemonSet }}
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
@@ -85,6 +126,7 @@ spec:
labelSelector:
matchLabels:
{{- include "proxmox-cloud-controller-manager.selectorLabels" . | nindent 14 }}
{{- end }}
volumes:
{{- if .Values.existingConfigSecret }}
- name: cloud-config
@@ -100,3 +142,6 @@ spec:
secretName: {{ include "proxmox-cloud-controller-manager.fullname" . }}
defaultMode: 416
{{- end }}
{{- with .Values.extraVolumes }}
{{- toYaml . | nindent 8 }}
{{- end }}

10
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/values.edge.yaml
View File

@@ -1,10 +1,14 @@
image:
pullPolicy: Always
tag: edge
nodeSelector:
node-role.kubernetes.io/control-plane: ""
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
logVerbosityLevel: 4

10
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/values.talos.yaml
View File

@@ -1,6 +1,10 @@
nodeSelector:
node-role.kubernetes.io/control-plane: ""
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
logVerbosityLevel: 4

54
packages/system/proxmox-csi/charts/proxmox-cloud-controller-manager/values.yaml
View File

@@ -61,6 +61,36 @@ serviceAccount:
# -- CCM pods' priorityClassName.
priorityClassName: system-cluster-critical
# -- Add additional init containers to the CCM pods.
# ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
initContainers: []
# - name: loadbalancer
# restartPolicy: Always
# image: ghcr.io/sergelogvinov/haproxy:2.8.3-alpine3.18
# imagePullPolicy: IfNotPresent
# env:
# - name: SVC
# value: "proxmox.domain.com"
# - name: PORT
# value: "8006"
# securityContext:
# runAsUser: 99
# runAsGroup: 99
# resources:
# limits:
# cpu: 50m
# memory: 64Mi
# requests:
# cpu: 50m
# memory: 32Mi
# -- hostAliases Deployment pod host aliases
# ref: https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/
hostAliases: []
# - ip: 127.0.0.1
# hostnames:
# - proxmox.domain.com
# -- Annotations for data pods.
# ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
podAnnotations: {}
@@ -98,7 +128,12 @@ resources:
cpu: 10m
memory: 32Mi
# -- Deployment update stategy type.
# -- Deploy CCM in Daemonset mode.
# CCM will use hostNetwork.
# It allows to use CCM without CNI plugins.
useDaemonSet: false
# -- Deployment update strategy type.
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment
updateStrategy:
type: RollingUpdate
@@ -123,3 +158,20 @@ tolerations:
# -- Affinity for data pods assignment.
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: node-role.kubernetes.io/control-plane
# operator: Exists
# -- Additional volumes for Pods
extraVolumes: []
# - name: ca
# secret:
# secretName: my-ca
# -- Additional volume mounts for Pods
extraVolumeMounts: []
# - mountPath: /etc/ssl/certs/ca-certificates.crt
# name: ca
# subPath: ca.crt

25
packages/system/proxmox-csi/charts/proxmox-csi-plugin/Chart.yaml
View File

@@ -1,26 +1,27 @@
# yaml-language-server: $schema=https://json.schemastore.org/chart.json
apiVersion: v2
name: proxmox-csi-plugin
description: A CSI plugin for Proxmox
description: Container Storage Interface plugin for Proxmox
type: application
home: https://github.com/sergelogvinov/proxmox-csi-plugin
icon: https://proxmox.com/templates/yoo_nano2/favicon.ico
icon: https://raw.githubusercontent.com/sergelogvinov/proxmox-csi-plugin/main/charts/proxmox-csi-plugin/icon.png
sources:
- https://github.com/sergelogvinov/proxmox-csi-plugin
- https://github.com/sergelogvinov/proxmox-csi-plugin
keywords:
- storage
- block-storage
- volume
- csi
- storage
- block-storage
- volume
- proxmox
maintainers:
- name: sergelogvinov
url: https://github.com/sergelogvinov
- name: sergelogvinov
url: https://github.com/sergelogvinov
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.6
version: 0.2.13
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: v0.3.0
appVersion: v0.8.2

63
packages/system/proxmox-csi/charts/proxmox-csi-plugin/README.md
View File

@@ -1,8 +1,19 @@
# proxmox-csi-plugin
![Version: 0.1.6](https://img.shields.io/badge/Version-0.1.6-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.3.0](https://img.shields.io/badge/AppVersion-v0.3.0-informational?style=flat-square)
![Version: 0.2.13](https://img.shields.io/badge/Version-0.2.13-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.8.2](https://img.shields.io/badge/AppVersion-v0.8.2-informational?style=flat-square)
A CSI plugin for Proxmox
Container Storage Interface plugin for Proxmox
The Container Storage Interface (CSI) plugin is a specification designed to standardize the way container orchestration systems like Kubernetes, interact with different storage systems. The CSI plugin abstracts the underlying storage, enabling the seamless integration of different storage solutions (such as local block devices, file systems, or cloud-based storage) with containerized applications.
This plugin allows Kubernetes to use `Proxmox VE` storage as a persistent storage solution for stateful applications.
Supported storage types:
- Directory
- LVM
- LVM-thin
- ZFS
- NFS
- Ceph
**Homepage:** <https://github.com/sergelogvinov/proxmox-csi-plugin>
@@ -16,7 +27,18 @@ A CSI plugin for Proxmox
* <https://github.com/sergelogvinov/proxmox-csi-plugin>
Example:
## Proxmox permissions
```shell
# Create role CSI
pveum role add CSI -privs "VM.Audit VM.Config.Disk Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"
# Create user and grant permissions
pveum user add kubernetes-csi@pve
pveum aclmod / -user kubernetes-csi@pve -role CSI
pveum user token add kubernetes-csi@pve csi -privsep 0
```
## Helm values example
```yaml
# proxmox-csi.yaml
@@ -58,11 +80,15 @@ storageClass:
cache: writethrough
```
Deploy chart:
## Deploy
```shell
# Prepare namespace
kubectl create ns csi-proxmox
kubectl label ns csi-proxmox pod-security.kubernetes.io/enforce=privileged
# Install Proxmox CSI plugin
helm upgrade -i --namespace=csi-proxmox -f proxmox-csi.yaml \
proxmox-csi-plugin charts/proxmox-csi-plugin/
proxmox-csi-plugin oci://ghcr.io/sergelogvinov/charts/proxmox-csi-plugin
```
## Values
@@ -73,6 +99,7 @@ helm upgrade -i --namespace=csi-proxmox -f proxmox-csi.yaml \
| imagePullSecrets | list | `[]` | |
| nameOverride | string | `""` | |
| fullnameOverride | string | `""` | |
| createNamespace | bool | `false` | Create namespace. Very useful when using helm template. |
| priorityClassName | string | `"system-cluster-critical"` | Controller pods priorityClassName. |
| serviceAccount | object | `{"annotations":{},"create":true,"name":""}` | Pods Service Account. ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ |
| provisionerName | string | `"csi.proxmox.sinextra.dev"` | CSI Driver provisioner name. Currently, cannot be customized. |
@@ -83,34 +110,40 @@ helm upgrade -i --namespace=csi-proxmox -f proxmox-csi.yaml \
| existingConfigSecretKey | string | `"config.yaml"` | Proxmox cluster config stored in secrets key. |
| configFile | string | `"/etc/proxmox/config.yaml"` | Proxmox cluster config path. |
| config | object | `{"clusters":[]}` | Proxmox cluster config. |
| storageClass | list | `[]` | Storage class defenition. |
| storageClass | list | `[]` | Storage class definition. |
| controller.podAnnotations | object | `{}` | Annotations for controller pod. ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ |
| controller.plugin.image | object | `{"pullPolicy":"IfNotPresent","repository":"ghcr.io/sergelogvinov/proxmox-csi-controller","tag":""}` | Controller CSI Driver. |
| controller.plugin.resources | object | `{"requests":{"cpu":"10m","memory":"16Mi"}}` | Controller resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| controller.attacher.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-attacher","tag":"v4.3.0"}` | CSI Attacher. |
| controller.attacher.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-attacher","tag":"v4.4.4"}` | CSI Attacher. |
| controller.attacher.resources | object | `{"requests":{"cpu":"10m","memory":"16Mi"}}` | Attacher resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| controller.provisioner.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-provisioner","tag":"v3.5.0"}` | CSI Provisioner. |
| controller.provisioner.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-provisioner","tag":"v3.6.4"}` | CSI Provisioner. |
| controller.provisioner.resources | object | `{"requests":{"cpu":"10m","memory":"16Mi"}}` | Provisioner resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| controller.resizer.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-resizer","tag":"v1.8.0"}` | CSI Resizer. |
| controller.resizer.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-resizer","tag":"v1.9.4"}` | CSI Resizer. |
| controller.resizer.resources | object | `{"requests":{"cpu":"10m","memory":"16Mi"}}` | Resizer resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| node.plugin.image | object | `{"pullPolicy":"IfNotPresent","repository":"ghcr.io/sergelogvinov/proxmox-csi-node","tag":""}` | Node CSI Driver. |
| node.plugin.resources | object | `{}` | Node CSI Driver resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| node.driverRegistrar.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-node-driver-registrar","tag":"v2.8.0"}` | Node CSI driver registrar. |
| node.driverRegistrar.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/csi-node-driver-registrar","tag":"v2.9.4"}` | Node CSI driver registrar. |
| node.driverRegistrar.resources | object | `{"requests":{"cpu":"10m","memory":"16Mi"}}` | Node registrar resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| node.kubeletDir | string | `"/var/lib/kubelet"` | Location of the /var/lib/kubelet directory as some k8s distribution differ from the standard. |
| node.nodeSelector | object | `{}` | Node labels for node-plugin assignment. ref: https://kubernetes.io/docs/user-guide/node-selection/ |
| node.tolerations | list | `[{"effect":"NoSchedule","key":"node.kubernetes.io/unschedulable","operator":"Exists"},{"effect":"NoSchedule","key":"node.kubernetes.io/disk-pressure","operator":"Exists"}]` | Tolerations for node-plugin assignment. ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ |
| livenessprobe.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/livenessprobe","tag":"v2.10.0"}` | Common livenessprobe sidecar. |
| livenessprobe.image | object | `{"pullPolicy":"IfNotPresent","repository":"registry.k8s.io/sig-storage/livenessprobe","tag":"v2.11.0"}` | Common livenessprobe sidecar. |
| livenessprobe.failureThreshold | int | `5` | Failure threshold for livenessProbe |
| livenessprobe.initialDelaySeconds | int | `10` | Initial delay seconds for livenessProbe |
| livenessprobe.timeoutSeconds | int | `10` | Timeout seconds for livenessProbe |
| livenessprobe.periodSeconds | int | `60` | Period seconds for livenessProbe |
| livenessprobe.resources | object | `{"requests":{"cpu":"10m","memory":"16Mi"}}` | Liveness probe resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
| initContainers | list | `[]` | Add additional init containers for the CSI controller pods. ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ |
| hostAliases | list | `[]` | hostAliases Deployment pod host aliases ref: https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/ |
| podAnnotations | object | `{}` | Annotations for controller pod. ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ |
| podSecurityContext | object | `{"fsGroup":65532,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":65532,"runAsNonRoot":true,"runAsUser":65532}` | Controller Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
| securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"seccompProfile":{"type":"RuntimeDefault"}}` | Controller Container Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Controller deployment update stategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Controller deployment update strategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
| metrics | object | `{"enabled":false,"port":8080,"type":"annotation"}` | Prometheus metrics |
| metrics.enabled | bool | `false` | Enable Prometheus metrics. |
| metrics.port | int | `8080` | Prometheus metrics port. |
| nodeSelector | object | `{}` | Node labels for controller assignment. ref: https://kubernetes.io/docs/user-guide/node-selection/ |
| tolerations | list | `[]` | Tolerations for controller assignment. ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ |
| affinity | object | `{}` | Affinity for controller assignment. ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity |
----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)
| extraVolumes | list | `[]` | Additional volumes for Pods |
| extraVolumeMounts | list | `[]` | |

34
packages/system/proxmox-csi/charts/proxmox-csi-plugin/README.md.gotmpl
View File

@@ -6,6 +6,17 @@
{{ template "chart.description" . }}
The Container Storage Interface (CSI) plugin is a specification designed to standardize the way container orchestration systems like Kubernetes, interact with different storage systems. The CSI plugin abstracts the underlying storage, enabling the seamless integration of different storage solutions (such as local block devices, file systems, or cloud-based storage) with containerized applications.
This plugin allows Kubernetes to use `Proxmox VE` storage as a persistent storage solution for stateful applications.
Supported storage types:
- Directory
- LVM
- LVM-thin
- ZFS
- NFS
- Ceph
{{ template "chart.homepageLine" . }}
{{ template "chart.maintainersSection" . }}
@@ -14,7 +25,18 @@
{{ template "chart.requirementsSection" . }}
Example:
## Proxmox permissions
```shell
# Create role CSI
pveum role add CSI -privs "VM.Audit VM.Config.Disk Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"
# Create user and grant permissions
pveum user add kubernetes-csi@pve
pveum aclmod / -user kubernetes-csi@pve -role CSI
pveum user token add kubernetes-csi@pve csi -privsep 0
```
## Helm values example
```yaml
# proxmox-csi.yaml
@@ -56,13 +78,15 @@ storageClass:
cache: writethrough
```
Deploy chart:
## Deploy
```shell
# Prepare namespace
kubectl create ns csi-proxmox
kubectl label ns csi-proxmox pod-security.kubernetes.io/enforce=privileged
# Install Proxmox CSI plugin
helm upgrade -i --namespace=csi-proxmox -f proxmox-csi.yaml \
proxmox-csi-plugin charts/proxmox-csi-plugin/
proxmox-csi-plugin oci://ghcr.io/sergelogvinov/charts/proxmox-csi-plugin
```
{{ template "chart.valuesSection" . }}
{{ template "helm-docs.versionFooter" . }}

BIN
packages/system/proxmox-csi/charts/proxmox-csi-plugin/icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

13
packages/system/proxmox-csi/charts/proxmox-csi-plugin/templates/_storage.tpl Normal file
View File

@@ -0,0 +1,13 @@
{{/*
storageClass parameters uses to merge the default parameters with the user provided parameters.
*/}}
{{- define "storageClass.parameters" -}}
csi.storage.k8s.io/fstype: {{ default "ext4" .fstype }}
storage: {{ .storage | required "Proxmox Storage name must be provided." }}
{{- with .cache }}
cache: {{ . }}
{{- end }}
{{- if .ssd }}
ssd: "true"
{{- end }}
{{- end }}

23
packages/system/proxmox-csi/charts/proxmox-csi-plugin/templates/controller-deployment.yaml
View File

@@ -18,9 +18,13 @@ spec:
metadata:
annotations:
checksum/config: {{ toJson .Values.config | sha256sum }}
{{- with .Values.podAnnotations }}
{{- with default .Values.podAnnotations .Values.controller.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if and .Values.metrics.enabled (eq .Values.metrics.type "annotation") }}
prometheus.io/scrape: "true"
prometheus.io/port: {{ .Values.metrics.port | quote }}
{{- end }}
labels:
{{- include "proxmox-csi-plugin.selectorLabels" . | nindent 8 }}
spec:
@@ -35,6 +39,8 @@ spec:
serviceAccountName: {{ include "proxmox-csi-plugin.serviceAccountName" . }}-controller
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
hostAliases: {{- toYaml .Values.hostAliases | nindent 8 }}
initContainers: {{- toYaml .Values.initContainers | nindent 8 }}
containers:
- name: {{ include "proxmox-csi-plugin.fullname" . }}-controller
securityContext:
@@ -45,6 +51,15 @@ spec:
- "-v={{ .Values.logVerbosityLevel }}"
- "--csi-address=unix:///csi/csi.sock"
- "--cloud-config={{ .Values.configFile }}"
{{- if .Values.metrics.enabled }}
- "--metrics-address=:{{ .Values.metrics.port }}"
{{- end }}
ports:
{{- if .Values.metrics.enabled }}
- name: metrics
containerPort: {{ .Values.metrics.port }}
protocol: TCP
{{- end }}
resources:
{{- toYaml .Values.controller.plugin.resources | nindent 12 }}
volumeMounts:
@@ -52,6 +67,9 @@ spec:
mountPath: /csi
- name: cloud-config
mountPath: /etc/proxmox/
{{- with .Values.extraVolumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
- name: csi-attacher
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
@@ -136,6 +154,9 @@ spec:
secret:
secretName: {{ include "proxmox-csi-plugin.fullname" . }}
{{- end }}
{{- with .Values.extraVolumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}

10
packages/system/proxmox-csi/charts/proxmox-csi-plugin/templates/namespace.yaml Normal file
View File

@@ -0,0 +1,10 @@
{{- if and .Values.createNamespace (ne .Release.Namespace "kube-system") }}
apiVersion: v1
kind: Namespace
metadata:
name: {{ .Release.Namespace }}
labels:
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/audit: baseline
pod-security.kubernetes.io/warn: baseline
{{- end }}

12
packages/system/proxmox-csi/charts/proxmox-csi-plugin/templates/node-deployment.yaml
View File

@@ -59,7 +59,7 @@ spec:
- name: socket
mountPath: /csi
- name: kubelet
mountPath: /var/lib/kubelet
mountPath: {{ .Values.node.kubeletDir }}
mountPropagation: Bidirectional
- name: dev
mountPath: /dev
@@ -71,7 +71,7 @@ spec:
capabilities:
drop:
- ALL
# readOnlyRootFilesystem: true
readOnlyRootFilesystem: true
seccompProfile:
type: RuntimeDefault
image: "{{ .Values.node.driverRegistrar.image.repository }}:{{ .Values.node.driverRegistrar.image.tag }}"
@@ -79,7 +79,7 @@ spec:
args:
- "-v={{ .Values.logVerbosityLevel }}"
- "--csi-address=unix:///csi/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/plugins/{{ .Values.provisionerName }}/csi.sock"
- "--kubelet-registration-path={{ .Values.node.kubeletDir }}/plugins/{{ .Values.provisionerName }}/csi.sock"
volumeMounts:
- name: socket
mountPath: /csi
@@ -107,15 +107,15 @@ spec:
volumes:
- name: socket
hostPath:
path: /var/lib/kubelet/plugins/{{ .Values.provisionerName }}/
path: {{ .Values.node.kubeletDir }}/plugins/{{ .Values.provisionerName }}/
type: DirectoryOrCreate
- name: registration
hostPath:
path: /var/lib/kubelet/plugins_registry/
path: {{ .Values.node.kubeletDir }}/plugins_registry/
type: Directory
- name: kubelet
hostPath:
path: /var/lib/kubelet
path: {{ .Values.node.kubeletDir }}
type: Directory
- name: dev
hostPath:

27
packages/system/proxmox-csi/charts/proxmox-csi-plugin/templates/storageclass.yaml
View File

@@ -2,19 +2,28 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: {{ $storage.name }}
name: {{ $storage.name | required "StorageClass name must be provided." }}
{{- with $storage.labels }}
labels:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with $storage.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
provisioner: {{ $.Values.provisionerName }}
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: {{ default "Delete" $storage.reclaimPolicy }}
parameters:
csi.storage.k8s.io/fstype: {{ default "ext4" $storage.fstype }}
storage: {{ $storage.storage }}
{{- if $storage.cache }}
cache: {{ $storage.cache }}
{{- end }}
{{- if $storage.ssd }}
ssd: "true"
{{- end }}
{{- mustMergeOverwrite (default (dict) $storage.extraParameters) (include "storageClass.parameters" . | fromYaml) | toYaml | nindent 2 -}}
{{- with $storage.mountOptions }}
mountOptions:
{{- . | toYaml | nindent 2 }}
{{- end }}
{{- with $storage.allowedTopologies }}
allowedTopologies:
{{- . | toYaml | nindent 2 }}
{{- end }}
---
{{- end }}

5
packages/system/proxmox-csi/charts/proxmox-csi-plugin/values.edge.yaml
View File

@@ -1,4 +1,6 @@
createNamespace: true
controller:
plugin:
image:
@@ -11,9 +13,6 @@ node:
pullPolicy: Always
tag: edge
nodeSelector:
node.cloudprovider.kubernetes.io/platform: nocloud
nodeSelector:
node-role.kubernetes.io/control-plane: ""
tolerations:

2
packages/system/proxmox-csi/charts/proxmox-csi-plugin/values.talos.yaml
View File

@@ -1,4 +1,6 @@
createNamespace: true
node:
nodeSelector:
node.cloudprovider.kubernetes.io/platform: nocloud

100
packages/system/proxmox-csi/charts/proxmox-csi-plugin/values.yaml
View File

@@ -8,6 +8,10 @@ imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
# -- Create namespace.
# Very useful when using helm template.
createNamespace: false
# -- Controller pods priorityClassName.
priorityClassName: system-cluster-critical
@@ -54,7 +58,7 @@ config:
# token_secret: "secret"
# region: cluster-1
# -- Storage class defenition.
# -- Storage class definition.
storageClass: []
# - name: proxmox-data-xfs
# storage: data
@@ -64,8 +68,35 @@ storageClass: []
# # https://pve.proxmox.com/wiki/Performance_Tweaks
# cache: directsync|none|writeback|writethrough
# ssd: true
# extraParameters:
# # https://kubernetes-csi.github.io/docs/secrets-and-credentials-storage-class.html
# csi.storage.k8s.io/node-stage-secret-name: "proxmox-csi-secret"
# csi.storage.k8s.io/node-stage-secret-namespace: "kube-system"
# csi.storage.k8s.io/node-expand-secret-name: "proxmox-csi-secret"
# csi.storage.k8s.io/node-expand-secret-namespace: "kube-system"
#
# mountOptions:
# - discard
#
# # I am not recommending to use this option, but it is possible.
# # You should use region and zone topology keys together.
# allowedTopologies:
# - matchLabelExpressions:
# - key: topology.kubernetes.io/region
# values:
# - Region-1
# - key: topology.kubernetes.io/zone
# values:
# - pve-1
# - pve-3
controller:
# -- Annotations for controller pod.
# ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
podAnnotations: {}
# prometheus.io/scrape: "true"
# prometheus.io/port: "8080"
plugin:
# -- Controller CSI Driver.
image:
@@ -84,7 +115,7 @@ controller:
image:
repository: registry.k8s.io/sig-storage/csi-attacher
pullPolicy: IfNotPresent
tag: v4.3.0
tag: v4.4.4
# -- Attacher resource requests and limits.
# ref: https://kubernetes.io/docs/user-guide/compute-resources/
resources:
@@ -96,7 +127,7 @@ controller:
image:
repository: registry.k8s.io/sig-storage/csi-provisioner
pullPolicy: IfNotPresent
tag: v3.5.0
tag: v3.6.4
# -- Provisioner resource requests and limits.
# ref: https://kubernetes.io/docs/user-guide/compute-resources/
resources:
@@ -108,7 +139,7 @@ controller:
image:
repository: registry.k8s.io/sig-storage/csi-resizer
pullPolicy: IfNotPresent
tag: v1.8.0
tag: v1.9.4
# -- Resizer resource requests and limits.
# ref: https://kubernetes.io/docs/user-guide/compute-resources/
resources:
@@ -132,7 +163,7 @@ node:
image:
repository: registry.k8s.io/sig-storage/csi-node-driver-registrar
pullPolicy: IfNotPresent
tag: v2.8.0
tag: v2.9.4
# -- Node registrar resource requests and limits.
# ref: https://kubernetes.io/docs/user-guide/compute-resources/
resources:
@@ -140,6 +171,9 @@ node:
cpu: 10m
memory: 16Mi
# -- Location of the /var/lib/kubelet directory as some k8s distribution differ from the standard.
kubeletDir: /var/lib/kubelet
# -- Node labels for node-plugin assignment.
# ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}
@@ -159,7 +193,7 @@ livenessprobe:
image:
repository: registry.k8s.io/sig-storage/livenessprobe
pullPolicy: IfNotPresent
tag: v2.10.0
tag: v2.11.0
# -- Failure threshold for livenessProbe
failureThreshold: 5
# -- Initial delay seconds for livenessProbe
@@ -175,6 +209,36 @@ livenessprobe:
cpu: 10m
memory: 16Mi
# -- Add additional init containers for the CSI controller pods.
# ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
initContainers: []
# - name: loadbalancer
# restartPolicy: Always
# image: ghcr.io/sergelogvinov/haproxy:2.8.3-alpine3.18
# imagePullPolicy: IfNotPresent
# env:
# - name: SVC
# value: "proxmox.domain.com"
# - name: PORT
# value: "8006"
# securityContext:
# runAsUser: 99
# runAsGroup: 99
# resources:
# limits:
# cpu: 50m
# memory: 64Mi
# requests:
# cpu: 50m
# memory: 32Mi
# -- hostAliases Deployment pod host aliases
# ref: https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/
hostAliases: []
# - ip: 127.0.0.1
# hostnames:
# - proxmox.domain.com
# -- Annotations for controller pod.
# ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
podAnnotations: {}
@@ -199,13 +263,23 @@ securityContext:
type: RuntimeDefault
readOnlyRootFilesystem: true
# -- Controller deployment update stategy type.
# -- Controller deployment update strategy type.
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
# -- Prometheus metrics
metrics:
# -- Enable Prometheus metrics.
enabled: false
# -- Prometheus metrics port.
port: 8080
type: annotation
# -- Node labels for controller assignment.
# ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}
@@ -220,3 +294,15 @@ tolerations: []
# -- Affinity for controller assignment.
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}
# -- Additional volumes for Pods
extraVolumes: []
# - name: ca
# secret:
# secretName: my-ca
# -- Additional volume mounts for Pods
extraVolumeMounts: []
# - mountPath: /etc/ssl/certs/ca-certificates.crt
# name: ca
# subPath: ca.crt