Compare commits

...

1 Commits

Author SHA1 Message Date
Andrei Kvapil
a78b76f324 [linstor] Add linstor-affinity-controller
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-12-16 02:43:06 +01:00
16 changed files with 655 additions and 0 deletions

View File

@@ -0,0 +1 @@
examples

View File

@@ -0,0 +1,3 @@
apiVersion: v2
name: cozy-linstor
version: 0.0.0 # Placeholder, the actual version will be automatically set during the build process

View File

@@ -0,0 +1,10 @@
export NAME=linstor-affinity-controller
export NAMESPACE=cozy-linstor
include ../../../hack/package.mk
update:
rm -rf charts
helm repo add piraeus-charts https://piraeus.io/helm-charts/
helm repo update piraeus-charts
helm pull piraeus-charts/linstor-affinity-controller --untar --untardir charts

View File

@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

View File

@@ -0,0 +1,17 @@
apiVersion: v2
appVersion: v1.2.0
description: 'Deploys the LINSTOR Affinity Controller. It periodically checks the
state of Piraeus/LINSTOR volumes compared to PersistentVolumes (PV), and updates
the PV Affinity if changes are detected. '
home: https://github.com/piraeusdatastore/helm-charts
icon: https://raw.githubusercontent.com/piraeusdatastore/piraeus/master/artwork/sandbox-artwork/icon/color.svg
keywords:
- storage
maintainers:
- name: The Piraeus Maintainers
url: https://github.com/piraeusdatastore/
name: linstor-affinity-controller
sources:
- https://github.com/piraeusdatastore/linstor-affinity-controller
type: application
version: 1.5.0

View File

@@ -0,0 +1,72 @@
# LINSTOR Affinity Controller
The LINSTOR Affinity Controller keeps the affinity of your volumes in sync between Kubernetes and LINSTOR.
Affinity is used by Kubernetes to track on which node a specific resource can be accessed. For example, you can use
affinity to restrict access to a volume to a specific zone. While this is all supported by Piraeus and LINSTOR, and you
could tune your volumes to support almost any cluster topology, there was one important thing missing: updating affinity
after volume migration.
After the initial PersistentVolume (PV) object in Kubernetes is created, it is not possible to alter the affinity
later[^1]. This becomes a problem if your volumes need to migrate, for example if using ephemeral infrastructure, where
nodes are created and discard on demand. Using a strict affinity setting could mean that your volume is not accessible
from where you want it to: the LINSTOR resource might be there, but Kubernetes will see the volume as only accessible on
some other nodes. So you had to specify a rather relaxed affinity setting for your volumes, at the cost of less optimal
workload placement.
There is one other solution (or rather workaround): recreating your PersistentVolume whenever the backing LINSTOR
resource changed. This is where the LINSTOR Affinity Controller comes in: it automates these required steps, so that
using strict affinity just works. With strict affinity, the Kubernetes scheduler can place workloads on the same nodes
as the volumes they are using, benefiting from local data access for increased read performance.
It also enables strict affinity settings should you use ephemeral infrastructure: even if you rotate out all nodes,
your PV affinity will always match the actual volume placement in LINSTOR.
## Deployment
The best way to deploy the LINSTOR Affinity Controller is by helm chart. If deployed to the same namespace
as [our operator](https://github.com/piraeusdatastore/piraeus-operator) this is quite simple:
```
helm repo add piraeus-charts https://piraeus.io/helm-charts/
helm install linstor-affinity-controller piraeus-charts/linstor-affinity-controller
```
If deploying to a different namespace, ensure that `linstor.endpoint` and `linstor.clientSecret` are set appropriately.
For more information on the available options, see below.
### Options
The following options can be set on the chart:
| Option | Usage | Default |
|-------------------------------|----------------------------------------------------------------------------------------------|---------------------------------------------------------------|
| `replicaCount` | Number of replicas to deploy. | `1` |
| `options.v` | Set verbosity for controller | `1` |
| `options.leaderElection` | Enable leader election to coordinate betwen multiple replicas. | `true` |
| `options.reconcileRate` | Set the reconcile rate, i.e. how often the cluster state will be checked and updated | `15s` |
| `options.resyncRate` | How often the controller will resync it's internal cache of Kubernetes resources | `15m` |
| `options.propertyNamespace` | Namespace used by LINSTOR CSI to search for node labels. | `""` (auto-detected based on existing node labels on startup) |
| `linstor.endpoint` | URL of the LINSTOR Controller API. | `""` (auto-detected when using Piraeus-Operator) |
| `linstor.clientSecret` | TLS secret to use to authenticate with the LINSTOR API | `""` (auto-detected when using Piraeus-Operator) |
| `image.repository` | Repository to pull the linstor-affinity-controller image from. | `quay.io/piraeusdatastore/linstor-affinity-controller` |
| `image.pullPolicy` | Pull policy to use. Possible values: `IfNotPresent`, `Always`, `Never` | `IfNotPresent` |
| `image.tag` | Override the tag to pull. If not given, defaults to charts `AppVersion`. | `""` |
| `resources` | Resources to request and limit on the container. | `{requests: {cpu: 50m, mem: 100Mi}}` |
| `securityContext` | Configure container security context. | `{capabilities: {drop: [ALL]}, readOnlyRootFilesystem: true}` |
| `podSecurityContext` | Security context to set on the pod. | `{runAsNonRoot: true, runAsUser: 1000}` |
| `imagePullSecrets` | Image pull secrets to add to the deployment. | `[]` |
| `podAnnotations` | Annotations to add to every pod in the deployment. | `{}` |
| `nodeSelector` | Node selector to add to a pod. | `{}` |
| `tolerations` | Tolerations to add to a pod. | `[]` |
| `affinity` | Affinity to set on a pod. | `{}` |
| `rbac.create` | Create the necessary roles and bindings for the controller. | `true` |
| `serviceAccount.create` | Create the service account resource | `true` |
| `serviceAccount.name` | Sets the name of the service account. If left empty, will use the release name as default | `""` |
| `podDisruptionBudget.enabled` | Enable creation of a pod disruption budget to protect the availability of the scheduler | `true` |
| `autoscaling.enabled` | Enable creation of a horizontal pod autoscaler to ensure availability in case of high usage` | `"false` |
| `monitoring.enabled` | Enable creation of resources for monitoring via Prometheus Operator | `"false"` |
***
[^1]: That is not 100% true: you can _add_ affinity if it was previously unset, but once set, it can't be modified.

View File

@@ -0,0 +1,3 @@
LINSTOR Affinity Controller deployed.
Used LINSTOR URL: {{ include "linstor-affinity-controller.linstorEndpoint" .}}

View File

@@ -0,0 +1,160 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "linstor-affinity-controller.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "linstor-affinity-controller.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "linstor-affinity-controller.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "linstor-affinity-controller.labels" -}}
helm.sh/chart: {{ include "linstor-affinity-controller.chart" . }}
{{ include "linstor-affinity-controller.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "linstor-affinity-controller.selectorLabels" -}}
app.kubernetes.io/name: {{ include "linstor-affinity-controller.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "linstor-affinity-controller.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "linstor-affinity-controller.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
{{/*
Find the linstor client secret containing TLS certificates
*/}}
{{- define "linstor-affinity-controller.linstorClientSecretName" -}}
{{- if .Values.linstor.clientSecret }}
{{- .Values.linstor.clientSecret }}
{{- else if .Capabilities.APIVersions.Has "piraeus.io/v1/LinstorCluster" }}
{{- $crs := (lookup "piraeus.io/v1" "LinstorCluster" "" "").items }}
{{- if $crs }}
{{- if eq (len $crs) 1 }}
{{- $item := index $crs 0 }}
{{- if hasKey $item.spec "apiTLS" }}
{{- default "linstor-client-tls" $item.spec.apiTLS.clientSecretName }}
{{- end }}
{{- end }}
{{- end }}
{{- else if .Capabilities.APIVersions.Has "piraeus.linbit.com/v1/LinstorController" }}
{{- $crs := (lookup "piraeus.linbit.com/v1" "LinstorController" .Release.Namespace "").items }}
{{- if $crs }}
{{- if eq (len $crs) 1 }}
{{- $item := index $crs 0 }}
{{- $item.spec.linstorHttpsClientSecret }}
{{- end }}
{{- end }}
{{- else if .Capabilities.APIVersions.Has "linstor.linbit.com/v1/LinstorController" }}
{{- $crs := (lookup "linstor.linbit.com/v1" "LinstorController" .Release.Namespace "").items }}
{{- if $crs }}
{{- if eq (len $crs) 1 }}
{{- $item := index $crs 0 }}
{{- $item.spec.linstorHttpsClientSecret }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Find the linstor URL by operator resources
*/}}
{{- define "linstor-affinity-controller.linstorEndpointFromCRD" -}}
{{- if .Capabilities.APIVersions.Has "piraeus.io/v1/LinstorCluster" }}
{{- $crs := (lookup "piraeus.io/v1" "LinstorCluster" "" "").items }}
{{- if $crs }}
{{- if eq (len $crs) 1 }}
{{- $item := index $crs 0 }}
{{- range $index, $service := (lookup "v1" "Service" "" "").items }}
{{- if and (eq (dig "metadata" "labels" "app.kubernetes.io/component" "" $service) "linstor-controller") (eq (dig "metadata" "labels" "app.kubernetes.io/instance" "" $service) $item.metadata.name) }}
{{- if include "linstor-affinity-controller.linstorClientSecretName" $ }}
{{- printf "https://%s.%s.svc:3371" $service.metadata.name $service.metadata.namespace }}
{{- else }}
{{- printf "http://%s.%s.svc:3370" $service.metadata.name $service.metadata.namespace }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- else if .Capabilities.APIVersions.Has "piraeus.linbit.com/v1/LinstorController" }}
{{- $crs := (lookup "piraeus.linbit.com/v1" "LinstorController" .Release.Namespace "").items }}
{{- if $crs }}
{{- if eq (len $crs) 1 }}
{{- $item := index $crs 0 }}
{{- if include "linstor-affinity-controller.linstorClientSecretName" . }}
{{- printf "https://%s.%s.svc:3371" $item.metadata.name $item.metadata.namespace }}
{{- else }}
{{- printf "http://%s.%s.svc:3370" $item.metadata.name $item.metadata.namespace }}
{{- end }}
{{- end }}
{{- end }}
{{- else if .Capabilities.APIVersions.Has "linstor.linbit.com/v1/LinstorController" }}
{{- $crs := (lookup "linstor.linbit.com/v1" "LinstorController" .Release.Namespace "").items }}
{{- if $crs }}
{{- if eq (len $crs) 1 }}
{{- $item := index $crs 0 }}
{{- if include "linstor-affinity-controller.linstorClientSecretName" . }}
{{- printf "https://%s.%s.svc:3371" $item.metadata.name $item.metadata.namespace }}
{{- else }}
{{- printf "http://%s.%s.svc:3370" $item.metadata.name $item.metadata.namespace }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Find the linstor URL either by override or cluster resources
*/}}
{{- define "linstor-affinity-controller.linstorEndpoint" -}}
{{- if .Values.linstor.endpoint }}
{{- .Values.linstor.endpoint }}
{{- else }}
{{- $piraeus := include "linstor-affinity-controller.linstorEndpointFromCRD" . }}
{{- if $piraeus }}
{{- $piraeus }}
{{- else }}
{{- fail "Please specify linstor.endpoint, no default URL could be determined" }}
{{- end }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,98 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "linstor-affinity-controller.fullname" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "linstor-affinity-controller.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "linstor-affinity-controller.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "linstor-affinity-controller.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
args:
- /linstor-affinity-controller
{{- if .Values.monitoring }}
- --metrics-address=:8001
{{- end }}
{{- range $opt, $val := .Values.options }}
- --{{ $opt | kebabcase }}={{ $val }}
{{- end }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- if .Values.monitoring }}
ports:
- name: metrics
protocol: TCP
containerPort: 8001
{{- end }}
env:
- name: LEASE_LOCK_NAME
value: {{ include "linstor-affinity-controller.fullname" . }}
- name: LEASE_HOLDER_IDENTITY
valueFrom:
fieldRef:
fieldPath: metadata.name
apiVersion: v1
- name: LS_CONTROLLERS
value: {{ include "linstor-affinity-controller.linstorEndpoint" . }}
{{- if include "linstor-affinity-controller.linstorClientSecretName" . }}
- name: LS_USER_CERTIFICATE
valueFrom:
secretKeyRef:
name: {{ include "linstor-affinity-controller.linstorClientSecretName" . }}
key: tls.crt
- name: LS_USER_KEY
valueFrom:
secretKeyRef:
name: {{ include "linstor-affinity-controller.linstorClientSecretName" . }}
key: tls.key
- name: LS_ROOT_CA
valueFrom:
secretKeyRef:
name: {{ include "linstor-affinity-controller.linstorClientSecretName" . }}
key: ca.crt
{{- end }}
readinessProbe:
httpGet:
port: 8000
path: /readyz
livenessProbe:
httpGet:
port: 8000
path: /healthz
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@@ -0,0 +1,32 @@
{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "linstor-affinity-controller.fullname" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ include "linstor-affinity-controller.fullname" . }}
minReplicas: {{ .Values.autoscaling.minReplicas }}
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
metrics:
{{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
{{- end }}
{{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,30 @@
{{ if .Values.monitoring.enabled }}
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "linstor-affinity-controller.fullname" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
spec:
endpoints:
- port: metrics
selector:
matchLabels:
{{- include "linstor-affinity-controller.selectorLabels" . | nindent 6 }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "linstor-affinity-controller.fullname" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
spec:
selector:
{{- include "linstor-affinity-controller.selectorLabels" . | nindent 4 }}
ports:
- name: metrics
port: 8001
targetPort: metrics
protocol: TCP
{{ end }}

View File

@@ -0,0 +1,18 @@
{{- if .Values.podDisruptionBudget.enabled -}}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "linstor-affinity-controller.fullname" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
spec:
selector:
matchLabels:
{{- include "linstor-affinity-controller.selectorLabels" . | nindent 6 }}
{{- if .Values.podDisruptionBudget.minAvailable }}
minAvailable: {{ .Values.podDisruptionBudget.minAvailable }}
{{- end }}
{{- if .Values.podDisruptionBudget.maxUnavailable }}
maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable }}
{{- end }}
{{- end -}}

View File

@@ -0,0 +1,99 @@
{{- if .Values.rbac.create }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
rules:
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- "storage.k8s.io"
resources:
- storageclasses
verbs:
- get
- apiGroups:
- events.k8s.io
resources:
- events
verbs:
- create
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
subjects:
- kind: ServiceAccount
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- if .Values.options.leaderElection }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
subjects:
- kind: ServiceAccount
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,12 @@
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "linstor-affinity-controller.serviceAccountName" . }}
labels:
{{- include "linstor-affinity-controller.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,73 @@
replicaCount: 1
linstor:
endpoint: ""
clientSecret: ""
image:
repository: quay.io/piraeusdatastore/linstor-affinity-controller
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
imagePullSecrets: [ ]
nameOverride: ""
fullnameOverride: ""
options:
v: 1
leaderElection: true
#propertyNamespace: ""
#reconcileRate: 15s
#resyncRate: 15m
#workers: 10
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: { }
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
rbac:
# Specifies whether RBAC resources should be created
create: true
podAnnotations: { }
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
securityContext:
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
resources:
requests:
cpu: 50m
memory: 100Mi
nodeSelector: { }
tolerations: []
affinity: { }
podDisruptionBudget:
enabled: true
minAvailable: 1
# maxUnavailable: 1
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 3
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
monitoring:
enabled: false

View File

@@ -0,0 +1,4 @@
linstor-affinity-controller:
linstor:
endpoint: "https://linstor-controller.cozy-linstor.svc:3371"
clientSecret: "linstor-client-tls"