feat: daemonset deployment

Deploy CCM as DaemonSet.
In this case CCM will use local node to connect to the Talos api.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
This commit is contained in:
Serge Logvinov
2023-03-11 15:22:03 +02:00
parent 5a4413f353
commit 86818165f5
8 changed files with 465 additions and 24 deletions

View File

@@ -76,6 +76,11 @@ docs:
helm template -n kube-system talos-cloud-controller-manager \
--set-string image.tag=$(TAG) \
charts/talos-cloud-controller-manager > docs/deploy/cloud-controller-manager.yml
helm template -n kube-system talos-cloud-controller-manager \
--set-string image.tag=$(TAG) \
--set useDaemonSet=true \
charts/talos-cloud-controller-manager > docs/deploy/cloud-controller-manager-daemonset.yml
helm-docs charts/talos-cloud-controller-manager
git-chglog --config hack/chglog-config.yml -o CHANGELOG.md

View File

@@ -1,6 +1,6 @@
apiVersion: v2
appVersion: "1.3.0"
version: 0.0.1
version: 0.1.0
kubeVersion: ">= 1.24.0"
type: application

View File

@@ -1,5 +1,52 @@
# talos-cloud-controller-manager
Deploys the Talos Cloud Controller Manager to your cluster.
![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.3.0](https://img.shields.io/badge/AppVersion-1.3.0-informational?style=flat-square)
## How To install
Talos Cloud Controller Manager Helm Chart
**Homepage:** <https://github.com/siderolabs/talos-cloud-controller-manager>
## Maintainers
| Name | Email | Url |
| ---- | ------ | --- |
| sergelogvinov | | <https://github.com/sergelogvinov> |
## Requirements
Kubernetes: `>= 1.24.0`
## Values
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| affinity | object | `{}` | Affinity for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity |
| enabledControllers | list | `["cloud-node"]` | List of controllers should be enabled. Use '*' to enable all controllers. Support only `cloud-node` controller. |
| extraArgs | list | `[]` | Any extra arguments for talos-cloud-controller-manager |
| fullnameOverride | string | `""` | String to fully override deployment name. |
| image.pullPolicy | string | `"IfNotPresent"` | Pull policy: IfNotPresent or Always. |
| image.repository | string | `"ghcr.io/siderolabs/talos-cloud-controller-manager"` | CCM image repository. |
| image.tag | string | `""` | Overrides the image tag whose default is the chart appVersion. |
| imagePullSecrets | list | `[]` | Optionally specify an array of imagePullSecrets. Secrets must be manually created in the namespace. ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ |
| logVerbosityLevel | int | `2` | Log verbosity level. See https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md for description of individual verbosity levels. |
| nameOverride | string | `""` | String to partially override deployment name. |
| nodeSelector | object | `{"node-role.kubernetes.io/control-plane":""}` | Node labels for data pods assignment. ref: https://kubernetes.io/docs/user-guide/node-selection/ |
| podAnnotations | object | `{}` | Annotations for data pods. ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ |
| podSecurityContext | object | `{"fsGroup":10258,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":10258,"runAsNonRoot":true,"runAsUser":10258}` | Pods Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
| priorityClassName | string | `"system-cluster-critical"` | CCM pods' priorityClassName. |
| replicaCount | int | `1` | Number of CCM replicas to deploy. |
| resources | object | `{"requests":{"cpu":"10m","memory":"64Mi"}}` | Resource requests and limits. ref: http://kubernetes.io/docs/user-guide/compute-resources/ |
| securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}` | Container Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
| service.annotations | object | `{}` | Additional custom annotations for Service. |
| service.containerPort | int | `50258` | Container HTTPS port. |
| service.port | int | `50258` | Service HTTPS port to expose controller. |
| serviceAccount | object | `{"annotations":{},"create":true,"name":""}` | Pods Service Account. ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ |
| serviceAccount.annotations | object | `{}` | Annotations to add to the service account. |
| serviceAccount.create | bool | `true` | Specifies whether a service account should be created. |
| serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the fullname template. |
| tolerations | list | `[{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane","operator":"Exists"},{"effect":"NoSchedule","key":"node.cloudprovider.kubernetes.io/uninitialized","operator":"Exists"}]` | Tolerations for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ |
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Deployment update stategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
| useDaemonSet | bool | `false` | Deploy CCM in Daemonset mode. |
----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)

View File

@@ -1,12 +1,23 @@
apiVersion: apps/v1
{{- if .Values.useDaemonSet }}
kind: DaemonSet
{{- else }}
kind: Deployment
{{- end }}
metadata:
name: {{ include "talos-cloud-controller-manager.fullname" . }}
labels:
{{- include "talos-cloud-controller-manager.labels" . | nindent 4 }}
namespace: {{ .Release.Namespace }}
spec:
{{- if not .Values.useDaemonSet }}
replicas: {{ .Values.replicaCount }}
strategy:
type: {{ .Values.updateStrategy.type }}
{{- else }}
updateStrategy:
type: {{ .Values.updateStrategy.type }}
{{- end }}
selector:
matchLabels:
{{- include "talos-cloud-controller-manager.selectorLabels" . | nindent 6 }}
@@ -26,6 +37,10 @@ spec:
serviceAccountName: {{ include "talos-cloud-controller-manager.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
{{- if .Values.useDaemonSet }}
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true
{{- end }}
containers:
- name: {{ .Chart.Name }}
securityContext:
@@ -40,18 +55,25 @@ spec:
- --controllers={{- trimAll "," (include "talos-cloud-controller-manager.enabledControllers" . ) }}
- --leader-elect-resource-name=cloud-controller-manager-talos
- --use-service-account-credentials
- --secure-port=10258
- --secure-port={{ .Values.service.containerPort }}
{{- with .Values.extraArgs }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if .Values.useDaemonSet }}
env:
- name: TALOS_ENDPOINTS
valueFrom:
fieldRef:
fieldPath: status.podIP
{{- end }}
ports:
- containerPort: 10258
- containerPort: {{ .Values.service.containerPort }}
name: https
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: 10258
port: https
scheme: HTTPS
initialDelaySeconds: 20
periodSeconds: 30

View File

@@ -0,0 +1,21 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "talos-cloud-controller-manager.serviceAccountName" . }}
labels:
{{- include "talos-cloud-controller-manager.labels" . | nindent 4 }}
{{- with .Values.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
namespace: {{ .Release.Namespace }}
spec:
clusterIP: None
type: ClusterIP
ports:
- name: https
port: {{ .Values.service.port }}
targetPort: {{ .Values.service.containerPort }}
protocol: TCP
selector:
{{- include "talos-cloud-controller-manager.selectorLabels" . | nindent 8 }}

View File

@@ -2,49 +2,63 @@
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
# -- Number of CCM replicas to deploy.
replicaCount: 1
image:
# -- CCM image repository.
repository: ghcr.io/siderolabs/talos-cloud-controller-manager
# -- Pull policy: IfNotPresent or Always.
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
# -- Overrides the image tag whose default is the chart appVersion.
tag: ""
# -- Optionally specify an array of imagePullSecrets.
# Secrets must be manually created in the namespace.
# ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
imagePullSecrets: []
# -- String to partially override deployment name.
nameOverride: ""
# -- String to fully override deployment name.
fullnameOverride: ""
# Any extra arguments for talos-cloud-controller-manager
# -- Any extra arguments for talos-cloud-controller-manager
extraArgs: []
# - --cluster-name=kubernetes
# List of controllers should be enabled.
# -- List of controllers should be enabled.
# Use '*' to enable all controllers.
# Prefix a controller with '-' to disable it.
# Support only `cloud-node` controller.
enabledControllers:
- cloud-node
# - cloud-node-lifecycle
# - route
# - service
# Log verbosity level.
# See https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md
# -- Log verbosity level. See https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md
# for description of individual verbosity levels.
logVerbosityLevel: 2
# -- Pods Service Account.
# ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
serviceAccount:
# Specifies whether a service account should be created
# -- Specifies whether a service account should be created.
create: true
# Annotations to add to the service account
# -- Annotations to add to the service account.
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
# -- The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template.
name: ""
# -- CCM pods' priorityClassName.
priorityClassName: system-cluster-critical
# -- Annotations for data pods.
# ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
podAnnotations: {}
# -- Pods Security Context.
# ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod
podSecurityContext:
runAsNonRoot: true
runAsUser: 10258
@@ -52,6 +66,8 @@ podSecurityContext:
fsGroup: 10258
fsGroupChangePolicy: "OnRootMismatch"
# -- Container Security Context.
# ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod
securityContext:
allowPrivilegeEscalation: false
capabilities:
@@ -60,6 +76,16 @@ securityContext:
seccompProfile:
type: RuntimeDefault
service:
# -- Service HTTPS port to expose controller.
port: 50258
# -- Container HTTPS port.
containerPort: 50258
# -- Additional custom annotations for Service.
annotations: {}
# -- Resource requests and limits.
# ref: http://kubernetes.io/docs/user-guide/compute-resources/
resources:
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
@@ -72,9 +98,23 @@ resources:
cpu: 10m
memory: 64Mi
# -- Deploy CCM in Daemonset mode.
useDaemonSet: false
# -- Deployment update stategy type.
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
# -- Node labels for data pods assignment.
# ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector:
node-role.kubernetes.io/control-plane: ""
# -- Tolerations for data pods assignment.
# ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
@@ -83,4 +123,6 @@ tolerations:
key: node.cloudprovider.kubernetes.io/uninitialized
operator: Exists
# -- Affinity for data pods assignment.
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}

View File

@@ -0,0 +1,278 @@
---
# Source: talos-cloud-controller-manager/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
namespace: kube-system
---
# Source: talos-cloud-controller-manager/templates/serviceaccount.yaml
apiVersion: talos.dev/v1alpha1
kind: ServiceAccount
metadata:
name: talos-cloud-controller-manager-talos-secrets
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
namespace: kube-system
spec:
roles:
- os:reader
---
# Source: talos-cloud-controller-manager/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
namespace: kube-system
data:
ccm-config.yaml: |
global:
---
# Source: talos-cloud-controller-manager/templates/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- create
- update
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- update
- patch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- create
- get
- apiGroups:
- ""
resources:
- serviceaccounts/token
verbs:
- create
# - apiGroups:
# - certificates.k8s.io
# resources:
# - certificatesigningrequests
# verbs:
# - list
# - watch
# - apiGroups:
# - certificates.k8s.io
# resources:
# - certificatesigningrequests/approval
# verbs:
# - update
# - apiGroups:
# - certificates.k8s.io
# resources:
# - signers
# resourceNames:
# - kubernetes.io/kubelet-serving
# verbs:
# - approve
---
# Source: talos-cloud-controller-manager/templates/rolebinding.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:talos-cloud-controller-manager
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:talos-cloud-controller-manager
subjects:
- kind: ServiceAccount
name: talos-cloud-controller-manager
namespace: kube-system
---
# Source: talos-cloud-controller-manager/templates/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: system:talos-cloud-controller-manager:extension-apiserver-authentication-reader
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
name: talos-cloud-controller-manager
namespace: kube-system
---
# Source: talos-cloud-controller-manager/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
namespace: kube-system
spec:
clusterIP: None
type: ClusterIP
ports:
- name: https
port: 50258
targetPort: 50258
protocol: TCP
selector:
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
---
# Source: talos-cloud-controller-manager/templates/deployment.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
namespace: kube-system
spec:
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
template:
metadata:
labels:
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
spec:
serviceAccountName: talos-cloud-controller-manager
securityContext:
fsGroup: 10258
fsGroupChangePolicy: OnRootMismatch
runAsGroup: 10258
runAsNonRoot: true
runAsUser: 10258
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true
containers:
- name: talos-cloud-controller-manager
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
image: "ghcr.io/siderolabs/talos-cloud-controller-manager:v1.3.0"
imagePullPolicy: IfNotPresent
command: ["/talos-cloud-controller-manager"]
args:
- --v=2
- --cloud-provider=talos
- --cloud-config=/etc/talos/ccm-config.yaml
- --controllers=cloud-node
- --leader-elect-resource-name=cloud-controller-manager-talos
- --use-service-account-credentials
- --secure-port=50258
env:
- name: TALOS_ENDPOINTS
valueFrom:
fieldRef:
fieldPath: status.podIP
ports:
- containerPort: 50258
name: https
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: https
scheme: HTTPS
initialDelaySeconds: 20
periodSeconds: 30
timeoutSeconds: 5
resources:
requests:
cpu: 10m
memory: 64Mi
volumeMounts:
- name: cloud-config
mountPath: /etc/talos
readOnly: true
- name: talos-secrets
mountPath: /var/run/secrets/talos.dev
readOnly: true
nodeSelector:
node-role.kubernetes.io/control-plane: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Exists
- effect: NoSchedule
key: node.cloudprovider.kubernetes.io/uninitialized
operator: Exists
volumes:
- name: cloud-config
configMap:
name: talos-cloud-controller-manager
defaultMode: 416 # 0640
- name: talos-secrets
secret:
secretName: talos-cloud-controller-manager-talos-secrets
defaultMode: 416 # 0640

View File

@@ -5,7 +5,7 @@ kind: ServiceAccount
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.0.1
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
@@ -18,7 +18,7 @@ kind: ServiceAccount
metadata:
name: talos-cloud-controller-manager-talos-secrets
labels:
helm.sh/chart: talos-cloud-controller-manager-0.0.1
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
@@ -34,7 +34,7 @@ kind: ConfigMap
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.0.1
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
@@ -50,7 +50,7 @@ kind: ClusterRole
metadata:
name: system:talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.0.1
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
@@ -152,13 +152,37 @@ subjects:
name: talos-cloud-controller-manager
namespace: kube-system
---
# Source: talos-cloud-controller-manager/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
app.kubernetes.io/managed-by: Helm
namespace: kube-system
spec:
clusterIP: None
type: ClusterIP
ports:
- name: https
port: 50258
targetPort: 50258
protocol: TCP
selector:
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
---
# Source: talos-cloud-controller-manager/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: talos-cloud-controller-manager
labels:
helm.sh/chart: talos-cloud-controller-manager-0.0.1
helm.sh/chart: talos-cloud-controller-manager-0.1.0
app.kubernetes.io/name: talos-cloud-controller-manager
app.kubernetes.io/instance: talos-cloud-controller-manager
app.kubernetes.io/version: "1.3.0"
@@ -166,6 +190,8 @@ metadata:
namespace: kube-system
spec:
replicas: 1
strategy:
type: RollingUpdate
selector:
matchLabels:
app.kubernetes.io/name: talos-cloud-controller-manager
@@ -202,15 +228,15 @@ spec:
- --controllers=cloud-node
- --leader-elect-resource-name=cloud-controller-manager-talos
- --use-service-account-credentials
- --secure-port=10258
- --secure-port=50258
ports:
- containerPort: 10258
- containerPort: 50258
name: https
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: 10258
port: https
scheme: HTTPS
initialDelaySeconds: 20
periodSeconds: 30