initial commit

Chart.lock

@@ -0,0 +1,6 @@
+dependencies:
+- name: postgresql-ha
+  repository: https://charts.bitnami.com/bitnami
+  version: 7.11.0
+digest: sha256:849759b9fd9d89bf0d47a271334889601010d1d11dd5c00562c18feafd93356d
+generated: "2021-10-13T14:02:45.428151972-04:00"

Chart.yaml

@@ -0,0 +1,29 @@
+apiVersion: v2
+name: netmaker
+description: A Helm chart to run HA Netmaker on Kubernetes
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "0.9.0"
+
+dependencies:
+  - name: "postgresql-ha"
+    version: "7.11.0"
+    repository: https://charts.bitnami.com/bitnami

README.md

@@ -0,0 +1,124 @@
+# Netmaker Helm
+
+![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.9.0](https://img.shields.io/badge/AppVersion-0.9.0-informational?style=flat-square)
+
+A Helm chart to run Netmaker with High Availability on Kubernetes
+
+## Requirements
+
+To run HA Netmaker on Kubernetes, your cluster must have the following:
+- RWO and RWX Storage Classes (RWX is only required if running Netmaker with DNS Management enabled).
+- An Ingress Controller and valid TLS certificates 
+	- This chart can currently generate ingress for:
+		- Nginx Ingress + LetsEncrypt/Cert-Manager
+		- Traefik Ingress + LetsEncrypt/Cert-Manager
+	- to generate automatically, make sure one of the two is configured for your cluster
+
+Furthermore, the chart will by default install and use a postgresql cluster as its datastore: 
+
+| Repository | Name | Version |
+|------------|------|---------|
+| https://charts.bitnami.com/bitnami | postgresql-ha | 7.11.0 |
+
+### Example Install
+
+```
+helm install ./netmaker --generate-name \ # generate a random id for the deploy 
+--set baseDomain=nm.example.com \ # the base wildcard domain to use for the netmaker api/dashboard/grpc ingress 
+--set replicas=3 \ # number of server replicas to deploy (3 by default) 
+--set ingress.enabled=true \ # deploy ingress automatically (requires nginx or traefik and cert-manager + letsencrypt) 
+--set ingress.className=nginx \ # ingress class to use 
+--set ingress.tls.issuerName=letsencrypt-prod \ # LetsEncrypt certificate issuer to use 
+--set dns.enabled=true \ # deploy and enable private DNS management with CoreDNS 
+--set dns.clusterIP=10.245.75.75 --set dns.RWX.storageClassName=nfs \ # required fields for DNS 
+--set postgresql-ha.postgresql.replicaCount=2 \ # number of DB replicas to deploy (default 2)
+```
+
+### Recommended Settings:
+A minimal HA install of Netmaker can be run with the following command:
+`helm install netmaker --generate-name --set baseDomain=nm.example.com`
+This install has some notable exceptions:
+- Ingress **must** be manually configured post-install (need to create valid Ingress with TLS)
+- Server will use "userspace" WireGuard, which is slower than kernel WG
+- DNS will be disabled
+
+Below, we discuss the considerations for Ingress, Kernel WireGuard, and DNS.
+
+#### Ingress	
+To run HA Netmaker, you must have ingress installed and enabled on your cluster with valid TLS certificates (not self-signed). If you are running Nginx as your Ingress Controller and LetsEncrypt for TLS certificate management, you can run the helm install with the following settings:
+`--set ingress.enabled=true`
+`--set ingress.annotations.cert-manager.io/cluster-issuer=<your LE issuer name>`
+
+If you are not using Nginx and LetsEncrypt, we recommend leaving ingress.enabled=false (default), and then manually creating the ingress objects post-install. You will need three ingress objects with TLS:
+`dashboard.<baseDomain>`
+`api.<baseDomain>`
+`grpc.<baseDomain>`
+
+The gRPC ingress object must include annotations to use the gRPC protocol, which is supported by most ingress controllers. For instance, on Traefik, the annotation is:
+`ingress.kubernetes.io/protocol: h2c`
+
+You can find example ingress objects in the kube/example folder.
+
+#### Kernel WireGuard
+If you have control of the Kubernetes worker node servers, we recommend **first** installing WireGuard on the hosts, and then installing HA Netmaker in Kernel mode. By default, Netmaker will install with userspace WireGuard (wireguard-go) for maximum compatibility, and to avoid needing permissions at the host level. If you have installed WireGuard on your hosts, you should install Netmaker's helm chart with the following option:
+`--set wireguard.kernel=true`
+
+#### DNS
+By Default, the helm chart will deploy without DNS enabled. To enable DNS, specify with:
+`--set dns.enabled=true` 
+This will require specifying a RWX storage class, e.g.:
+`--set dns.RWX.storageClassName=nfs`
+This will also require specifying a service address for DNS. Choose a valid ipv4 address from the service IP CIDR for your cluster, e.g.:
+`--set dns.clusterIP=10.245.69.69`
+
+**This address will only be reachable from hosts that have access to the cluster service CIDR.** It is only designed for use cases related to k8s. If you want a more general-use Netmaker server on Kubernetes for use cases outside of k8s, you will need to do one of the following:
+- bind the CoreDNS service to port 53 on one of your worker nodes and set the COREDNS_ADDRESS equal to the public IP of the worker node
+- Create a private Network with Netmaker and set the COREDNS_ADDRESS equal to the private address of the host running CoreDNS. For this, CoreDNS will need a node selector and will ideally run on the same host as one of the Netmaker server instances.
+
+
+
+## Values
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| dns.enabled | bool | `false` | whether or not to run with DNS (CoreDNS) |
+| dns.storageSize | string | `"128Mi"` | volume size for DNS (only needs to hold one file) |
+| fullnameOverride | string | `""` | override the full name for netmaker objects  |
+| image.pullPolicy | string | `"Always"` | Pull Policy for images |
+| image.repository | string | `"gravitl/netmaker"` | The image repo to pull Netmaker image from  |
+| image.tag | string | `"v0.8.4"` | Override the image tag to pull  |
+| ingress.annotations.base."kubernetes.io/ingress.allow-http" | string | `"false"` | annotation to generate ACME certs if available |
+| ingress.annotations.grpc.nginx."nginx.ingress.kubernetes.io/backend-protocol" | string | `"GRPC"` | annotation to use grpc protocol on grpc domain |
+| ingress.annotations.grpc.traefik."ingress.kubernetes.io/protocol" | string | `"h2c"` | annotation to use grpc protocol on grpc domain |
+| ingress.annotations.nginx."nginx.ingress.kubernetes.io/rewrite-target" | string | `"/"` | destination addr for route |
+| ingress.annotations.nginx."nginx.ingress.kubernetes.io/ssl-redirect" | string | `"true"` | Redirect http to https  |
+| ingress.annotations.tls."kubernetes.io/tls-acme" | string | `"true"` | use acme cert if available |
+| ingress.annotations.traefik."traefik.ingress.kubernetes.io/redirect-entry-point" | string | `"https"` | Redirect to https |
+| ingress.annotations.traefik."traefik.ingress.kubernetes.io/redirect-permanent" | string | `"true"` | Redirect to https permanently |
+| ingress.annotations.traefik."traefik.ingress.kubernetes.io/rule-type" | string | `"PathPrefixStrip"` | rule type |
+| ingress.enabled | bool | `false` | attempts to configure ingress if true |
+| ingress.hostPrefix.grpc | string | `"grpc."` | grpc route subdomain |
+| ingress.hostPrefix.rest | string | `"api."` | api (REST) route subdomain |
+| ingress.hostPrefix.ui | string | `"dashboard."` | ui route subdomain |
+| ingress.tls.enabled | bool | `true` |  |
+| ingress.tls.issuerName | string | `"letsencrypt-prod"` |  |
+| nameOverride | string | `""` | override the name for netmaker objects  |
+| podAnnotations | object | `{}` | pod annotations to add |
+| podSecurityContext | object | `{}` | pod security contect to add |
+| postgresql-ha.persistence.size | string | `"3Gi"` | size of postgres DB |
+| postgresql-ha.postgresql.database | string | `"netmaker"` | postgress db to generate |
+| postgresql-ha.postgresql.password | string | `"netmaker"` | postgres pass to generate |
+| postgresql-ha.postgresql.username | string | `"netmaker"` | postgres user to generate |
+| replicas | int | `3` | number of netmaker server replicas to create  |
+| service.grpcPort | int | `443` | port for GRPC service |
+| service.restPort | int | `8081` | port for API service |
+| service.type | string | `"ClusterIP"` | type for netmaker server services |
+| service.uiPort | int | `80` | port for UI service |
+| serviceAccount.annotations | object | `{}` | Annotations to add to the service account |
+| serviceAccount.create | bool | `true` | Specifies whether a service account should be created |
+| serviceAccount.name | string | `""` | Name of SA to use. If not set and create is true, a name is generated using the fullname template |
+| ui.replicas | int | `2` | how many UI replicas to create |
+| wireguard.enabled | bool | `true` | whether or not to use WireGuard on server |
+| wireguard.kernel | bool | `false` | whether or not to use Kernel WG (should be false unless WireGuard is installed on hosts). |
+| wireguard.networkLimit | int | `10` | max number of networks that Netmaker will support if running with WireGuard enabled |
+

index.yaml

@@ -0,0 +1,53 @@
+apiVersion: v1
+entries:
+  netmaker:
+  - apiVersion: v2
+    appVersion: 0.9.0
+    created: "2021-10-19T11:43:01.375725537-04:00"
+    dependencies:
+    - name: postgresql-ha
+      repository: https://charts.bitnami.com/bitnami
+      version: 7.11.0
+    description: A Helm chart to run HA Netmaker on Kubernetes
+    digest: 0a9cf7d64d69fb6e2604ea0d42ba762899c423640bd554539359095939647f9c
+    name: netmaker
+    type: application
+    urls:
+    - https://gravitl.github.io/netmaker-helm/netmaker-0.1.0.tgz
+    version: 0.1.0
+  postgresql-ha:
+  - annotations:
+      category: Database
+    apiVersion: v2
+    appVersion: 11.13.0
+    created: "2021-10-19T11:43:01.386221323-04:00"
+    dependencies:
+    - name: common
+      repository: https://charts.bitnami.com/bitnami
+      version: 1.x.x
+    description: Chart for PostgreSQL with HA architecture (using Replication Manager
+      (repmgr) and Pgpool).
+    digest: c06171bd61488b019ce3d85f3100014ca810d493a4bc778412775ac126c163ca
+    home: https://github.com/bitnami/charts/tree/master/bitnami/postgresql-ha
+    icon: https://bitnami.com/assets/stacks/postgresql/img/postgresql-stack-220x234.png
+    keywords:
+    - postgresql
+    - repmgr
+    - pgpool
+    - postgres
+    - database
+    - sql
+    - replication
+    - cluster
+    - high availability
+    maintainers:
+    - email: containers@bitnami.com
+      name: Bitnami
+    name: postgresql-ha
+    sources:
+    - https://github.com/bitnami/bitnami-docker-postgresql
+    - https://www.postgresql.org/
+    urls:
+    - https://gravitl.github.io/netmaker-helm/charts/postgresql-ha-7.11.0.tgz
+    version: 7.11.0
+generated: "2021-10-19T11:43:01.366929589-04:00"

templates/NOTES.txt

@@ -0,0 +1,22 @@
+1. Get the application URL by running these commands:
+{{- if .Values.ingress.enabled }}
+{{- range $host := .Values.ingress.hosts }}
+  {{- range .paths }}
+  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
+  {{- end }}
+{{- end }}
+{{- else if contains "NodePort" .Values.service.type }}
+  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "netmaker.fullname" . }})
+  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo http://$NODE_IP:$NODE_PORT
+{{- else if contains "LoadBalancer" .Values.service.type }}
+     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
+           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "netmaker.fullname" . }}'
+  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "netmaker.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
+  echo http://$SERVICE_IP:{{ .Values.service.port }}
+{{- else if contains "ClusterIP" .Values.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "netmaker.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
+  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
+  echo "Visit http://127.0.0.1:8080 to use your application"
+  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
+{{- end }}

templates/_helpers.tpl

@@ -0,0 +1,70 @@
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "netmaker.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "netmaker.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "netmaker.masterKey" -}}
+{{- randAlphaNum 12 | nospace -}}
+{{- end -}}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "netmaker.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "netmaker.labels" -}}
+helm.sh/chart: {{ include "netmaker.chart" . }}
+{{ include "netmaker.selectorLabels" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "netmaker.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "netmaker.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "netmaker.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "netmaker.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}

templates/coredns.yaml

@@ -0,0 +1,85 @@
+{{- if .Values.dns.enabled -}}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ include "netmaker.fullname" . }}-coredns
+  labels:
+    app: {{ include "netmaker.fullname" . }}-coredns
+spec:
+  selector:
+    matchLabels:
+      app: {{ include "netmaker.fullname" . }}-coredns
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        app: {{ include "netmaker.fullname" . }}-coredns
+    spec:
+      containers:
+      - args:
+        - -conf
+        - /root/dnsconfig/Corefile
+        image: coredns/coredns
+        imagePullPolicy: Always
+        name: netmaker-dns
+        ports:
+        - containerPort: 53
+          name: dns
+          protocol: UDP
+        - containerPort: 53
+          name: dns-tcp
+          protocol: TCP
+        volumeMounts:
+        - mountPath: /root/dnsconfig
+          name: {{ include "netmaker.fullname" . }}-dns-pvc
+          readOnly: true
+        securityContext:
+          allowPrivilegeEscalation: false
+          capabilities:
+            add:
+            - NET_BIND_SERVICE
+            drop:
+            - all
+      dnsPolicy: "None"
+      dnsConfig:
+        nameservers:
+          - 127.0.0.1
+      volumes:
+      - name: {{ include "netmaker.fullname" . }}-dns-pvc
+        persistentVolumeClaim:
+          claimName: {{ include "netmaker.fullname" . }}-dns-pvc
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    app: {{ include "netmaker.fullname" . }}-coredns
+  name: {{ include "netmaker.fullname" . }}-coredns
+spec:
+  ports:
+  - port: 53
+    protocol: UDP
+    targetPort: 53
+    name: udp
+  - port: 53
+    protocol: TCP
+    targetPort: 53
+    name: tcp
+  selector:
+    app: {{ include "netmaker.fullname" . }}-coredns
+  sessionAffinity: None
+  type: ClusterIP
+  clusterIP: {{ required "A valid .Values.dns.clusterIP entry required! Choose an IP from your k8s service IP CIDR" .Values.dns.clusterIP}}
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ include "netmaker.fullname" . }}-dns-pvc
+spec:
+  storageClassName: {{ required "A valid .Values.dns.RWX.storageClassName entry required! Specify an available RWX storage class." .Values.dns.RWX.storageClassName}}
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: {{ .Values.dns.storageSize }}
+{{- end }}

templates/ingress.yaml

@@ -0,0 +1,236 @@
+{{- if .Values.ingress.enabled -}}
+{{- $fullName := include "netmaker.fullname" . -}}
+{{- $fullUIName := printf "%s-%s" $fullName "ui" -}}
+{{- $fullRESTName := printf "%s-%s" $fullName "rest" -}}
+{{- $fullGRPCName := printf "%s-%s" $fullName "grpc" -}}
+{{- $uiSvcPort := .Values.service.uiPort -}}
+{{- $restSvcPort := .Values.service.restPort -}}
+{{- $grpcSvcPort := .Values.service.grpcPort -}}
+{{- $classname := required "A valid .Values.ingress.className entry required! Please set this to your ingress class (nginx, traefik)" .Values.ingress.className}}
+{{- if and .Values.ingress.className (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) }}
+  {{- if not (hasKey .Values.ingress.annotations "kubernetes.io/ingress.class") }}
+  {{- $_ := set .Values.ingress.annotations "kubernetes.io/ingress.class" .Values.ingress.className}}
+  {{- end }}
+{{- end }}
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullUIName }}
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  {{- with .Values.ingress }}
+  annotations:
+    {{- toYaml .annotations.base | nindent 4 }}
+    {{- if or (eq .className "nginx") (eq .className "public") }}
+    {{- toYaml .annotations.nginx | nindent 4 }}
+    {{- end }}
+    {{- if eq .className "traefik" }}
+    {{- toYaml .annotations.traefik | nindent 4 }}
+    {{- end }}
+    {{- if and .tls.enabled (eq .tls.issuerName "" )}}
+    {{- toYaml .annotations.tls | nindent 4 }}
+    {{- else if .tls.enabled}}
+    cert-manager.io/cluster-issuer: {{ .tls.issuerName }}
+    {{- end }}
+  {{- end }}
+spec:
+  {{- if (not (eq .Values.ingress.className "traefik")) }}
+  {{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }}
+  ingressClassName: {{ required "A valid .Values.ingress.className entry required!" .Values.ingress.className}}
+  {{- end }}
+  {{- end }}
+  {{- if .Values.ingress.tls.enabled }}
+  tls:
+    - hosts:
+        - {{ .Values.ingress.hostPrefix.ui }}{{ .Values.baseDomain }}
+      secretName: {{ $fullUIName }}-tls-secret
+  {{- end}}
+  rules:
+    - host: {{ .Values.ingress.hostPrefix.ui }}{{ .Values.baseDomain }}
+      http:
+        paths:
+          - path: /
+            {{- if (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }}
+            pathType: Prefix
+            {{- end }}
+            backend:
+              {{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
+              service:
+                name: {{ $fullUIName }}
+                port:
+                  number: {{ $uiSvcPort }}
+              {{- else }}
+              serviceName: {{ $fullUIName }}
+              servicePort: {{ $uiSvcPort }}
+              {{- end }}
+---
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullRESTName }}
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  {{- with .Values.ingress }}
+  annotations:
+    {{- toYaml .annotations.base | nindent 4 }}
+    {{- if or (eq .className "nginx") (eq .className "public") }}
+    {{- toYaml .annotations.nginx | nindent 4 }}
+    {{- end }}
+    {{- if eq .className "traefik" }}
+    {{- toYaml .annotations.traefik | nindent 4 }}
+    {{- end }}
+    {{- if and .tls.enabled (eq .tls.issuerName "" )}}
+    {{- toYaml .annotations.tls | nindent 4 }}
+    {{- else if .tls.enabled}}
+    cert-manager.io/cluster-issuer: {{ .tls.issuerName }}
+    {{- end }}
+  {{- end }}
+spec:
+  {{- if (not (eq .Values.ingress.className "traefik")) }}
+  {{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }}
+  ingressClassName:  {{ required "A valid .Values.ingress.className entry required!" .Values.ingress.className}}
+  {{- end }}
+  {{- end }}
+  {{- if .Values.ingress.tls.enabled }}
+  tls:
+    - hosts:
+        - {{ .Values.ingress.hostPrefix.rest }}{{ .Values.baseDomain }}
+      secretName: {{ $fullRESTName }}-tls-secret
+  {{- end }}
+  rules:
+    - host: {{ .Values.ingress.hostPrefix.rest }}{{ .Values.baseDomain }}
+      http:
+        paths:
+          - path: /
+            {{- if (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }}
+            pathType: Prefix
+            {{- end }}
+            backend:
+              {{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
+              service:
+                name: {{ $fullRESTName }}
+                port:
+                  number: {{ $restSvcPort }}
+              {{- else }}
+              serviceName: {{ $fullRESTName }}
+              servicePort: {{ $restSvcPort }}
+              {{- end }}
+---
+{{- if not (eq .Values.ingress.className "traefik") }}
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullGRPCName }}
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  {{- with .Values.ingress }}
+  annotations:
+    {{- toYaml .annotations.base | nindent 4 }}
+    {{- if or (eq .className "nginx") (eq .className "public") }}
+    {{- toYaml .annotations.nginx | nindent 4 }}
+    {{- toYaml .annotations.grpc.nginx | nindent 4 }}
+    {{- end }}
+    {{- if eq .className "traefik" }}
+    {{- toYaml .annotations.traefik | nindent 4 }}
+    {{- end }}
+    {{- if and .tls.enabled (eq .tls.issuerName "" )}}
+    {{- toYaml .annotations.tls | nindent 4 }}
+    {{- else if .tls.enabled}}
+    cert-manager.io/cluster-issuer: {{ .tls.issuerName }}
+    {{- end }}
+  {{- end }}
+spec:
+  {{- if (not (eq .Values.ingress.className "traefik")) }}
+  {{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }}
+  ingressClassName:  {{ required "A valid .Values.ingress.className entry required!" .Values.ingress.className}}
+  {{- end }}
+  {{- end }}
+  {{- if .Values.ingress.tls.enabled }}
+  tls:
+    - hosts:
+        - {{ .Values.ingress.hostPrefix.grpc }}{{ .Values.baseDomain }}
+      secretName: {{ $fullGRPCName }}-tls-secret
+  {{- end }}
+  rules:
+    - host: {{ .Values.ingress.hostPrefix.grpc }}{{ .Values.baseDomain }}
+      http:
+        paths:
+          - path: /
+            {{- if (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }}
+            pathType: Prefix
+            {{- end }}
+            backend:
+              {{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
+              service:
+                name: {{ $fullGRPCName }}
+                port:
+                  number: {{ $grpcSvcPort }}
+              {{- else }}
+              serviceName: {{ $fullGRPCName }}
+              servicePort: {{ $grpcSvcPort }}
+              {{- end }}
+{{- end }}
+{{- if eq .Values.ingress.className "traefik" }}
+---
+apiVersion: traefik.containo.us/v1alpha1
+kind: IngressRouteTCP
+metadata:
+  name: {{ $fullGRPCName }}
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+spec:
+  entryPoints:
+    - websecure
+  routes:
+  - match: HostSNI(`{{ .Values.ingress.hostPrefix.grpc }}{{ .Values.baseDomain }}`)
+    services:
+    - name: {{ $fullGRPCName }}
+      port: {{ $grpcSvcPort }}
+    passthrough: true
+    scheme: https
+  tls:
+    secretName: {{ $fullGRPCName }}-tls-secret
+    domains:
+    - main: {{ .Values.ingress.hostPrefix.grpc }}{{ .Values.baseDomain }}
+{{- if and .Values.ingress.tls.enabled (not (eq .Values.ingress.tls.issuerName "" ))}}
+---
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  annotations:
+    acme.cert-manager.io/http01-override-ingress-name: {{ $fullRESTName }}
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  name: {{ $fullGRPCName }}-tls-secret
+spec:
+  dnsNames:
+  - {{ .Values.ingress.hostPrefix.grpc }}{{ .Values.baseDomain }}
+  issuerRef:
+    group: cert-manager.io
+    kind: ClusterIssuer
+    name:  {{ .Values.ingress.tls.issuerName }}
+  secretName: {{ $fullGRPCName }}-tls-secret
+  usages:
+  - digital signature
+  - key encipherment
+{{- end }}
+{{- end }}
+{{- end }}

templates/netmaker-statefulset.yaml

@@ -0,0 +1,133 @@
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  labels:
+    app: {{ include "netmaker.fullname" . }}
+  name: {{ include "netmaker.fullname" . }}
+spec:
+  replicas: {{ .Values.replicas }}
+  serviceName: {{ include "netmaker.fullname" . }}-headless
+  selector:
+    matchLabels:
+      app: {{ include "netmaker.fullname" . }}
+  template:
+    metadata:
+      labels:
+        app: {{ include "netmaker.fullname" . }}
+    spec:
+      {{- if .Values.wireguard.enabled }}
+      {{- if .Values.setIpForwarding.enabled }}
+      initContainers:
+      - name: init-sysctl
+        image: busybox
+        imagePullPolicy: IfNotPresent
+        command: ["sysctl", "-w", "net.ipv4.ip_forward=1"]
+        securityContext:
+          privileged: true
+      {{- end }}
+      dnsPolicy: ClusterFirstWithHostNet
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+          - labelSelector:
+              matchExpressions:
+              - key: app
+                operator: In
+                values:
+                - {{ include "netmaker.fullname" . }}
+            topologyKey: "kubernetes.io/hostname"
+      {{- end }}
+      containers:
+      - env:
+        - name: SERVER_API_CONN_STRING
+          value: api.{{ required "A valid .Values.baseDomain entry required!" .Values.baseDomain}}:443
+        - name: SERVER_GRPC_CONN_STRING
+          value: grpc.{{ required "A valid .Values.baseDomain entry required!" .Values.baseDomain}}:443
+        - name: GRPC_SSL
+          value: "on"
+        - name: SERVER_HTTP_HOST
+          value: api.{{ required "A valid .Values.baseDomain entry required!" .Values.baseDomain}}
+        - name: SERVER_GRPC_HOST
+          value: grpc.{{ required "A valid .Values.baseDomain entry required!" .Values.baseDomain}}
+        - name: API_PORT
+          value: "8081"
+        {{- if not .Values.wireguard.kernel }}
+        - name: WG_QUICK_USERSPACE_IMPLEMENTATION
+          value: wireguard-go
+        {{- end }}
+        - name: GRPC_PORT
+          value: "443"
+        {{- if .Values.dns.enabled }}
+        - name: DNS_MODE
+          value: "on"
+        - name: COREDNS_ADDR
+          value: {{ required "A valid .Values.dns.clusterIP entry required! Choose an IP from your k8s service IP CIDR" .Values.dns.clusterIP }}
+        {{- else }}
+        - name: DNS_MODE
+          value: "off"
+        {{- end }}
+        {{- if .Values.wireguard.enabled }}
+        - name: CLIENT_MODE
+          value: "on"
+        {{- else }}
+        - name: CLIENT_MODE
+          value: "off"
+        {{- end }}
+        - name: MASTER_KEY
+          value: {{ include "netmaker.masterKey" . }}
+        - name: PLATFORM
+          value: Kubernetes
+        - name: CORS_ALLOWED_ORIGIN
+          value: '*'
+        - name: NODE_ID
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: metadata.name
+        - name: SQL_HOST
+          value: '{{ .Release.Name }}-postgresql-ha-pgpool.{{ .Release.Namespace }}.svc.cluster.local'
+        - name: SQL_PORT
+          value: "5432"
+        - name: SQL_DB
+          value: {{ index .Values "postgresql-ha" "postgresql" "database" }}
+        - name: SQL_USER
+          value: {{ index .Values "postgresql-ha" "postgresql" "username" }}
+        - name: SQL_PASS
+          value: {{ index .Values "postgresql-ha" "postgresql" "password" }}
+        - name: DATABASE
+          value: postgres
+      {{- if or (not .Values.wireguard.enabled) (.Values.wireguard.kernel) }}
+        image: gravitl/netmaker:v0.8.4
+      {{- else }}
+        image: gravitl/netmaker:v0.8.4-userspace
+      {{- end }}
+        imagePullPolicy: Always
+        name: {{ include "netmaker.fullname" . }}
+        ports:
+        - containerPort: {{ .Values.service.restPort }}
+          protocol: TCP
+        - containerPort: {{ .Values.service.grpcPort }}
+          protocol: TCP
+      {{- if .Values.wireguard.enabled }}
+        {{ $count := (add .Values.wireguard.networkLimit 1 | int) }}
+        {{- range untilStep 1 $count 1 }}
+        - containerPort: {{ add 31820 . }}
+          protocol: UDP      
+        {{- end }}
+      {{- end }}
+        resources: {}
+        {{- if .Values.wireguard.enabled }}
+        securityContext:
+          capabilities:
+            add:
+            - NET_ADMIN
+        {{- end }}
+      {{- if .Values.dns.enabled }}
+        volumeMounts:
+        - name: {{ include "netmaker.fullname" . }}-dns-pvc
+          mountPath: /root/config/dnsconfig
+      volumes:
+      - name: {{ include "netmaker.fullname" . }}-dns-pvc
+        persistentVolumeClaim:
+          claimName: {{ include "netmaker.fullname" . }}-dns-pvc
+      {{- end }}

templates/netmaker-ui-deployment.yaml

@@ -0,0 +1,25 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: {{ include "netmaker.fullname" . }}-ui
+  name: {{ include "netmaker.fullname" . }}-ui
+spec:
+  replicas: {{ .Values.ui.replicas }}
+  selector:
+    matchLabels:
+      app: {{ include "netmaker.fullname" . }}-ui
+  template:
+    metadata:
+      labels:
+        app: {{ include "netmaker.fullname" . }}-ui
+    spec:
+      containers:
+      - name: {{ include "netmaker.fullname" . }}-ui
+        image: gravitl/netmaker-ui:v0.8
+        ports:
+        - containerPort: {{ .Values.service.grpcPort }}
+        env:
+        - name: BACKEND_URL
+          value: 'https://{{ .Values.ingress.hostPrefix.rest }}{{ required "A valid .Values.baseDomain entry required!" .Values.baseDomain}}'
+      terminationGracePeriodSeconds: 15

templates/serviceaccount.yaml

@@ -0,0 +1,12 @@
+{{- if .Values.serviceAccount.create -}}
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: {{ include "netmaker.serviceAccountName" . }}
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  {{- with .Values.serviceAccount.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+{{- end }}

templates/services.yaml

@@ -0,0 +1,72 @@
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  name: '{{ include "netmaker.fullname" . }}-ui'
+spec:
+  ports:
+  - port: {{ .Values.service.uiPort }}
+    protocol: TCP
+    targetPort: {{ .Values.service.uiPort }}
+  selector:
+    app: '{{ include "netmaker.fullname" . }}-ui'
+  sessionAffinity: None
+  type: '{{ .Values.service.type }}'
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  name: '{{ include "netmaker.fullname" . }}-rest'
+spec:
+  ports:
+  - name: rest
+    port: {{ .Values.service.restPort }}
+    protocol: TCP
+    targetPort: {{ .Values.service.restPort }}
+  selector:
+    app: '{{ include "netmaker.fullname" . }}'
+  sessionAffinity: None
+  type: {{ .Values.service.type }}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  name: '{{ include "netmaker.fullname" . }}-grpc'
+spec:
+  ports:
+  - name: rest
+    port: {{ .Values.service.grpcPort }}
+    protocol: TCP
+    targetPort: {{ .Values.service.grpcPort }}
+  selector:
+    app: '{{ include "netmaker.fullname" . }}'
+  sessionAffinity: None
+  type: {{ .Values.service.type }}
+{{- if .Values.wireguard.enabled }}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  name: '{{ include "netmaker.fullname" . }}-wireguard'
+spec:
+  externalTrafficPolicy: Local
+  type: NodePort
+  ports:
+  {{ $count := (add .Values.wireguard.networkLimit 1 | int) }}
+  {{- range untilStep 1 $count 1 }}
+  - port: {{ add 31820 . }}
+    nodePort: {{ add 31820 . }}
+    protocol: UDP
+    targetPort: {{ add 31820 . }}
+    name: wg-iface-{{ add 31820 . }}
+  {{- end }}
+  selector:
+    app: '{{ include "netmaker.fullname" . }}'
+{{- end }}

templates/tests/test-connection.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: "{{ include "netmaker.fullname" . }}-test-connection"
+  labels:
+    {{- include "netmaker.labels" . | nindent 4 }}
+  annotations:
+    "helm.sh/hook": test
+spec:
+  containers:
+    - name: wget
+      image: busybox
+      command: ['wget']
+      args: ['{{ include "netmaker.fullname" . }}:{{ .Values.service.port }}']
+  restartPolicy: Never

values.yaml

@@ -0,0 +1,124 @@
+# Default values for netmaker.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+# -- number of netmaker server replicas to create 
+replicas: 3
+
+image:
+  # -- The image repo to pull Netmaker image from 
+  repository: gravitl/netmaker
+  # -- Pull Policy for images
+  pullPolicy: Always
+  # -- Override the image tag to pull 
+  tag: "v0.8.4"
+
+# -- override the name for netmaker objects 
+nameOverride: ""  
+
+# -- override the full name for netmaker objects 
+fullnameOverride: ""
+
+serviceAccount:
+  # -- Specifies whether a service account should be created
+  create: true
+  # -- Annotations to add to the service account
+  annotations: {}
+  # -- Name of SA to use. If not set and create is true, a name is generated using the fullname template
+  name: ""
+
+# -- pod annotations to add
+podAnnotations: {}
+
+# -- pod security contect to add
+podSecurityContext: {}
+  # fsGroup: 2000
+
+ui:
+  # -- how many UI replicas to create
+  replicas: 2
+
+setIpForwarding:
+  enabled: true
+
+service:
+  # -- type for netmaker server services
+  type: ClusterIP
+  # -- port for API service
+  restPort: 8081
+  # -- port for GRPC service
+  grpcPort: 443
+  # -- port for UI service
+  uiPort: 80
+
+ingress:
+  # -- attempts to configure ingress if true
+  enabled: false
+  tls:
+    enabled: true
+    issuerName: "letsencrypt-prod"
+  annotations:
+    base:
+      # -- annotation to generate ACME certs if available
+      kubernetes.io/ingress.allow-http: "false"    
+    tls:
+      # -- use acme cert if available
+      kubernetes.io/tls-acme: "true"
+    nginx:
+      # -- Redirect http to https 
+      nginx.ingress.kubernetes.io/ssl-redirect: 'true'
+      # -- destination addr for route
+      nginx.ingress.kubernetes.io/rewrite-target: /
+    traefik:
+      # -- Redirect to https
+      traefik.ingress.kubernetes.io/redirect-entry-point: https
+      # -- Redirect to https permanently
+      traefik.ingress.kubernetes.io/redirect-permanent: "true"
+      # -- rule type
+      traefik.ingress.kubernetes.io/rule-type: "PathPrefixStrip"
+      # -- enforce https
+      traefik.ingress.kubernetes.io/router.entrypoints: websecure
+      # -- enforce tls
+      traefik.ingress.kubernetes.io/router.tls: "true"
+    grpc:
+      nginx:
+        # -- annotation to use grpc protocol on grpc domain
+        nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
+      traefik:
+        # -- annotation to use grpc protocol on grpc domain
+        ingress.kubernetes.io/protocol: "h2c"
+  hostPrefix:
+    # -- ui route subdomain
+    ui: 'dashboard.'
+    # -- api (REST) route subdomain
+    rest: 'api.'
+    # -- grpc route subdomain
+    grpc: 'grpc.'
+
+wireguard:
+  # -- whether or not to use WireGuard on server
+  enabled: true
+  # -- whether or not to use Kernel WG (should be false unless WireGuard is installed on hosts).
+  kernel: false
+  # -- max number of networks that Netmaker will support if running with WireGuard enabled
+  networkLimit: 10
+
+dns:
+  # -- whether or not to run with DNS (CoreDNS)
+  enabled: false
+  # -- volume size for DNS (only needs to hold one file)
+  storageSize: 128Mi
+
+postgresql-ha:
+  postgresql:
+    # -- postgres user to generate
+    username: netmaker
+    # -- postgres pass to generate
+    password: netmaker
+    # -- postgress db to generate
+    database: netmaker
+    # -- postgress number of replicas to deploy
+    replicaCount: 2
+  persistence:
+    # -- size of postgres DB
+    size: 3Gi