chore(README): Updated README bootstrapping instructions

This should hopefully make it easier to get started with a similar setup

COMPONENTS.md

@@ -1,10 +1,7 @@
-kubespray
-Argo autopilot
-
 CNI: Cilium
-LoadBalancer: MetalLB
+LoadBalancer: Cilium
 Ingress: Traefik
-Certs: cert-manager
+Certs: Cert-manager
 CD: ArgoCD
 Monitoring: Prometheus
 Observability: Grafana

QUICKSTART.md

@@ -1,59 +0,0 @@
-# Kubernetes
-
-## Disable swap
-
-```shell
-swapoff -a
-```
-
-## Start Kubernetes
-
-```shell
-sudo kubeadm init
-```
-
-## Set up kubectl
-
-```shell
-sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config && sudo chown $(id -u):$(id -g) $HOME/.kube/config
-```
-
-## Remove taint for single node use
-
-```shell
-kubectl taint nodes --all node-role.kubernetes.io/control-plane-
-```
-
-# Cilium
-
-## Install Cilium as a CNI
-
-```shell
-cilium install
-```
-
-# MetalLB
-
-## Install MetalLB for LoadBalancing
-
-https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml
-
-```shell
-kubectl apply -k infra/metallb
-```
-
-# Traefik reverse proxy
-
-https://doc.traefik.io/traefik/v2.9/user-guides/crd-acme/
-
-```shell
-kubectl kustomize --enable-helm infra/traefik | kubectl apply -f -
-```
-
-# ArgoCD
-
-https://argo-cd.readthedocs.io/en/stable/getting_started/
-
-```shell
-kubectl apply -k infra/argocd
-```

README.md

@@ -20,7 +20,7 @@ sudo apt-get update
 sudo apt-get install -y containerd conntrack socat kubelet kubeadm kubectl 
 ```
 
-Kubelet 1.26 requires containerd 1.6.0 or later.
+Kubelet ≥ 1.26 requires containerd ≥ 1.6.0.
 
 ## Initialise cluster
 
@@ -37,13 +37,14 @@ https://kubernetes.io/docs/tasks/tools/
 
 ```shell
 mkdir -p $HOME/.kube
-sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config && sudo chown $(id -u):$(id -g) $HOME/.kube/config
+sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
+sudo chown $(id -u):$(id -g) $HOME/.kube/config
 ```
 
 For remote kubectl copy the config file to local machine
 
 ```shell
-scp veh@192.168.1.12:/home/veh/.kube/config ~/.kube/config
+scp gauss@192.168.1.12:/home/gauss/.kube/config ~/.kube/config
 ```
 
 ## (Optional) Remove taint for single node use
@@ -60,18 +61,14 @@ Remove taint on master node to allow scheduling of all deployments
 kubectl taint nodes --all node-role.kubernetes.io/control-plane-
 ```
 
-## Install Cilium as Container Network Interface (CNI)
+## Install Cilium as CNI (Container Network Interface)
+
+To bootstrap the cluster we can install Cilium using its namesake CLI.
+
+For Linux this can be done by running
 
 ```shell
-kubectl kustomize --enable-helm infra/cilium | kubectl apply -f -
+CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
-```
-
-https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/
-
-Install Cilium CLI
-
-```shell
-CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
 CLI_ARCH=amd64
 if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
 curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
@@ -80,12 +77,23 @@ sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
 rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
 ```
 
-Install Cilium
+See the [Cilium official docs](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/) for more options.
+
+Next we install Cilium in Kube proxy replacement mode and enable L2 announcements to reply to ARP requests.
+To not run into rate limiting while doing L2 announcements we also increase the k8s rate limits.
 
 ```shell
-cilium install
+cilium install \
+  --set kubeProxyReplacement=true \
+  --set l2announcements.enabled=true \
+  --set externalIPs.enabled=true \
+  --set k8sClientRateLimit.qps=50 \
+  --set k8sClientRateLimit.burst=100
 ```
 
+See [this blog post](https://blog.stonegarden.dev/articles/2023/12/migrating-from-metallb-to-cilium/#l2-announcements)
+for more details.
+
 Validate install
 
 ```shell
@@ -95,13 +103,16 @@ cilium status
 ## Cilium LB IPAM
 
 For [Cilium to act as a load balancer](https://docs.cilium.io/en/stable/network/lb-ipam/) and start assigning IPs
-to `LoadBalancer` services create a viable IP pool, e.g. `192.168.1.128/5`, by creating a `CiliumLoadBalancerIPPool`
+to `LoadBalancer` `Service` resources we need to create a `CiliumLoadBalancerIPPool` with a valid pool.
+
+Edit the cidr range to fit your network before applying it
 
 ```shell
 kubectl apply infra/cilium/ip-pool.yaml
 ```
 
-and announce using a `CiliumL2AnnouncementPolicy`
+Next create a `CiliumL2AnnouncementPolicy` to announce the assigned IPs.
+Leaving the `interfaces` field empty announces on all interfaces.
 
 ```shell
 kubectl apply infra/cilium/announce.yaml
@@ -125,6 +136,9 @@ kubectl -n kube-system get secrets
 
 # Traefik
 
+Remove the `deployment.dnsConfig` from `infra/traefik/values.yaml` and change the `io.cilium/lb-ipam-ips` annotation to
+a valid IP address for your network.
+
 Install Traefik
 
 ```shell
@@ -134,44 +148,31 @@ kubectl kustomize --enable-helm infra/traefik | kubectl apply -f -
 ## Port forward Traefik
 
 Port forward Traefik ports in router from 8000 to 80 for http and 4443 to 443 for https.
-IP can be found with `kubectl get svc`.
+IP can be found with `kubectl get svc` (it should be the same as the one you gave in the annotation).
 
-# Test-application
+# Test-application (Optional)
 
-## Generate secret
+Deploy a test-application by editing the manifests in `apps/test/whoami` and apply them
-
-```yaml
-apiVersion: v1
-kind: Secret
-metadata:
-  name: traefik-forward-auth-secrets
-  namespace: whoami
-type: Opaque
-data:
-  google-client-id: <...>
-  google-client-secret: <...>
-  secret: <...>
-```
-
-Deploy a test-application by running
 
 ```shell
-kubectl apply -k apps/whoami
+kubectl apply -k apps/test/whoami
 ```
 
 An unsecured test-application `whoami` should be available at [https://test.${DOMAIN}](https://test.${DOMAIN}).
-If you configured `apps/whoami/traefik-forward-auth` correctly a secured version should be available
+If you configured `apps/test/whoami/traefik-forward-auth` correctly a secured version should be available
-at [https://whoami.${DOMAIN}](https://whoami.${DOMAIN})
+at [https://whoami.${DOMAIN}](https://whoami.${DOMAIN}).
 
 # ArgoCD
 
-[ArgoCD](https://argo-cd.readthedocs.io/en/stable/getting_started/) is configured to bootstrap the rest of the cluster
+[ArgoCD](https://argo-cd.readthedocs.io/en/stable/getting_started/) is used to bootstrap the rest of the cluster.
+The cluster uses a combination of Helm and Kustomize to configure infrastructure and applications.
+For more details read [this blog post](https://blog.stonegarden.dev/articles/2023/09/argocd-kustomize-with-helm/)
 
 ```shell
 kubectl apply -k infra/argocd
 ```
 
-Get ArgoCD initial secret
+Get ArgoCD initial secret by running
 
 ```shell
 kubectl -n argocd get secrets argocd-initial-admin-secret -o json | jq -r .data.password | base64 -d
@@ -195,6 +196,8 @@ kubectl -n kubernetes-dashboard create token admin-user
 
 # ApplicationSets
 
+*NB!*: This will not work before you've changed all the domain names and IP addresses.
+
 Once you've tested everything get the ball rolling with
 
 ```shell
@@ -231,10 +234,6 @@ runtime_path = "/usr/bin/runc"
 runtime_type = "io.containerd.runc.v2"
 ```
 
-## Wrong containerd version
-
-1.7.x doesn't work?
-
 ## Sealed Secrets
 
-Restart pod after applying master-key
+Restart pod after applying master-key.

RESOURCES.md

@@ -1,10 +1,10 @@
 https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
-https://metallb.universe.tf/installation/
 https://kubernetes.io/docs/concepts/services-networking/service/
+
+https://docs.cilium.io/en/stable/
+
+https://github.com/bitnami-labs/sealed-secrets#usage
+
 https://doc.traefik.io/traefik/v2.8/user-guides/crd-acme/
 
-https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/guides/getting-started
+https://www.smarthomebeginner.com/traefik-forward-auth-google-oauth-2022/
-
-https://www.smarthomebeginner.com/traefik-forward-auth-google-oauth-2022/
-
-https://github.com/bitnami-labs/sealed-secrets#usage