Compare commits

...

4 Commits

Author SHA1 Message Date
Gary Larizza
87ad4407b9 Start of Holos Hello 2024-11-07 14:06:45 -08:00
Gary Larizza
67ba145a29 Unify both Overview pages
This commit combines the Overview page contents that Jeff and
I developed independently. If this change stripped any important points
or messed up anything contextually then I'll be happy to revert it.
2024-11-07 11:22:31 -08:00
Gary Larizza
35d10eeb3b Fix markdown links 2024-11-07 10:52:51 -08:00
Gary Larizza
d00795574d Add Setup and Local Cluster Topic
The Setup page references the k3d Local Cluster guide so I needed to
throw it somewhere. Using 'Topics' as that place for now.
2024-11-07 10:26:14 -08:00
4 changed files with 613 additions and 29 deletions

View File

@@ -0,0 +1,272 @@
---
description: Build a local Cluster to use with these guides.
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Local Cluster
In this guide we'll set up a local k3d cluster to apply and explore the
configuration described in our other guides. After completing this guide you'll
have a standard Kubernetes API server with proper DNS and TLS certificates.
You'll be able to easily reset the cluster to a known good state to iterate on
your own Platform.
## Reset the Cluster
If you've already followed this guide, reset the cluster by running the
following commands. Skip this section if you're creating a cluster for the
first time.
First, delete the cluster.
<Tabs groupId="k3d-cluster-delete">
<TabItem value="command" label="Command">
```bash
k3d cluster delete workload
```
</TabItem>
<TabItem value="output" label="Output">
```txt showLineNumbers
INFO[0000] Deleting cluster 'workload'
INFO[0000] Deleting cluster network 'k3d-workload'
INFO[0000] Deleting 1 attached volumes...
INFO[0000] Removing cluster details from default kubeconfig...
INFO[0000] Removing standalone kubeconfig file (if there is one)...
INFO[0000] Successfully deleted cluster workload!
```
</TabItem>
</Tabs>
Then create the cluster again.
<Tabs groupId="k3d-cluster-create">
<TabItem value="command" label="Command">
```bash
k3d cluster create workload \
--registry-use k3d-registry.holos.localhost:5100 \
--port "443:443@loadbalancer" \
--k3s-arg "--disable=traefik@server:0"
```
</TabItem>
<TabItem value="output" label="Output">
```txt showLineNumbers
INFO[0000] portmapping '443:443' targets the loadbalancer: defaulting to [servers:*:proxy agents:*:proxy]
INFO[0000] Prep: Network
INFO[0000] Created network 'k3d-workload'
INFO[0000] Created image volume k3d-workload-images
INFO[0000] Starting new tools node...
INFO[0000] Starting node 'k3d-workload-tools'
INFO[0001] Creating node 'k3d-workload-server-0'
INFO[0001] Creating LoadBalancer 'k3d-workload-serverlb'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] HostIP: using network gateway 172.17.0.1 address
INFO[0001] Starting cluster 'workload'
INFO[0001] Starting servers...
INFO[0001] Starting node 'k3d-workload-server-0'
INFO[0003] All agents already running.
INFO[0003] Starting helpers...
INFO[0003] Starting node 'k3d-workload-serverlb'
INFO[0009] Injecting records for hostAliases (incl. host.k3d.internal) and for 3 network members into CoreDNS configmap...
INFO[0012] Cluster 'workload' created successfully!
INFO[0012] You can now use it like this:
kubectl cluster-info
```
</TabItem>
</Tabs>
Finally, add your trusted certificate authority.
<Tabs groupId="apply-local-ca">
<TabItem value="command" label="Command">
```bash
kubectl apply --server-side=true -f "$(mkcert -CAROOT)/namespace.yaml"
kubectl apply --server-side=true -n cert-manager -f "$(mkcert -CAROOT)/local-ca.yaml"
```
</TabItem>
<TabItem value="output" label="Output">
```txt showLineNumbers
namespace/cert-manager serverside-applied
secret/local-ca serverside-applied
```
</TabItem>
</Tabs>
You're back to the same state as the first time you completed this guide.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a k8s api server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
5. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to interact with the k8s api server.
6. [mkcert](https://github.com/FiloSottile/mkcert?tab=readme-ov-file#installation) - to make trusted TLS certificates.
7. [jq](https://jqlang.github.io/jq/download/) - to fiddle with JSON output.
## Configure DNS {#configure-dns}
Configure your machine to resolve `*.holos.localhost` to your loopback
interface. This is necessary for requests to reach the workload cluster. Save
this script to a file and execute it.
```bash showLineNumbers
#! /bin/bash
#
set -euo pipefail
tmpdir="$(mktemp -d)"
finish() {
[[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"
brew install dnsmasq
cat <<EOF >"$(brew --prefix)/etc/dnsmasq.d/holos.localhost.conf"
# Refer to https://holos.run/docs/tutorial/local/k3d/
address=/holos.localhost/127.0.0.1
EOF
if [[ -r /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist ]]; then
echo "dnsmasq already configured"
else
sudo cp "$(brew list dnsmasq | grep 'dnsmasq.plist$')" \
/Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo launchctl unload /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
dscacheutil -flushcache
echo "dnsmasq configured"
fi
sudo mkdir -p /etc/resolver
sudo tee /etc/resolver/holos.localhost <<EOF
domain holos.localhost
nameserver 127.0.0.1
EOF
sudo killall -HUP mDNSResponder
echo "all done."
```
## Create the Cluster {#create-the-cluster}
The Workload Cluster is where your applications and services will be deployed.
In production this is usually an EKS, GKE, or AKS cluster.
:::tip
Holos supports all compliant Kubernetes clusters. Holos was developed and tested
on GKE, EKS, Talos, k3s, and Kubeadm clusters.
:::
Create a local registry to speed up image builds and pulls.
```bash
k3d registry create registry.holos.localhost --port 5100
```
Create the workload cluster configured to use the local registry.
```bash
k3d cluster create workload \
--registry-use k3d-registry.holos.localhost:5100 \
--port "443:443@loadbalancer" \
--k3s-arg "--disable=traefik@server:0"
```
Traefik is disabled because Istio provides the same functionality.
## Setup Root CA {#setup-root-ca}
Platforms most often use cert-manager to issue tls certificates. The browser
and tools we're using need to trust these certificates to work together.
Generate a local, trusted root certificate authority with the following script.
Admin access is necessary for `mkcert` to manage the certificate into your trust
stores.
```bash
sudo -v
```
Manage the local CA and copy the CA key to the workload cluster so that cert
manager can manage trusted certificates.
Save this script to a file and execute it to configure a trusted certificate
authority.
```bash showLineNumbers
#! /bin/bash
#
set -euo pipefail
mkcert --install
tmpdir="$(mktemp -d)"
finish() {
[[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"
# Create the local CA Secret with ca.crt, tls.crt, tls.key
mkdir local-ca
cd local-ca
CAROOT="$(mkcert -CAROOT)"
cp -p "${CAROOT}/rootCA.pem" ca.crt
cp -p "${CAROOT}/rootCA.pem" tls.crt
cp -p "${CAROOT}/rootCA-key.pem" tls.key
kubectl create secret generic --from-file=. --dry-run=client -o yaml local-ca > ../local-ca.yaml
echo 'type: kubernetes.io/tls' >> ../local-ca.yaml
cd ..
cat <<EOF > namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: cert-manager
name: cert-manager
spec:
finalizers:
- kubernetes
EOF
kubectl apply --server-side=true -f namespace.yaml
kubectl apply -n cert-manager --server-side=true -f local-ca.yaml
# Save the Secret to easily reset the cluster later.
install -m 0644 namespace.yaml "${CAROOT}/namespace.yaml"
install -m 0600 local-ca.yaml "${CAROOT}/local-ca.yaml"
```
:::warning
Take care to run the local-ca script each time you create the workload cluster
so that Certificates are issued correctly.
:::
## Clean Up {#clean-up}
If you'd like to clean up the resources you created in this guide, remove them
with:
```bash
k3d cluster delete workload
```
## Next Steps
Now that you have a real cluster, continue your Holos journey by following the
[Tutorial](../tutorial/1-overview.mdx) and setting up your own Platform.

View File

@@ -9,11 +9,10 @@ import RenderComponentDiagram from '../diagrams/render-component-sequence.mdx';
# Tutorial
Holos is a configuration management tool for Kubernetes resources. It provides
the building blocks needed for implementing the [rendered manifests pattern].
It gives the flexibility to manage a wide range of configurations, from large
software delivery platforms spanning multiple clusters and regions to generating
a single resource on your local device.
Holos is an open-source configuration management tool for Kubernetes resources,
designed to simplify software integration for platform teams. It supports a wide range of
configurations, from managing large, multi-cluster platforms across regions to
generating a single resource on a local device.
{/* truncate */}
@@ -27,18 +26,21 @@ At a high level, Holos provides a few major components:
## Holos' role in your organization
Platform engineers run the `holos render platform` command locally and in CI to
produce Kubernetes manifests which are committed to version control. GitOps
tools like ArgoCD or Flux deploy the manifests produced by Holos.
Platform engineers use the `holos render platform` command locally or in CI to
generate Kubernetes manifests, which are then committed to version control.
GitOps tools like ArgoCD or Flux handle the deployment of these manifests.
Other tools focus on individual applications. Holos focuses holistically on the
integration of applications into a whole platform. In this way a single one
line configuration change, like changing the domain name, is clearly visible
across the whole platform.
Unlike most Kubernetes tools that focus solely on application management, Holos
takes a holistic approach, targeting the integration layer where applications
and organizational data converge. Its well-defined, typed structures ensure
consistent validation, reducing errors, streamlining integration, and providing
clear pathways for teams to integrate their services.
Rendering a platform may produce a single ConfigMap resource or produce millions
of lines of fully rendered manifests configuring multiple environments,
clusters, and regions.
Holos focuses on the platform as a whole rather than individual applications,
making it easy to see the impact of a single configuration change, like updating
a domain name, across the entire system. Whether generating a single ConfigMap
or producing millions of lines of manifests for multiple environments, clusters,
and regions, Holos ensures platform-wide consistency and visibility.
<RenderPlatformDiagram />
@@ -48,26 +50,34 @@ This section outlines some advantages of Holos.
### Safe
Holos uses [CUE] to provide strong typing and constraints to configuration data.
Additionally, holos adds strong validation to verify the output produced by Helm
and other tools.
* [CUE] provides strong typing and constraints for Holos Components that surface
validation errors early in the development process, reducing the number of
failed deployments and the time spent troubleshooting them.
* Holos natively provides a "blast radius" to code
changes by identifying the rendered manifests across your fleet of Kubernetes
clusters that will be affected by the change.
* [CUE's][CUE] unification strategy allows multiples teams to contribute to the desired
state of a service:
* The Platform team provides definitions for shared resources.
* Engineering teams populate definitions with service-specific data.
* The Security team provides concrete values that cannot be changed to harden the company's security posture.
### Consistent
Holos offers a consistent way to incorporate a wide variety of tools into a well
defined data pipeline. Configuration produced from CUE, Helm, and Kustomize are
all handled with the same consistent process.
### Easy
Holos makes it easy to define configuration data once, then safely use the data
in multiple Helm charts, Kustomize bases, or any other supported component kind.
* Holos manages the execution context for Helm and Kustomize, ensuring that
rendered manifests are consistently and reliably reproducible, no matter where
Holos is run.
* CUE constraints ensure that data abstractions include the required information
in the expected format, triggering early failures if any required data is
missing.
### Flexible
Holos is designed to be flexible. Holos offers flexible building blocks for
data generation, transformation, validation, and integration. Find the perfect
fit for your team by assembling these building blocks to your unique needs.
* [CUE] is adept at modeling variation and organizational complexity at scale,
while Holos enables seamless integration of [CUE] data with native Kubernetes
tools such as [Helm] and [Kustomize].
* Holos is extensible, allowing Holos Components to be modeled from any tool that
generates (e.g. `helm`) or transforms (e.g. `kustomize`) manifest data.
Holos does not have an opinion on many common aspects of platform configuration.
For example, environments and clusters are explicitly kept out of the core and
@@ -116,6 +126,8 @@ the most experienced among us. We all start somewhere and are happy to help.
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[CUE]: https://cuelang.org/
[Helm]: https://helm.sh/
[Kustomize]: https://kustomize.io/
[Discord]: https://discord.gg/JgDVbNpye7
[GitHub discussions]: https://github.com/holos-run/holos/discussions
[Why CUE for Configuration]: https://holos.run/blog/why-cue-for-configuration/

View File

@@ -1 +1,89 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Setup
This tutorial will guide you through the installation of Holos and its
dependencies, as well as the initialization of a minimal Platform that you can
extend to meet your specific needs.
# Prerequisites
Holos integrates with the following tools that should be installed to enable
their functionality.
* [Helm][helm] to fetch and render Helm chart Components
* [Kubectl][kubectl] to [kustomize][kustomize] components.
# Install Holos
Holos is distributed as a single file executable that can be installed in a couple of ways.
### Releases
Download `holos` from the [releases] page and place the executable into your shell path.
### Go install
Alternatively, install directly into your go bin path using:
```shell
go install github.com/holos-run/holos/cmd/holos@latest
```
# Do I need a Kubernetes cluster?
Holos only generates rendered Kubernetes manifests, so you don't need a
Kubernetes cluster to start using the tool or understand its workflow. However,
you will need a cluster eventually to apply the rendered manifests and verify
that they achieve the desired end state.
We recommend using [K3d] to set up a minimal Kubernetes cluster with [Orbstack]
or [Docker]. To simplify this, we've created [the K3d local cluster
guide][localcluster] which automates the process, ensuring proper DNS and TLS
certificates, and includes a script to reset the cluster to a known good state.
When you're ready to experiment with a live Kubernetes cluster, refer to [the
k3d local cluster guide][localcluster].
# Initialize a new Platform
Use Holos to generate a minimal platform using the recommended directory structure
by creating an empty directory and then using the `holos generate platform` subcommand:
<Tabs groupId="tutorial-setup-generate-platform">
<TabItem value="command" label="Command">
```bash
mkdir holos_platform
cd holos_platform
holos generate platform v1alpha4
```
</TabItem>
<TabItem value="output" label="Output">
```txt
no output
```
</TabItem>
</Tabs>
Holos creates a `platform` directory with a `platform.gen.cue` file that serves
as the foundation for your newly initialized Holos Platform. For each Holos
Component you model, you'll add a new CUE file to the platform directory,
mapping it to the location of the Component's CUE code, and extending the
capabilities of your platform.
# Next Steps
Now that you've got Holos and its prerequisites installed, continue on to the
[Hello Holos page][helloholos] where we will guide you through the
process of creating your first Component and modeling a Helm chart.
[K3d]: https://k3d.io/
[Orbstack]: https://docs.orbstack.dev/install
[Docker]: https://docs.docker.com/get-started/get-docker/
[localcluster]: ../topics/4-local-cluster.mdx
[helloholos]: ./3-hello-holos.mdx
[helm]: https://github.com/helm/helm/releases
[kubectl]: https://kubernetes.io/docs/tasks/tools/
[releases]: https://github.com/holos-run/holos/releases
[kustomize]: https://kustomize.io/

View File

@@ -1 +1,213 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Hello Holos
One of the first exercises you perform when learning a new programming language
is to print out the "Hello World!" greeting. For Holos, our "Hello Holos"
exercise involves modeling the [podinfo Helm chart][podinfo] that produces a
similar greeting message from a Kubernetes Pod.
By the end of this tutorial you will gain the understanding of how to model
an individual Holos Component using a Helm chart as its source.
# The code
### Podinfo Helm Chart
Let's start by creating a directory for `podinfo`, touching an empty CUE file,
and then populating it with the CUE code below:
<Tabs groupId="tutorial-hello-podinfo-helm-cue-code">
<TabItem value="projects/tutorial/components/podinfo/podinfo.cue" label="Podinfo Helm Chart">
```bash
mkdir -p projects/tutorial/components/podinfo
touch projects/tutorial/components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: HelmChart.BuildPlan
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
</TabItem>
</Tabs>
### Register the podinfo component
Now let's register the code modeling the `podinfo` Helm chart as a Holos
Component. Create a new file in the `platform` directory with this CUE code:
<Tabs groupId="tutorial-hello-register-podinfo-component">
<TabItem value="platform/podinfo.cue" label="Register Podinfo">
```bash
touch plaform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "projects/tutorial/components/podinfo"
}
```
</TabItem>
</Tabs>
### Render the podinfo manifest file
Finally we're ready to render the podinfo Kubernetes manifest using the
`holos render platform` subcommand, targeting the `platform` directory.
<Tabs groupId="tutorial-hello-render-manifests">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```
rendered podinfo in 116.142583ms
rendered platform in 116.204042ms
```
</TabItem>
<TabItem value="manifest" label="Manifest Data">
```yaml
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
helm.sh/chart: podinfo-6.6.2
name: podinfo
spec:
ports:
- name: http
port: 9898
protocol: TCP
targetPort: http
- name: grpc
port: 9999
protocol: TCP
targetPort: grpc
selector:
app.kubernetes.io/name: podinfo
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
helm.sh/chart: podinfo-6.6.2
name: podinfo
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: podinfo
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
prometheus.io/port: "9898"
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: podinfo
spec:
containers:
- command:
- ./podinfo
- --port=9898
- --cert-path=/data/cert
- --port-metrics=9797
- --grpc-port=9999
- --grpc-service-name=podinfo
- --level=info
- --random-delay=false
- --random-error=false
env:
- name: PODINFO_UI_COLOR
value: '#34577c'
image: ghcr.io/stefanprodan/podinfo:6.6.2
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/healthz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: podinfo
ports:
- containerPort: 9898
name: http
protocol: TCP
- containerPort: 9797
name: http-metrics
protocol: TCP
- containerPort: 9999
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/readyz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits: null
requests:
cpu: 1m
memory: 16Mi
volumeMounts:
- mountPath: /data
name: data
terminationGracePeriodSeconds: 30
volumes:
- emptyDir: {}
name: data
```
</TabItem>
</Tabs>
Success! Holos generated the `deploy/components/podinfo/podinfo.gen.yaml` file
containing manifest data from the `podinfo` Helm chart.
# Breaking it down
Let's take some time to review what we just did.
[podinfo]: https://github.com/stefanprodan/podinfo