Compare commits

..

16 Commits

Author SHA1 Message Date
Jeff McCune
e9d1240d63 docs: make update-docs for version 0.103.0 2025-01-12 14:26:27 -08:00
Gary Larizza
03fa4eaaa2 docs: Helm Values test updates
* Convert all files with.period.separators to hyphen-separators.
* Rename and markdown_test.go to be specific to Helm Values.
* Move helm-values_test.go to be in the same directory as the Helm Values doc.
* Move Blackbox common configuration CUE file to `config/prometheus` so it can be imported as necessary.
* Use explicit import statements for Blackbox common config in `blackbox` and `prometheus` components.

Closes: #399
2025-01-12 14:25:44 -08:00
Jeff McCune
e363f3a597 docs: add make update-docs task
We need to run this prior to tagging a release otherwise the tests fail
for the new version string.
2025-01-12 14:22:58 -08:00
Jeff McCune
8b49ed93be docs: release version 0.103.0 2025-01-12 14:09:45 -08:00
Jeff McCune
d2be9fe278 helm: add valueFiles for migration from an ApplicationSet
Without this patch migrating from [helm hierarchies] to Holos requires
the user to unify the value hierarchy.  This is a problem because helm
hierarchies are difficult to unify because it's not clear if or why a
value is used in the final results.  This makes it difficult to identify
how to resolve conflicts.

This patch adds `valueFiles` field to the Helm component kind.  This
field is intended to provide a direct migration path from the
ApplicationSet.spec.template.spec.sources.helm.valueFiles field.  With
this patch, users can directly migrate the values files to CUE using
`@embed`, then directly migrate the valueFiles field to reference the
values from within CUE.

Note we actively discourage the use of Helm value hierarchies.  The
feature is intended as a temporary migration tool.  We encourage the use
of CUE unification instead.  After migration, the valueFiles field
should be refactored to the values field as one unified structure in
CUE.  The valueFiles field makes this second order migration easier
becuase we can inspect and verify the complete rendered output, allowing
us to determine if a value is actually used in the final configuration
or is overridden.

[helm hierarchies]: https://medium.com/containers-101/using-helm-hierarchies-in-multi-source-argo-cd-applications-for-promoting-to-different-gitops-133c3bc93678
2025-01-12 13:30:29 -08:00
Jeff McCune
6ec341bbb1 docs: redirect /docs/api/core 2025-01-10 15:02:12 -08:00
Jeff McCune
13a4305b78 docs: add redirect for /blog/rendered-manifest-pattern (manifest instead of manifests) 2025-01-10 14:50:26 -08:00
Jeff McCune
0cfce3a823 docs: redirect rendered manifests pattern for now
Need a URL we can redirect when we publish our own variation on the
pattern with a link back to Akuity.
2025-01-10 10:55:06 -08:00
Jeff McCune
61d7539e1c docs: fix /docs/guides/ redirect 2025-01-09 16:03:50 -08:00
Jeff McCune
bf84724137 docs: add redirects for github.com/holos-run readme 2025-01-09 15:11:04 -08:00
Jeff McCune
9f0de7555c init: change to holos.example default cue module
Match the cue mod init behavior of a module named `cue.example`.
2025-01-09 13:57:26 -08:00
Gary Larizza
650636f944 Merge pull request #393 from holos-run/gl/update-helm-docs
Update Helm Values Tutorial to use testscript
2025-01-09 12:01:09 -08:00
Gary Larizza
b28c110694 Update Helm Values tutorial to use testscript
PROBLEM:

The Helm Values tutorial contains a fair bit of code/scripts, and we
need a way to test the steps we recommend to make sure nothing breaks
or slips out of date.

SOLUTION:

* Use `testscript` as a way to automate the execution of the steps in the doc and verify that none of the steps produce errors.
* Update the MDX file to directly reference the files embedded into the testscript.

OUTCOME:

* We have an automated way to perform the steps in the Helm Values document.
* We have unit tests that will fail should any of the commands being executed in the doc fail.
* The doc's MDX file directly references the files within the testscript, so we only need to modify the MDX file to update wording.
2025-01-09 11:53:53 -08:00
Gary Larizza
5bb3e90b38 Install raw-loader module
We use this module within our markdown tutorials (like the Helm Values
tutorial) to load in files generated by testscript.
2025-01-09 11:53:13 -08:00
Jeff McCune
6a60b613ff render: fix selectors (#394)
Without this patch selectors don't work as expected.  This patch
fixes selectors such that each --selector flag value configures one
selector containing multiple positive or negative label matchers.

Result:

Render build plans for cluster dev or cluster test.  Note the use of two
flags indicating logical OR.

    holos render platform --selector cluster=test --selector cluster=dev
    rendered external-secrets for cluster test in 299.897542ms
    rendered external-secrets for cluster dev in 299.9225ms
    rendered external-secrets-crds for cluster test in 667.6075ms
    rendered external-secrets-crds for cluster dev in 708.126541ms
    rendered platform in 708.795625ms

Render build plans for prod clusters that are not customer facing.  Note
the use of one selector with comma separated labels.

    holos render platform --selector "tier=prod,scope!=customer"
2025-01-08 21:09:00 -08:00
Jeff McCune
5862725bab builder: deprecate ExtractYAML, use cue embed instead
Easier to place the data, better supported in the ecosystem.
2025-01-02 18:53:10 -08:00
62 changed files with 1010 additions and 332 deletions

View File

@@ -154,6 +154,10 @@ website: ## Build website
unity: ## https://cuelabs.dev/unity/
./scripts/unity
.PHONY: update-docs
update-docs: ## Update doc examples
HOLOS_UPDATE_SCRIPTS=1 go test -v ./doc/md/...
.PHONY: help
help: ## Display this help menu.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)

View File

@@ -84,6 +84,9 @@ type Helm struct {
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.

View File

@@ -118,8 +118,12 @@ type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
@@ -130,6 +134,17 @@ type Helm struct {
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// ValueFile represents one Helm value file produced from CUE.
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any

View File

@@ -86,6 +86,9 @@ type Helm struct {
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.

View File

@@ -43,6 +43,7 @@ Package core contains schemas for a [Platform](<#Platform>) and [BuildPlan](<#Bu
- [type Resources](<#Resources>)
- [type Transformer](<#Transformer>)
- [type Validator](<#Validator>)
- [type ValueFile](<#ValueFile>)
- [type Values](<#Values>)
@@ -283,8 +284,12 @@ type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
@@ -493,6 +498,23 @@ type Validator struct {
}
```
<a name="ValueFile"></a>
## type ValueFile {#ValueFile}
ValueFile represents one Helm value file produced from CUE.
```go
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
```
<a name="Values"></a>
## type Values {#Values}

View File

@@ -0,0 +1,7 @@
exec bash -c 'bash -euo pipefail $WORK/command.sh 2>&1'
cmp stdout $WORK/output.txt
-- command.sh --
holos --version
-- output.txt --
0.103.0

View File

@@ -0,0 +1,374 @@
# Set $HOME because:
# - Helm uses it for temporary files
# - Git requires it for setting author name/email globally
env HOME=$WORK/.tmp
chmod 0755 $WORK/update.sh
# Configure git author for testscript execution
exec git config --global user.name 'Holos Docs'
exec git config --global user.email 'hello@holos.run'
exec git config --global init.defaultBranch main
# Remove the tutorial directory if it already exists
exec rm -rf holos-helm-values-tutorial
# Create and change to the tutorial directory, and then initialize the Holos platform
exec bash -c 'bash -euo pipefail mkdir-and-init.sh'
cd holos-helm-values-tutorial
# Git init and create the component directories
exec bash -c 'bash -euo pipefail $WORK/git-init.sh'
exec bash -c 'bash -euo pipefail $WORK/mkdir-components.sh'
# Combine and execute the multiline prometheus/blackbox component header/body/trailer files
exec cat $WORK/prometheus-component-header.sh ../prometheus-component-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
exec cat $WORK/blackbox-component-header.sh ../blackbox-component-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Combine and execute the multiline platform registration header/body/trailer files.
exec cat $WORK/register-components-header.sh ../register-components-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Render the platform, capture stdout, and use update.sh to gate whether the
# output file should be updated.
#
# NOTE: The [net] condition will test whether external network access is available
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/register-components-output.txt
# Commit and conditionally update the output file
exec bash -c 'bash -euo pipefail $WORK/register-components-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/register-components-git-commit-output.txt
# Import values
exec bash -c 'bash -euo pipefail $WORK/import-prometheus-values.sh'
exec bash -c 'bash -euo pipefail $WORK/import-blackbox-values.sh'
# Render, update the output file, commit, and update the commit output file.
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/import-values-render-output.txt
exec bash -c 'bash -euo pipefail $WORK/import-values-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/import-values-git-output.txt
# Create the common configuration path
exec bash -c 'bash -euo pipefail $WORK/mkdir-common-config.sh'
# Combine and execute the common configuration header/body/trailer to write the cue file.
exec cat $WORK/blackbox-common-config-header.sh ../blackbox-common-config-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Git commit blackbox common config
exec bash -c 'bash -euo pipefail $WORK/blackbox-common-config-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/blackbox-common-config-git-output.txt
# Patch the common config values file and write to output file.
#
# NOTE: Using a symlink here because the patch script references values.patch
# within the same directory, but it actually lives one directory up in the
# testscript $WORK dir.
exec ln -s $WORK/values.patch values.patch
exec bash -c 'bash -euo pipefail $WORK/common-config-patch.sh'
stdin stdout
exec $WORK/update.sh $WORK/common-config-patch.txt
# Remove patch and commit changes
exec bash -c 'bash -euo pipefail $WORK/common-config-rm.sh'
exec bash -c 'bash -euo pipefail $WORK/common-config-git.sh'
stdin stdout
exec $WORK/update.sh $WORK/common-config-git-output.txt
# Final render and update of output file.
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/reviewing-changes-git-output.txt
# Git diff and write to output file.
exec bash -c 'bash -euo pipefail $WORK/git-diff.sh'
stdin stdout
exec $WORK/update.sh $WORK/git.diff
# Final commit and write to output file
exec bash -c 'bash -euo pipefail $WORK/reviewing-changes-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/reviewing-changes-git-output.txt
# Clean up the tutorial directory and tmp $HOME directory
cd $WORK
exec rm -rf holos-helm-values-tutorial
exec rm -rf $HOME
-- update.sh --
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"
-- mkdir-and-init.sh --
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5
-- git-init.sh --
git init . && git add . && git commit -m "initial commit"
-- mkdir-components.sh --
mkdir -p components/prometheus components/blackbox
-- prometheus-component-header.sh --
cat <<EOF > components/prometheus/prometheus.cue
-- prometheus-component-body.cue --
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
-- eof-trailer.sh --
EOF
-- blackbox-component-header.sh --
cat <<EOF > components/blackbox/blackbox.cue
-- blackbox-component-body.cue --
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
-- register-components-header.sh --
cat <<EOF > platform/prometheus.cue
-- register-components-body.cue --
package holos
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}
-- render.sh --
holos render platform
-- register-components-output.txt --
cached prometheus-blackbox-exporter 9.0.1
rendered blackbox in 3.825430417s
cached prometheus 25.27.0
rendered prometheus in 4.840089667s
rendered platform in 4.840137792s
-- register-components-git-commit.sh --
git add . && git commit -m 'add blackbox and prometheus'
-- register-components-git-commit-output.txt --
[main b5df111] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue
-- import-prometheus-values.sh --
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml
-- import-blackbox-values.sh --
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
-- import-values-render-output.txt --
rendered blackbox in 365.936792ms
rendered prometheus in 371.855875ms
rendered platform in 372.109916ms
-- import-values-git-commit.sh --
git add . && git commit -m 'import values'
-- import-values-git-output.txt --
[main 52e90ea] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue
-- mkdir-common-config.sh --
mkdir -p config/prometheus
-- blackbox-common-config-header.sh --
cat <<EOF > config/prometheus/blackbox.cue
-- blackbox-common-config-body.cue --
package prometheus
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}
-- blackbox-common-config-git-commit.sh --
git add . && git commit -m 'add blackbox configuration'
-- blackbox-common-config-git-output.txt --
[main 1adcd08] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 components/blackbox.cue
-- common-config-patch.sh --
patch -p1 < values.patch
-- values.patch --
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,11 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
+ fullnameOverride: prometheus.blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +197,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: prometheus.blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1,5 +1,8 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
// yaml-language-server: $schema=values.schema.json
// Default values for prometheus.
@@ -1083,7 +1086,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(prometheus.blackbox.host):\(prometheus.blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
-- common-config-patch.txt --
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'
-- common-config-rm.sh --
rm values.patch
-- common-config-git.sh --
git add . && git commit -m 'integrate blackbox and prometheus together'
-- common-config-git-output.txt --
[main 4221803] integrate blackbox and prometheus together
2 files changed, 4 insertions(+), 2 deletions(-)
-- reviewing-changes-render-output.txt --
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms
-- git-diff.sh --
git diff
-- git.diff --
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target
-- reviewing-changes-git-commit.sh --
git add . && git commit -m 'render integrated blackbox and prometheus manifests'
-- reviewing-changes-git-output.txt --
[main 67efe0d] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)

View File

@@ -0,0 +1 @@
holos --version

View File

@@ -0,0 +1 @@
0.102.5

View File

@@ -0,0 +1,15 @@
package prometheus
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}

View File

@@ -0,0 +1 @@
git add . && git commit -m 'add blackbox configuration'

View File

@@ -0,0 +1,3 @@
[main e1c6859] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 config/prometheus/blackbox.cue

View File

@@ -0,0 +1 @@
cat <<EOF > config/prometheus/blackbox.cue

View File

@@ -0,0 +1,15 @@
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/blackbox/blackbox.cue

View File

@@ -0,0 +1,3 @@
[main 668706a] integrate blackbox and prometheus together
3 files changed, 1348 insertions(+), 2 deletions(-)
create mode 100644 components/prometheus/values.cue.orig

View File

@@ -0,0 +1 @@
git add . && git commit -m 'integrate blackbox and prometheus together'

View File

@@ -0,0 +1 @@
patch -p1 < values.patch

View File

@@ -0,0 +1,2 @@
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'

View File

@@ -0,0 +1 @@
rm values.patch

View File

@@ -0,0 +1 @@
EOF

View File

@@ -0,0 +1 @@
git diff

View File

@@ -0,0 +1 @@
git init . && git add . && git commit -m "initial commit"

View File

@@ -0,0 +1,64 @@
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target

View File

@@ -0,0 +1,5 @@
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml

View File

@@ -0,0 +1,5 @@
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml

View File

@@ -0,0 +1 @@
git add . && git commit -m 'import values'

View File

@@ -0,0 +1,4 @@
[main 7bc6772] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue

View File

@@ -0,0 +1,3 @@
rendered blackbox in 146.654292ms
rendered prometheus in 178.845292ms
rendered platform in 178.9115ms

View File

@@ -0,0 +1,3 @@
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5

View File

@@ -0,0 +1 @@
mkdir -p config/prometheus

View File

@@ -0,0 +1 @@
mkdir -p components/prometheus components/blackbox

View File

@@ -0,0 +1,15 @@
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/prometheus/prometheus.cue

View File

@@ -0,0 +1,12 @@
package holos
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}

View File

@@ -0,0 +1,7 @@
[main d144f24] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue

View File

@@ -0,0 +1 @@
git add . && git commit -m 'add blackbox and prometheus'

View File

@@ -0,0 +1 @@
cat <<EOF > platform/prometheus.cue

View File

@@ -0,0 +1,3 @@
rendered blackbox in 1.794799666s
rendered prometheus in 1.835097625s
rendered platform in 1.835185792s

View File

@@ -0,0 +1 @@
holos render platform

View File

@@ -0,0 +1 @@
git add . && git commit -m 'render integrated blackbox and prometheus manifests'

View File

@@ -0,0 +1,2 @@
[main 1399737] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)

View File

@@ -0,0 +1,3 @@
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms

View File

@@ -0,0 +1,4 @@
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"

View File

@@ -0,0 +1,43 @@
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,11 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
+ fullnameOverride: prometheus.blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +197,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: prometheus.blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1,5 +1,8 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
// yaml-language-server: $schema=values.schema.json
// Default values for prometheus.
@@ -1083,7 +1086,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(prometheus.blackbox.host):\(prometheus.blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"

View File

@@ -8,6 +8,7 @@ sidebar_position: 40
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import YouTube from '@site/src/components/YouTube';
import CodeBlock from '@theme/CodeBlock';
<head>
<meta property="og:title" content="Helm Values | Holos" />
@@ -41,82 +42,57 @@ resource.
## The Code
### Holos Version
Ensure you have a current version of `holos` installed. This document was
tested with the following version.
import HolosVersionCommand from '!!raw-loader!./_helm-values/script-01-holos-version/command.sh';
import HolosVersionOutput from '!!raw-loader!./_helm-values/script-01-holos-version/output.txt';
<CodeBlock language="bash">{HolosVersionCommand}</CodeBlock>
<CodeBlock language="txt">{HolosVersionOutput}</CodeBlock>
### Generating the structure
Use `holos` to generate a minimal platform directory structure. First, create
and navigate into a blank directory, then use the `holos init platform` command:
```shell
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5
```
import MkdirAndInit from '!!raw-loader!./_helm-values/script-02-helm-values/mkdir-and-init.sh';
<CodeBlock language="bash">{MkdirAndInit}</CodeBlock>
Make an initial commit to track changes:
```bash
git init . && git add . && git commit -m "initial commit"
```
import GitInit from '!!raw-loader!./_helm-values/script-02-helm-values/git-init.sh';
<CodeBlock language="bash">{GitInit}</CodeBlock>
### Managing the Components
Create the `prometheus` and `blackbox` component directories, then add each of
the following file contents.
```bash
mkdir -p components/prometheus components/blackbox
```
import MkdirComponents from '!!raw-loader!./_helm-values/script-02-helm-values/mkdir-components.sh';
import PrometheusComponentHeader from '!!raw-loader!./_helm-values/script-02-helm-values/prometheus-component-header.sh';
import PrometheusComponentBody from '!!raw-loader!./_helm-values/script-02-helm-values/prometheus-component-body.cue';
import BlackboxComponentHeader from '!!raw-loader!./_helm-values/script-02-helm-values/blackbox-component-header.sh';
import BlackboxComponentBody from '!!raw-loader!./_helm-values/script-02-helm-values/blackbox-component-body.cue';
import EofTrailer from '!!raw-loader!./_helm-values/script-02-helm-values/eof-trailer.sh';
<CodeBlock language="bash">{MkdirComponents}</CodeBlock>
<Tabs groupId="D15A3008-1EFC-4D34-BED1-15BC0C736CC3">
<TabItem value="prometheus.cue" label="prometheus.cue">
```bash
cat <<EOF > components/prometheus/prometheus.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
```
```bash
EOF
```
<CodeBlock language="bash">{PrometheusComponentHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{PrometheusComponentBody}</CodeBlock>
<CodeBlock language="bash">{EofTrailer}</CodeBlock>
</TabItem>
<TabItem value="blackbox.cue" label="blackbox.cue">
```bash
cat <<EOF > components/blackbox/blackbox.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
```
```bash
EOF
```
<CodeBlock language="bash">{BlackboxComponentHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{BlackboxComponentBody}</CodeBlock>
<CodeBlock language="bash">{EofTrailer}</CodeBlock>
</TabItem>
</Tabs>
@@ -124,64 +100,38 @@ EOF
Register the components with the platform by adding the following file to the platform directory.
```bash
cat <<EOF > platform/prometheus.cue
```
```cue showLineNumbers
package holos
import RegisterComponentsHeader from '!!raw-loader!./_helm-values/script-02-helm-values/register-components-header.sh';
import RegisterComponentsBody from '!!raw-loader!./_helm-values/script-02-helm-values/register-components-body.cue';
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}
```
```bash
EOF
```
<CodeBlock language="bash">{RegisterComponentsHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{RegisterComponentsBody}</CodeBlock>
<CodeBlock language="bash">{EofTrailer}</CodeBlock>
Render the platform.
import RenderCommand from '!!raw-loader!./_helm-values/script-02-helm-values/render.sh';
import RegisterComponentsRenderOutput from '!!raw-loader!./_helm-values/script-02-helm-values/register-components-output.txt';
<Tabs groupId="33D6BFED-62D8-4A42-A26A-F3121D57C4E5">
<TabItem value="command" label="Command">
```bash
holos render platform
```
<CodeBlock language="bash">{RenderCommand}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
cached prometheus-blackbox-exporter 9.0.1
rendered blackbox in 3.825430417s
cached prometheus 25.27.0
rendered prometheus in 4.840089667s
rendered platform in 4.840137792s
```
<CodeBlock language="txt">{RegisterComponentsRenderOutput}</CodeBlock>
</TabItem>
</Tabs>
Commit the results.
import GitCommitRegisterComponents from '!!raw-loader!./_helm-values/script-02-helm-values/register-components-git-commit.sh';
import RegisterComponentsGitOutput from '!!raw-loader!./_helm-values/script-02-helm-values/register-components-git-commit-output.txt';
<Tabs groupId="446CC550-A634-45C0-BEC7-992E5C56D4FA">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add blackbox and prometheus'
```
<CodeBlock language="bash">{GitCommitRegisterComponents}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
[main b5df111] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue
```
<CodeBlock language="txt">{RegisterComponentsGitOutput}</CodeBlock>
</TabItem>
</Tabs>
@@ -190,21 +140,11 @@ git add . && git commit -m 'add blackbox and prometheus'
Holos renders Helm charts with their default values. We can import these default
values into CUE to work with them as structured data instead of text markup.
```bash
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml
```
import ImportPrometheusValues from '!!raw-loader!./_helm-values/script-02-helm-values/import-prometheus-values.sh';
import ImportBlackboxValues from '!!raw-loader!./_helm-values/script-02-helm-values/import-blackbox-values.sh';
```bash
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
```
<CodeBlock language="bash">{ImportPrometheusValues}</CodeBlock>
<CodeBlock language="bash">{ImportBlackboxValues}</CodeBlock>
These commands convert the YAML data into CUE code and nest the values under the
`Values` field of the `Helm` struct.
@@ -215,67 +155,43 @@ CUE unifies `values.cue` with the other `\*.cue` files in the same directory.
Render the platform using `holos render platform` and commit the results.
import ImportValuesRenderOutput from '!!raw-loader!./_helm-values/script-02-helm-values/import-values-render-output.txt';
import ImportValuesGitCommit from '!!raw-loader!./_helm-values/script-02-helm-values/import-values-git-commit.sh';
import ImportValuesGitOutput from '!!raw-loader!./_helm-values/script-02-helm-values/import-values-git-output.txt';
<Tabs groupId="BDDCD65A-2E9D-4BA6-AAE2-8099494D5E4B">
<TabItem value="command" label="Command">
```bash
holos render platform
```
<CodeBlock language="bash">{RenderCommand}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered blackbox in 365.936792ms
rendered prometheus in 371.855875ms
rendered platform in 372.109916ms
```
<CodeBlock language="txt">{ImportValuesRenderOutput}</CodeBlock>
</TabItem>
</Tabs>
<Tabs groupId="1636C619-258E-4D49-8052-F64B588C9177">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'import values'
```
<CodeBlock language="bash">{ImportValuesGitCommit}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 52e90ea] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue
```
<CodeBlock language="txt">{ImportValuesGitOutput}</CodeBlock>
</TabItem>
</Tabs>
### Managing Common Configuration
To manage shared configuration for both Helm charts, define a structure that
holds the common configuration values. Place this configuration in the
`components` directory to ensure it is accessible to all components.
holds the common configuration values. Create a `config` directory at the root
of the repository, and place the configuration file there to ensure it is
accessible to all components.
import BlackboxCommonConfigMkdir from '!!raw-loader!./_helm-values/script-02-helm-values/mkdir-common-config.sh';
import BlackboxCommonConfigHeader from '!!raw-loader!./_helm-values/script-02-helm-values/blackbox-common-config-header.sh';
import BlackboxCommonConfigBody from '!!raw-loader!./_helm-values/script-02-helm-values/blackbox-common-config-body.cue';
```bash
cat <<EOF > components/blackbox.cue
```
```cue showLineNumbers
package holos
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
Blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}
```
```bash
EOF
```
<CodeBlock language="bash">{BlackboxCommonConfigMkdir}</CodeBlock>
<CodeBlock language="bash">{BlackboxCommonConfigHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{BlackboxCommonConfigBody}</CodeBlock>
<CodeBlock language="bash" showLineNumbers>{EofTrailer}</CodeBlock>
:::important
1. CUE loads and unifies all `*.cue` files from the root directory containing
@@ -286,75 +202,41 @@ languages with only type checking.
Add and commit the configuration.
import BlackboxCommonConfigGit from '!!raw-loader!./_helm-values/script-02-helm-values/blackbox-common-config-git-commit.sh';
import BlackboxCommonConfigGitOutput from '!!raw-loader!./_helm-values/script-02-helm-values/blackbox-common-config-git-output.txt';
<Tabs groupId="A738CCE4-F0C6-4CC7-BE1F-2B92F0E86FDC">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add blackbox configuration'
```
<CodeBlock language="bash">{BlackboxCommonConfigGit}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 1adcd08] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 components/blackbox.cue
```
<CodeBlock language="bash">{BlackboxCommonConfigGitOutput}</CodeBlock>
</TabItem>
</Tabs>
### Using Common Configuration Across Components
Referencing common configuration across multiple components is straightforward
and reliable using Holos and CUE.
and reliable using Holos and CUE. Configuration can be imported where necessary
following [CUE module standards], which are similar to Golang.
To apply the common configuration, patch the two `values.cue` files, or manually
edit them to reference `Blackbox.host` and `Blackbox.port`.
edit them to import the configuration and reference `prometheus.blackbox.host`
and `prometheus.blackbox.port`.
import CommonConfigPatchCommand from '!!raw-loader!./_helm-values/script-02-helm-values/common-config-patch.sh';
import CommonConfigPatchDiff from '!!raw-loader!./_helm-values/script-02-helm-values/values.patch';
import CommonConfigPatchOutput from '!!raw-loader!./_helm-values/script-02-helm-values/common-config-patch.txt';
<Tabs groupId="5FFCE892-B8D4-4F5B-B2E2-39EC9E9F87A4">
<TabItem value="command" label="Command">
```bash
patch -p1 < values.patch
```
<CodeBlock language="bash">{CommonConfigPatchCommand}</CodeBlock>
</TabItem>
<TabItem value="patch" label="values.patch">
```diff
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,8 @@
package holos
Helm: Values: {
+ fullnameOverride: Blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +194,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: Blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1083,7 +1083,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(Blackbox.host):\(Blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
```
<CodeBlock language="diff">{CommonConfigPatchDiff}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'
```
<CodeBlock language="txt">{CommonConfigPatchOutput}</CodeBlock>
</TabItem>
</Tabs>
@@ -365,20 +247,17 @@ safely and easily.
Remove the patch file, then commit the changes.
import CommonConfigPatchRm from '!!raw-loader!./_helm-values/script-02-helm-values/common-config-rm.sh';
import CommonConfigPatchGitCommit from '!!raw-loader!./_helm-values/script-02-helm-values/common-config-git.sh';
import CommonConfigPatchGitCommitOutput from '!!raw-loader!./_helm-values/script-02-helm-values/common-config-git-output.txt';
<Tabs groupId="6498B00E-FADA-4EB2-885C-808F1D22E04D">
<TabItem value="command" label="Command">
```bash
rm values.patch
```
```bash
git add . && git commit -m 'integrate blackbox and prometheus together'
```
<CodeBlock language="bash">{CommonConfigPatchRm}</CodeBlock>
<CodeBlock language="bash">{CommonConfigPatchGitCommit}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 4221803] integrate blackbox and prometheus together
2 files changed, 4 insertions(+), 2 deletions(-)
```
<CodeBlock language="txt">{CommonConfigPatchGitCommitOutput}</CodeBlock>
</TabItem>
</Tabs>
@@ -387,97 +266,28 @@ git add . && git commit -m 'integrate blackbox and prometheus together'
Holos makes it easy to view and review platform-wide changes. Render the
platform to observe how both Prometheus and Blackbox update in sync.
import ReviewingChangesRenderOutput from '!!raw-loader!./_helm-values/script-02-helm-values/reviewing-changes-render-output.txt';
<Tabs groupId="E7F6D8B1-22FA-4075-9B44-D9F2815FE0D3">
<TabItem value="command" label="Command">
```bash
holos render platform
```
<CodeBlock language="bash">{RenderCommand}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms
```
<CodeBlock language="txt">{ReviewingChangesRenderOutput}</CodeBlock>
</TabItem>
</Tabs>
Changes are easily visible in version control.
import GitDiffCommand from '!!raw-loader!./_helm-values/script-02-helm-values/git-diff.sh';
import GitDiff from '!!raw-loader!./_helm-values/script-02-helm-values/git.diff';
<Tabs groupId="9789A0EF-24D4-4FB9-978A-3895C2778789">
<TabItem value="command" label="Command">
```bash
git diff
```
<CodeBlock language="bash">{GitDiffCommand}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target
```
<CodeBlock language="diff">{GitDiff}</CodeBlock>
</TabItem>
</Tabs>
@@ -494,17 +304,15 @@ Blackbox host or port will reconfigure both charts correctly.
Commit the changes and proceed to deploy them.
import ReviewingChangesGitCommit from '!!raw-loader!./_helm-values/script-02-helm-values/reviewing-changes-git-commit.sh';
import ReviewingChangesGitOutput from '!!raw-loader!./_helm-values/script-02-helm-values/reviewing-changes-git-output.txt';
<Tabs groupId="F8C9A98D-DE1E-4EF6-92C1-017A9166F6C7">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'render integrated blackbox and prometheus manifests'
```
<CodeBlock language="bash">{ReviewingChangesGitCommit}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 67efe0d] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)
```
<CodeBlock language="txt">{ReviewingChangesGitOutput}</CodeBlock>
</TabItem>
</Tabs>
@@ -524,7 +332,7 @@ service endpoint.
[prometheus]: https://github.com/prometheus-community/helm-charts/tree/prometheus-25.27.0/charts/prometheus
[blackbox]: https://github.com/prometheus-community/helm-charts/tree/prometheus-blackbox-exporter-9.0.1/charts/prometheus-blackbox-exporter
[httpbin]: https://github.com/mccutchen/go-httpbin/tree/v2.15.0
[CUE module standards]: https://cuelang.org/docs/concept/modules-packages-instances/
[Config Schema]: #config-schema
[Technical Overview]: ./overview.mdx

View File

@@ -0,0 +1,87 @@
package main
import (
"os"
"path/filepath"
"runtime"
"slices"
"strings"
"testing"
"github.com/holos-run/holos/cmd"
"github.com/rogpeppe/go-internal/testscript"
cue "cuelang.org/go/cmd/cue/cmd"
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cmd.MakeMain(),
"cue": cue.Main,
}))
}
// Run these with go test -v to see the verbose names
func TestHelmValues(t *testing.T) {
t.Run("TestHelmValues", func(t *testing.T) {
// Get an ordered list of test script files.
dir := "_helm-values"
for _, file := range sortedTestScripts(t, filepath.Join(dir, "examples")) {
t.Run("examples", func(t *testing.T) {
runOneScript(t, dir, file)
})
}
})
}
func runOneScript(t *testing.T, dir string, file string) {
params := testscript.Params{
Dir: "",
Files: []string{file},
RequireExplicitExec: true,
RequireUniqueNames: false,
WorkdirRoot: filepath.Join(testDir(t), dir),
UpdateScripts: os.Getenv("HOLOS_UPDATE_SCRIPTS") != "",
Setup: func(env *testscript.Env) error {
// Needed for update.sh to determine if we need to update output files.
env.Setenv("HOLOS_UPDATE_SCRIPTS", os.Getenv("HOLOS_UPDATE_SCRIPTS"))
// Just like cmd/cue/cmd.TestScript, set up separate cache and config dirs per test.
env.Setenv("CUE_CACHE_DIR", filepath.Join(env.WorkDir, "tmp/cachedir"))
configDir := filepath.Join(env.WorkDir, "tmp/configdir")
env.Setenv("CUE_CONFIG_DIR", configDir)
return nil
},
}
testscript.Run(t, params)
}
// testDir returns the path of the directory containing the go source file of
// the caller.
func testDir(t *testing.T) string {
_, file, _, ok := runtime.Caller(0)
if !ok {
t.Fatal("could not get runtime caller")
}
return filepath.Dir(file)
}
func sortedTestScripts(t *testing.T, dir string) (files []string) {
entries, err := os.ReadDir(dir)
if os.IsNotExist(err) {
// Continue to helpful error on len(files) == 0 below.
} else if err != nil {
t.Fatal(err)
}
for _, entry := range entries {
name := entry.Name()
if strings.HasSuffix(name, ".txtar") || strings.HasSuffix(name, ".txt") {
files = append(files, filepath.Join(dir, name))
}
}
if len(files) == 0 {
t.Fatalf("no txtar nor txt scripts found in dir %s", dir)
}
slices.Sort(files)
return files
}

View File

@@ -15,6 +15,7 @@
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
"raw-loader": "^4.0.2",
"react": "^18.0.0",
"react-dom": "^18.0.0"
},
@@ -14477,6 +14478,44 @@
"node": ">= 0.8"
}
},
"node_modules/raw-loader": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/raw-loader/-/raw-loader-4.0.2.tgz",
"integrity": "sha512-ZnScIV3ag9A4wPX/ZayxL/jZH+euYb6FcUinPcgiQW0+UBtEv0O6Q3lGd3cqJ+GHH+rksEv3Pj99oxJ3u3VIKA==",
"license": "MIT",
"dependencies": {
"loader-utils": "^2.0.0",
"schema-utils": "^3.0.0"
},
"engines": {
"node": ">= 10.13.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/webpack"
},
"peerDependencies": {
"webpack": "^4.0.0 || ^5.0.0"
}
},
"node_modules/raw-loader/node_modules/schema-utils": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/schema-utils/-/schema-utils-3.3.0.tgz",
"integrity": "sha512-pN/yOAvcC+5rQ5nERGuwrjLlYvLTbCibnZ1I7B1LaiAz9BRBlE9GMgE/eqV30P7aJQUf7Ddimy/RsbYO/GrVGg==",
"license": "MIT",
"dependencies": {
"@types/json-schema": "^7.0.8",
"ajv": "^6.12.5",
"ajv-keywords": "^3.5.2"
},
"engines": {
"node": ">= 10.13.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/webpack"
}
},
"node_modules/rc": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",

View File

@@ -22,6 +22,7 @@
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
"raw-loader": "^4.0.2",
"react": "^18.0.0",
"react-dom": "^18.0.0"
},

View File

@@ -6,6 +6,8 @@
/docs/quickstart/ /docs/v1alpha5/tutorial/overview/ 301
/docs/overview /docs/v1alpha5/tutorial/overview/ 301
/docs/overview/ /docs/v1alpha5/tutorial/overview/ 301
/docs/guides /docs/v1alpha5/tutorial/overview/ 301
/docs/guides/ /docs/v1alpha5/tutorial/overview/ 301
/docs/topics /docs/v1alpha5/topics/ 301
/docs/topics/ /docs/v1alpha5/topics/ 301
/docs/setup /docs/v1alpha5/tutorial/setup/ 301
@@ -20,3 +22,12 @@
/docs/comparison/ /docs/v1alpha5/topics/comparison/ 301
/docs/support /docs/v1alpha5/tutorial/overview/#getting-help 301
/docs/support/ /docs/v1alpha5/tutorial/overview/#getting-help 301
/docs/api/core /docs/v1alpha5/api/core/ 301
/docs/api/core/ /docs/v1alpha5/api/core/ 301
/docs/api/author /docs/v1alpha5/api/author/ 301
/docs/api/author/ /docs/v1alpha5/api/author/ 301
# Redirect to our own take on the pattern once we figure out what to call it.
# See: https://discord.com/channels/1299051862418395216/1299055980738383882/1327347525354524805
/blog/the-rendered-manifests-pattern https://akuity.io/blog/the-rendered-manifests-pattern 301
# without the s which I always forget
/blog/the-rendered-manifest-pattern https://akuity.io/blog/the-rendered-manifests-pattern 301

View File

@@ -28,6 +28,7 @@ var cueMutex sync.Mutex
// directory are loaded non-recursively.
//
// Attribution: https://github.com/cue-lang/cue/issues/3504
// Deprecated: Use cue embed instead.
func ExtractYAML(ctxt *cue.Context, filepaths []string) (cue.Value, error) {
value := ctxt.CompileString("")
files := make([]string, 0, 10*len(filepaths))
@@ -95,7 +96,6 @@ func LoadInstance(path string, filepaths []string, tags []string) (*Instance, er
if err != nil {
return nil, errors.Wrap(err)
}
// TODO: https://cuelang.org/docs/howto/place-data-go-api/
value = value.Unify(values[0])
inst := &Instance{

View File

@@ -16,7 +16,7 @@ import (
// platform.
type PlatformOpts struct {
Fn func(context.Context, int, holos.Component) error
Selector holos.Selector
Selectors holos.Selectors
Concurrency int
InfoEnabled bool
}
@@ -31,7 +31,7 @@ type Platform struct {
func (p *Platform) Build(ctx context.Context, opts PlatformOpts) error {
limit := max(opts.Concurrency, 1)
parentStart := time.Now()
components := p.Select(opts.Selector)
components := p.Select(opts.Selectors...)
total := len(components)
g, ctx := errgroup.WithContext(ctx)

View File

@@ -219,13 +219,38 @@ func (t generatorTask) helm(ctx context.Context) error {
}
}
// Write values file
// Write value files
tempDir, err := os.MkdirTemp("", "holos.helm")
if err != nil {
return errors.Format("could not make temp dir: %w", err)
}
defer util.Remove(ctx, tempDir)
// valueFiles represents the ordered list of value files to pass to helm
// template -f
var valueFiles []string
// valueFiles for the use case of migration from helm value hierarchies.
for _, valueFile := range t.generator.Helm.ValueFiles {
var data []byte
switch valueFile.Kind {
case "Values":
if data, err = yaml.Marshal(valueFile.Values); err != nil {
return errors.Format("could not marshal value file %s: %w", valueFile.Name, err)
}
default:
return errors.Format("could not marshal value file %s: unknown kind %s", valueFile.Name, valueFile.Kind)
}
valuesPath := filepath.Join(tempDir, valueFile.Name)
if err := os.WriteFile(valuesPath, data, 0666); err != nil {
return errors.Wrap(fmt.Errorf("could not write value file %s: %w", valueFile.Name, err))
}
log.DebugContext(ctx, fmt.Sprintf("wrote: %s", valuesPath))
valueFiles = append(valueFiles, valuesPath)
}
// The final values files
data, err := yaml.Marshal(t.generator.Helm.Values)
if err != nil {
return errors.Format("could not marshal values: %w", err)
@@ -235,7 +260,8 @@ func (t generatorTask) helm(ctx context.Context) error {
if err := os.WriteFile(valuesPath, data, 0666); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "wrote"+valuesPath)
log.DebugContext(ctx, fmt.Sprintf("wrote: %s", valuesPath))
valueFiles = append(valueFiles, valuesPath)
// Run charts
args := []string{"template"}
@@ -248,9 +274,11 @@ func (t generatorTask) helm(ctx context.Context) error {
if kubeVersion := t.generator.Helm.KubeVersion; kubeVersion != "" {
args = append(args, "--kube-version", kubeVersion)
}
args = append(args, "--include-crds")
for _, valueFilePath := range valueFiles {
args = append(args, "--values", valueFilePath)
}
args = append(args,
"--include-crds",
"--values", valuesPath,
"--namespace", t.generator.Helm.Namespace,
"--kubeconfig", "/dev/null",
"--version", t.generator.Helm.Chart.Version,

View File

@@ -45,8 +45,8 @@ func newPlatform(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
cmd.Flags().StringVar(&platform, "platform", "./platform", "platform directory path")
var extractYAMLs holos.StringSlice
cmd.Flags().Var(&extractYAMLs, "extract-yaml", "data file paths to extract and unify with the platform config")
var selector holos.Selector
cmd.Flags().VarP(&selector, "selector", "l", "label selector (e.g. label==string,label!=string)")
var selectors holos.Selectors
cmd.Flags().VarP(&selectors, "selector", "l", "label selector (e.g. label==string,label!=string)")
tagMap := make(holos.TagMap)
cmd.Flags().VarP(&tagMap, "inject", "t", tagHelp)
@@ -75,7 +75,7 @@ func newPlatform(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
}
opts := builder.PlatformOpts{
Fn: makeComponentRenderFunc(cmd.ErrOrStderr(), prefixArgs, tagMap.Tags()),
Selector: selector,
Selectors: selectors,
Concurrency: concurrency,
InfoEnabled: true,
}

View File

@@ -70,8 +70,8 @@ func newShowBuildPlanCmd() (cmd *cobra.Command) {
cmd.Flags().Var(&extractYAMLs, "extract-yaml", "data file paths to extract and unify with the platform config")
var format string
cmd.Flags().StringVar(&format, "format", "yaml", "yaml or json format")
var selector holos.Selector
cmd.Flags().VarP(&selector, "selector", "l", "label selector (e.g. label==string,label!=string)")
var selectors holos.Selectors
cmd.Flags().VarP(&selectors, "selector", "l", "label selector (e.g. label==string,label!=string)")
tagMap := make(holos.TagMap)
cmd.Flags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
var concurrency int
@@ -102,7 +102,7 @@ func newShowBuildPlanCmd() (cmd *cobra.Command) {
platformOpts := builder.PlatformOpts{
Fn: makeBuildFunc(encoder, buildPlanOpts),
Selector: selector,
Selectors: selectors,
Concurrency: concurrency,
}

View File

@@ -95,6 +95,10 @@ import "github.com/holos-run/holos/api/core/v1alpha5:core"
// Values represents data to marshal into a values.yaml for helm.
Values: core.#Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles?: [...core.#ValueFile] @go(,[]core.ValueFile)
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks: bool & (true | *false)

View File

@@ -129,9 +129,14 @@ package core
chart: #Chart @go(Chart)
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
// rendering the chart. Values follow ValueFiles when both are provided.
values: #Values @go(Values)
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
valueFiles?: [...#ValueFile] @go(ValueFiles,[]ValueFile)
// EnableHooks enables helm hooks when executing the `helm template` command.
enableHooks?: bool @go(EnableHooks)
@@ -145,6 +150,19 @@ package core
kubeVersion?: string @go(KubeVersion)
}
// ValueFile represents one Helm value file produced from CUE.
#ValueFile: {
// Name represents the file name, e.g. "region-values.yaml"
name: string @go(Name)
// Kind is a discriminator.
kind: string & "Values" @go(Kind)
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
values?: #Values @go(Values)
}
// Values represents [Helm] Chart values generated from CUE.
#Values: {...}

View File

@@ -1,2 +1,2 @@
module: "example.com/platform"
language: version: "v0.11.0"
module: "holos.example"
language: version: "v0.11.1"

View File

@@ -95,6 +95,7 @@ import (
release: string | *name
}
Values: _
ValueFiles?: _
EnableHooks: _
Namespace?: _
APIVersions?: _
@@ -110,8 +111,11 @@ import (
kind: "Helm"
output: HelmOutput
helm: core.#Helm & {
chart: Chart
values: Values
chart: Chart
values: Values
if ValueFiles != _|_ {
valueFiles: ValueFiles
}
enableHooks: EnableHooks
if Namespace != _|_ {
namespace: Namespace

View File

@@ -107,6 +107,28 @@ func (e *EnvFlagger) Flag(name feature) bool {
type Labels map[string]string
type Selectors []Selector
// String implements the flag.Value interface.
func (s *Selectors) String() string {
return fmt.Sprint(*s)
}
// Type implements the pflag.Value interface and describes the type.
func (s *Selectors) Type() string {
return "selectors"
}
// Set implements the flag.Value interface.
func (s *Selectors) Set(value string) error {
selector := Selector{}
if err := selector.Set(value); err != nil {
return err
}
*s = append(*s, selector)
return nil
}
type Selector struct {
Positive map[string]string
Negative map[string]string
@@ -118,14 +140,9 @@ func (s *Selector) IsSelected(labels Labels) bool {
return true // Nil selector selects everything
}
if len(s.Positive) == 0 && len(s.Negative) == 0 {
return true // Empty selector selects everything
}
// Check positive matches
for k, v := range s.Positive {
val, ok := labels[k]
if !ok || v != val {
if val, ok := labels[k]; !ok || v != val {
return false
}
}
@@ -251,15 +268,18 @@ func (y *yamlEncoder) Close() error {
return errors.Wrap(y.enc.Close())
}
// IsSelected returns true if all selectors select the given labels or no
// IsSelected returns true if any one selector selects the given labels or no
// selectors are given.
func IsSelected(labels Labels, selectors ...Selector) bool {
if len(selectors) == 0 {
return true
}
for _, selector := range selectors {
if !selector.IsSelected(labels) {
return false
if selector.IsSelected(labels) {
return true
}
}
return true
return false
}
type orderedEncoder struct {

View File

@@ -1 +1 @@
102
103

View File

@@ -1 +1 @@
3
0