Compare commits

...

33 Commits

Author SHA1 Message Date
Jeff McCune
13665fab55 core: define generic command core type for all kinds of tasks
Previously the Command core type was only useful for Validators and was
a bit hacky in that Holos appends the directory to the end of the
argument vector.

This patch changes the Command type to represent a generic command for
use as a Generator, Transformer, or Validator.  The type is extended to
support the semantics of reading output from a file, directory, or
standard output pipe.

Go templates are used to fill in Holos managed data values, for example
the temporary directory associated with the pipeline task.  The TaskData
structu represents these values passed into the template engine.
2025-03-03 17:07:56 -08:00
Jeff McCune
5a2571a745 docs: make generate for docs preview 2025-02-27 15:47:20 -08:00
Jeff McCune
bfd8b20b6a fixup: don't store a key with a trailing / in the artifact map 2025-02-27 15:29:27 -08:00
Jeff McCune
07cd8737b0 artifact: write multiple files if path ends in a /
To properly support the kubectl-slice use case the Artifact map needs to
write multiple files out to the filesystem.  This needs to be dynamic in
the sense holos and the end user don't know what files the kubectl-slice
transformer is producing.

As a hack, which may actually turn out to be "good enough" this patch
makes the Slice transformer behave like so:

1. Execute kubectl-slice outputting to an empty temp directory.
2. Holos saves all files in this directory into the artifact map.
3. At the end of the Artifact pipeline, if the final artifact produced
   ends in a /, then all keys in the artifact map having the prefix are
   written to the output directory.

This should be sufficient for the use case, but we'll need to consider
how this transformer and apporach works when subsequent transformers are
used in the pipeline.  I haven't thought deeply about it, but it should
ideally work pretty well if the tools involved truly only care about
directories and not the files within the directory.
2025-02-27 14:37:44 -08:00
Jeff McCune
ff5bdab948 transformer: wire up Slice transformer type
Copied from kustomize, spike for spiarh.

Result: head to artifact.go next.

could not run: could not save deploy/slice/components/slice: open deploy/slice/components/slice: is a directory at internal/artifact/artifact.go:71
could not run: could not render component: could not run command:
        holos '--log-level' 'debug' '--log-format' 'console' 'render' 'component' '--inject' 'outputBaseDir=slice' '--inject' 'holos_component_name=slice' '--inject' 'holos_component_path=components/slice' '--inject' 'holos_component_labels={"holos.run/component.name":"slice"}' '--inject' 'holos_component_annotations={"app.holos.run/description":"slice transformer"}' './components/slice'
        exit status 1 at cli/render/render.go:171

Relevant debug logs:

running command: kubectl 'kustomize' '/var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize4273326823'
tmp: removed
running command: kubectl-slice '-f' '/var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.slice2683550041/slice.gen.yaml' '-o' '/var/folders/22/zt67pphj6h1fgknqfy23ppl8000
0gn/T/holos.slice2683550041/slice'
storing: /var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.slice2683550041/slice/deployment-httpbin.yaml
storing: /var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.slice2683550041/slice/service-httpbin.yaml
tmp: removed
2025-02-27 12:56:01 -08:00
Jeff McCune
40093956d3 tools: add kubectl-slice 2025-02-27 09:22:13 -08:00
Jeff McCune
8e690b43ee ci: fix golangci-lint
Without this patch there are unexpected lint errors in version 1.60
where 1.61.0 passes locally on my machine.

This patch updates to:

    golangci-lint has version 1.64.5 built with go1.24.0 from 0a603e49 on 2025-02-13T21:19:55Z
2025-02-20 09:28:35 -08:00
Jeff McCune
a4ceb1cdb2 ci: update test cases with make bump
Previously using make bump to bump a version did not also update all of
the test cases and documentation to reflect the new version.  This patch
updates the make bump tasks call HOLOS_UPDATE_SCRIPTS=1 scripts/test to
keep the test cases and documentation in sync with the new version.
2025-02-20 09:14:37 -08:00
Jeff McCune
ddb5c0e07b ci: fix make lint failures resulting from version bumps
Without this patch lint fails with error:

    SA1019: testscript.RunMain is deprecated: use [Main]

This patch uses testscript.Main instead.
2025-02-20 09:07:47 -08:00
Jeff McCune
a14d3ba0f4 ci: fix make test failures resulting from version bumps
Previously the tests fail because they were not updated to use the new
version string in holos, or the new topo sort behavior in cue 0.12.0.

This patch updates the test cases using:

    HOLOS_UPDATE_SCRIPTS=1 scripts/test

Result: make test passes
2025-02-20 08:37:01 -08:00
Jeff McCune
f7e0470c48 version 0.104.0 with cue 0.12.0 2025-02-06 14:37:50 -08:00
Jeff McCune
d5c7b82684 go mod tidy 2025-02-06 14:33:14 -08:00
Jeff McCune
7d0392e596 update to cue 0.12.0
Most relevant for us: lots of fixes to the evaluator, enables the embed
and toposort experiments.
2025-02-06 14:31:39 -08:00
Gary Larizza
410b882d1d Merge pull request #403 from holos-run/gl/hello-holos-testscript
docs: Update Hello Holos tutorial to use testscript
2025-01-22 14:43:47 -08:00
Gary Larizza
e2648202dc Merge pull request #404 from holos-run/gl/kustomize-testscript
docs: Update Kustomize tutorial to use testscript
2025-01-22 14:43:33 -08:00
Jeff McCune
44c2fe220a test: fix helm capabilities test
Helm was upgraded in GitHub Actions resulting in an accidental failure
of the test case.
2025-01-17 12:33:28 -08:00
Jeff McCune
fe1ae2fa80 docs: migrate from an ApplicationSet blog post 2025-01-17 12:22:56 -08:00
Gary Larizza
8fbee1cbd9 docs: Update Kustomize tutorial to use testscript
PROBLEM:

The "Kustomize" tutorial has hardcoded code blocks and hasn't been
updated to use the automated testscript workflow.

SOLUTION:

Create a test for the Kustomize tutorial.
Create a testscript for the Kustomize test.
Update the Kustomize MDX file to load in data from the testscript directory.

OUTCOME:

The code content in the Kustomize tutorial now comes directly from the
testscript workflow.
2025-01-16 14:24:24 -08:00
Gary Larizza
982db2cccc docs: Update Hello Holos tutorial to use testscript
PROBLEM:

The "Hello Holos" tutorial has hardcoded code blocks and hasn't been
updated to use the automated testscript workflow.

SOLUTION:

* Create a test for the Hello Holos tutorial.
* Create a testscript for the Hello Holos test.
* Update the Hello Holos MDX file to load in data from the testscript directory.

OUTCOME:

The code content in the Hello Holos tutorial now comes directly from the
testscript workflow.
2025-01-16 10:12:17 -08:00
Jeff McCune
e9d1240d63 docs: make update-docs for version 0.103.0 2025-01-12 14:26:27 -08:00
Gary Larizza
03fa4eaaa2 docs: Helm Values test updates
* Convert all files with.period.separators to hyphen-separators.
* Rename and markdown_test.go to be specific to Helm Values.
* Move helm-values_test.go to be in the same directory as the Helm Values doc.
* Move Blackbox common configuration CUE file to `config/prometheus` so it can be imported as necessary.
* Use explicit import statements for Blackbox common config in `blackbox` and `prometheus` components.

Closes: #399
2025-01-12 14:25:44 -08:00
Jeff McCune
e363f3a597 docs: add make update-docs task
We need to run this prior to tagging a release otherwise the tests fail
for the new version string.
2025-01-12 14:22:58 -08:00
Jeff McCune
8b49ed93be docs: release version 0.103.0 2025-01-12 14:09:45 -08:00
Jeff McCune
d2be9fe278 helm: add valueFiles for migration from an ApplicationSet
Without this patch migrating from [helm hierarchies] to Holos requires
the user to unify the value hierarchy.  This is a problem because helm
hierarchies are difficult to unify because it's not clear if or why a
value is used in the final results.  This makes it difficult to identify
how to resolve conflicts.

This patch adds `valueFiles` field to the Helm component kind.  This
field is intended to provide a direct migration path from the
ApplicationSet.spec.template.spec.sources.helm.valueFiles field.  With
this patch, users can directly migrate the values files to CUE using
`@embed`, then directly migrate the valueFiles field to reference the
values from within CUE.

Note we actively discourage the use of Helm value hierarchies.  The
feature is intended as a temporary migration tool.  We encourage the use
of CUE unification instead.  After migration, the valueFiles field
should be refactored to the values field as one unified structure in
CUE.  The valueFiles field makes this second order migration easier
becuase we can inspect and verify the complete rendered output, allowing
us to determine if a value is actually used in the final configuration
or is overridden.

[helm hierarchies]: https://medium.com/containers-101/using-helm-hierarchies-in-multi-source-argo-cd-applications-for-promoting-to-different-gitops-133c3bc93678
2025-01-12 13:30:29 -08:00
Jeff McCune
6ec341bbb1 docs: redirect /docs/api/core 2025-01-10 15:02:12 -08:00
Jeff McCune
13a4305b78 docs: add redirect for /blog/rendered-manifest-pattern (manifest instead of manifests) 2025-01-10 14:50:26 -08:00
Jeff McCune
0cfce3a823 docs: redirect rendered manifests pattern for now
Need a URL we can redirect when we publish our own variation on the
pattern with a link back to Akuity.
2025-01-10 10:55:06 -08:00
Jeff McCune
61d7539e1c docs: fix /docs/guides/ redirect 2025-01-09 16:03:50 -08:00
Jeff McCune
bf84724137 docs: add redirects for github.com/holos-run readme 2025-01-09 15:11:04 -08:00
Jeff McCune
9f0de7555c init: change to holos.example default cue module
Match the cue mod init behavior of a module named `cue.example`.
2025-01-09 13:57:26 -08:00
Gary Larizza
650636f944 Merge pull request #393 from holos-run/gl/update-helm-docs
Update Helm Values Tutorial to use testscript
2025-01-09 12:01:09 -08:00
Gary Larizza
b28c110694 Update Helm Values tutorial to use testscript
PROBLEM:

The Helm Values tutorial contains a fair bit of code/scripts, and we
need a way to test the steps we recommend to make sure nothing breaks
or slips out of date.

SOLUTION:

* Use `testscript` as a way to automate the execution of the steps in the doc and verify that none of the steps produce errors.
* Update the MDX file to directly reference the files embedded into the testscript.

OUTCOME:

* We have an automated way to perform the steps in the Helm Values document.
* We have unit tests that will fail should any of the commands being executed in the doc fail.
* The doc's MDX file directly references the files within the testscript, so we only need to modify the MDX file to update wording.
2025-01-09 11:53:53 -08:00
Gary Larizza
5bb3e90b38 Install raw-loader module
We use this module within our markdown tutorials (like the Helm Values
tutorial) to load in files generated by testscript.
2025-01-09 11:53:13 -08:00
185 changed files with 4750 additions and 843 deletions

View File

@@ -27,4 +27,4 @@ jobs:
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.60
version: v1.64.5

1
.gitignore vendored
View File

@@ -12,3 +12,4 @@ tmp/
/holos-k3d/
/holos-infra/
node_modules/
.tmp/

View File

@@ -32,17 +32,20 @@ bump: bumppatch
.PHONY: bumppatch
bumppatch: ## Bump the patch version.
scripts/bump patch
HOLOS_UPDATE_SCRIPTS=1 scripts/test
.PHONY: bumpminor
bumpminor: ## Bump the minor version.
scripts/bump minor
scripts/bump patch 0
HOLOS_UPDATE_SCRIPTS=1 scripts/test
.PHONY: bumpmajor
bumpmajor: ## Bump the major version.
scripts/bump major
scripts/bump minor 0
scripts/bump patch 0
HOLOS_UPDATE_SCRIPTS=1 scripts/test
.PHONY: show-version
show-version: ## Print the full version.
@@ -75,6 +78,12 @@ build: ## Build holos executable.
@echo "GOPATH=${GOPATH}"
go build -trimpath -o bin/$(BIN_NAME) -ldflags $(LD_FLAGS) $(REPO_PATH)/cmd/$(BIN_NAME)
.PHONY: debug
debug: ## Build debug executable.
@echo "building ${BIN_NAME}-debug ${VERSION}"
@echo "GOPATH=${GOPATH}"
go build -o bin/$(BIN_NAME)-debug $(REPO_PATH)/cmd/$(BIN_NAME)
linux: ## Build holos executable for tilt.
@echo "building ${BIN_NAME}.linux ${VERSION}"
@echo "GOPATH=${GOPATH}"
@@ -154,6 +163,10 @@ website: ## Build website
unity: ## https://cuelabs.dev/unity/
./scripts/unity
.PHONY: update-docs
update-docs: ## Update doc examples
HOLOS_UPDATE_SCRIPTS=1 go test -v ./doc/md/...
.PHONY: help
help: ## Display this help menu.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)

View File

@@ -84,6 +84,9 @@ type Helm struct {
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.

View File

@@ -118,8 +118,12 @@ type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
@@ -130,6 +134,17 @@ type Helm struct {
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// ValueFile represents one Helm value file produced from CUE.
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
@@ -175,11 +190,13 @@ type AuthSource struct {
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
// 3. [Slice] - Slice an artifact into multiple artifacts using [kubectl-slice].
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
// [kubectl-slice]: https://github.com/patrickdappollonio/kubectl-slice
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\" | \"Slice\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
@@ -239,12 +256,46 @@ type Validator struct {
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Command represents a command vetting one or more artifacts. Holos appends
// fully qualified input file paths to the end of the args list, then executes
// the command. Inputs are written into a temporary directory prior to
// executing the command and removed afterwards.
// Command represents a generic command for use as a Generator, Transformer, or
// Validator. Holos uses the Go template engine to render the Args field using
// data provided by the TaskData field. For example to fill in the fully
// qualified temporary directory used to provide inputs to the task.
type Command struct {
// DisplayName represents a friendly display name for the command.
DisplayName string `json:"displayName,omitempty" yaml:"displayName,omitempty"`
// Args represents the complete command argument vector as a go template.
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
// OutputRef references the source of the output data.
OutputRef OutputRef `json:"outputRef,omitempty" yaml:"outputRef,omitempty"`
// TaskData populated by Holos for template rendering.
TaskData TaskData `json:"taskData,omitempty" yaml:"taskData,omitempty"`
// TODO(jjm): add command environment variable support similar to args.
}
// TaskData represents data values associated with a pipeline task necessary to
// execute the task. For example, the randomly generated temporary directory
// used to read and write artifact files when executing user defined task
// commands. Values of this struct are intended for the Go template engine.
//
// Holos populates this struct as needed. Holos may treat user provided values
// as an error condition.
type TaskData struct {
// TempDir represents the temp directory holos manages for task artifacts.
TempDir string `json:"tempDir,omitempty" yaml:"tempDir,omitempty"`
}
// OutputRef represents a reference to the data source used as the output of a
// task. For example, a Generator output may be sourced from standard output or
// a file path.
type OutputRef struct {
// Kind represents the kind of output produced.
Kind string `json:"kind,omitempty" yaml:"kind,omitempty" cue:"string | *\"Pipe\" | \"Path\""`
// Pipe represents stdout or stderr. Ignored unless kind is Pipe.
Pipe string `json:"pipe,omitempty" yaml:"pipe,omitempty" cue:"string | *\"stdout\" | \"stderr\""`
// Path represents an artifact path relative to the task temp directory.
// Ignored unless kind is Path.
Path string `json:"path,omitempty" yaml:"path,omitempty"`
// TODO(jjm): support jsonpath or cel references to the output data maybe?
}
// InternalLabel is an arbitrary unique identifier internal to holos itself.

View File

@@ -11,10 +11,11 @@ import (
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cmd.MakeMain(),
"cue": cue.Main,
}))
holosMain := cmd.MakeMain()
testscript.Main(m, map[string]func(){
"holos": func() { os.Exit(holosMain()) },
"cue": func() { os.Exit(cue.Main()) },
})
}
func TestGuides_v1alpha5(t *testing.T) {

View File

@@ -1,7 +1,9 @@
# https://github.com/holos-run/holos/issues/330
exec holos init platform v1alpha5 --force
# Make sure the helm chart works with plain helm
exec helm template ./components/capabilities/vendor/0.1.0/capabilities
cmp stdout want/helm-template.yaml
stdout 'name: has-foo-v1beta1'
stdout 'kubeVersion: v'
exec holos render platform ./platform
# When no capabilities are specified
cmp deploy/components/capabilities/capabilities.gen.yaml want/when-no-capabilities-specified.yaml

View File

@@ -31,7 +31,6 @@ spec:
- kind: Resources
output: resources.gen.yaml
resources: {}
validators: []
transformers:
- kind: Kustomize
inputs:
@@ -39,7 +38,8 @@ spec:
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
validators: []

View File

@@ -86,6 +86,9 @@ type Helm struct {
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.

View File

@@ -36,13 +36,16 @@ Package core contains schemas for a [Platform](<#Platform>) and [BuildPlan](<#Bu
- [type Kustomization](<#Kustomization>)
- [type Kustomize](<#Kustomize>)
- [type Metadata](<#Metadata>)
- [type OutputRef](<#OutputRef>)
- [type Platform](<#Platform>)
- [type PlatformSpec](<#PlatformSpec>)
- [type Repository](<#Repository>)
- [type Resource](<#Resource>)
- [type Resources](<#Resources>)
- [type TaskData](<#TaskData>)
- [type Transformer](<#Transformer>)
- [type Validator](<#Validator>)
- [type ValueFile](<#ValueFile>)
- [type Values](<#Values>)
@@ -150,11 +153,18 @@ type Chart struct {
<a name="Command"></a>
## type Command {#Command}
Command represents a command vetting one or more artifacts. Holos appends fully qualified input file paths to the end of the args list, then executes the command. Inputs are written into a temporary directory prior to executing the command and removed afterwards.
Command represents a generic command for use as a Generator, Transformer, or Validator. Holos uses the Go template engine to render the Args field using data provided by the TaskData field. For example to fill in the fully qualified temporary directory used to provide inputs to the task.
```go
type Command struct {
// DisplayName represents a friendly display name for the command.
DisplayName string `json:"displayName,omitempty" yaml:"displayName,omitempty"`
// Args represents the complete command argument vector as a go template.
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
// OutputRef references the source of the output data.
OutputRef OutputRef `json:"outputRef,omitempty" yaml:"outputRef,omitempty"`
// TaskData populated by Holos for template rendering.
TaskData TaskData `json:"taskData,omitempty" yaml:"taskData,omitempty"`
}
```
@@ -283,8 +293,12 @@ type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
@@ -382,6 +396,23 @@ type Metadata struct {
}
```
<a name="OutputRef"></a>
## type OutputRef {#OutputRef}
OutputRef represents a reference to the data source used as the output of a task. For example, a Generator output may be sourced from standard output or a file path.
```go
type OutputRef struct {
// Kind represents the kind of output produced.
Kind string `json:"kind,omitempty" yaml:"kind,omitempty" cue:"string | *\"Pipe\" | \"Path\""`
// Pipe represents stdout or stderr. Ignored unless kind is Pipe.
Pipe string `json:"pipe,omitempty" yaml:"pipe,omitempty" cue:"string | *\"stdout\" | \"stderr\""`
// Path represents an artifact path relative to the task temp directory.
// Ignored unless kind is Path.
Path string `json:"path,omitempty" yaml:"path,omitempty"`
}
```
<a name="Platform"></a>
## type Platform {#Platform}
@@ -452,6 +483,20 @@ Resources represents Kubernetes resources. Most commonly used to mix resources i
type Resources map[Kind]map[InternalLabel]Resource
```
<a name="TaskData"></a>
## type TaskData {#TaskData}
TaskData represents data values associated with a pipeline task necessary to execute the task. For example, the randomly generated temporary directory used to read and write artifact files when executing user defined task commands. Values of this struct are intended for the Go template engine.
Holos populates this struct as needed. Holos may treat user provided values as an error condition.
```go
type TaskData struct {
// TempDir represents the temp directory holos manages for task artifacts.
TempDir string `json:"tempDir,omitempty" yaml:"tempDir,omitempty"`
}
```
<a name="Transformer"></a>
## type Transformer {#Transformer}
@@ -459,11 +504,12 @@ Transformer combines multiple inputs from prior [Generator](<#Generator>) or [Tr
1. [Kustomize](<#Kustomize>) \- Patch and transform the output from prior generators or transformers. See [Introduction to Kustomize](<https://kubectl.docs.kubernetes.io/guides/config_management/introduction/>).
2. [Join](<#Join>) \- Concatenate multiple prior outputs into one output.
3. \[Slice\] \- Slice an artifact into multiple artifacts using [kubectl\\\-slice](<https://github.com/patrickdappollonio/kubectl-slice>).
```go
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\" | \"Slice\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
@@ -493,6 +539,23 @@ type Validator struct {
}
```
<a name="ValueFile"></a>
## type ValueFile {#ValueFile}
ValueFile represents one Helm value file produced from CUE.
```go
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
```
<a name="Values"></a>
## type Values {#Values}

View File

@@ -0,0 +1,7 @@
exec bash -c 'bash -euo pipefail $WORK/command.sh 2>&1'
cmp stdout $WORK/output.txt
-- command.sh --
holos --version
-- output.txt --
0.104.1

View File

@@ -0,0 +1,126 @@
# Set $HOME because:
# - Helm uses it for temporary files
# - Git requires it for setting author name/email globally
env HOME=$WORK/.tmp
chmod 0755 $WORK/update.sh
# Configure git author for testscript execution
exec git config --global user.name 'Holos Docs'
exec git config --global user.email 'hello@holos.run'
exec git config --global init.defaultBranch main
# Remove the tutorial directory if it already exists
exec rm -rf holos-tutorial
# Create and change to the tutorial directory, and then initialize the Holos platform
exec bash -c 'bash -euo pipefail $WORK/mkdir-and-init.sh'
cd holos-tutorial
# Create the components directory
exec bash -c 'bash -euo pipefail $WORK/mkdir-components.sh'
# Combine and execute the multiline podinfo component header/body/trailer files
exec cat $WORK/podinfo-component-header.sh ../podinfo-component-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Combine and execute the multiline platform registration header/body/trailer files
exec cat $WORK/register-podinfo-header.sh ../register-podinfo-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Render the platform, capture stdout, and use update.sh to gate whether the
# output file should be updated.
#
# NOTE: The [net] condition will test whether external network access is available
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/register-components-output.txt
# Generate and update the tree of the tutorial directory (omitting the cue.mod directory)
exec bash -c 'bash -euo pipefail $WORK/tree.sh'
stdin stdout
exec $WORK/update.sh $WORK/tree.txt
# Split the rendered manifest into two separate files to display separately
exec bash -c 'bash -euo pipefail $WORK/split-rendered-manifest.sh $WORK/holos-tutorial/deploy/components/podinfo/podinfo.gen.yaml $WORK'
# Grep for the Hello Holos message and write the output file
exec bash -c 'bash -euo pipefail $WORK/grep-for-message.sh'
stdin stdout
exec $WORK/update.sh $WORK/grepped-output.txt
# Clean up the tutorial directory and tmp $HOME directory
cd $WORK
exec rm -rf holos-tutorial
exec rm -rf $HOME
-- update.sh --
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"
-- mkdir-and-init.sh --
mkdir holos-tutorial && cd holos-tutorial
holos init platform v1alpha5
-- tree.sh --
tree -L 3 -I cue.mod .
-- mkdir-components.sh --
mkdir -p components/podinfo
-- podinfo-component-header.sh --
cat <<EOF > components/podinfo/podinfo.cue
-- podinfo-component-body.cue --
package holos
// Produce a helm chart build plan.
holos: HelmChart.BuildPlan
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
// Holos marshals Values into values.yaml for Helm.
Values: {
// message is a string with a default value. @tag indicates a value may
// be injected from the platform spec component parameters.
ui: {
message: string | *"Hello World" @tag(greeting, type=string)
}
}
}
-- eof-trailer.sh --
EOF
-- register-podinfo-header.sh --
cat <<EOF > platform/podinfo.cue
-- register-podinfo-body.cue --
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
// Inject a value into the component.
parameters: greeting: "Hello Holos!"
}
-- render.sh --
holos render platform
-- register-components-output.txt --
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
-- podinfo-rendered-path.sh --
deploy/components/podinfo/podinfo.gen.yaml
-- split-rendered-manifest.sh --
awk 'BEGIN {RS="---"} NR==1 {print > "service.yaml"} NR==2 {print > "deployment.yaml"}' $1
mv service.yaml $2/rendered-service.yaml
mv deployment.yaml $2/rendered-deployment.yaml
-- grep-for-message.sh --
grep -B2 Hello deploy/components/podinfo/podinfo.gen.yaml
-- grepped-output.txt --
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!

View File

@@ -0,0 +1 @@
holos --version

View File

@@ -0,0 +1 @@
0.104.1

View File

@@ -0,0 +1 @@
EOF

View File

@@ -0,0 +1 @@
grep -B2 Hello deploy/components/podinfo/podinfo.gen.yaml

View File

@@ -0,0 +1,3 @@
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!

View File

@@ -0,0 +1,2 @@
mkdir holos-tutorial && cd holos-tutorial
holos init platform v1alpha5

View File

@@ -0,0 +1 @@
mkdir -p components/podinfo

View File

@@ -0,0 +1,23 @@
package holos
// Produce a helm chart build plan.
holos: HelmChart.BuildPlan
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
// Holos marshals Values into values.yaml for Helm.
Values: {
// message is a string with a default value. @tag indicates a value may
// be injected from the platform spec component parameters.
ui: {
message: string | *"Hello World" @tag(greeting, type=string)
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/podinfo/podinfo.cue

View File

@@ -0,0 +1 @@
deploy/components/podinfo/podinfo.gen.yaml

View File

@@ -0,0 +1,2 @@
rendered podinfo in 312.472625ms
rendered platform in 312.557375ms

View File

@@ -0,0 +1,8 @@
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
// Inject a value into the component.
parameters: greeting: "Hello Holos!"
}

View File

@@ -0,0 +1 @@
cat <<EOF > platform/podinfo.cue

View File

@@ -0,0 +1 @@
holos render platform

View File

@@ -0,0 +1,93 @@
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
helm.sh/chart: podinfo-6.6.2
name: podinfo
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: podinfo
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
prometheus.io/port: "9898"
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: podinfo
spec:
containers:
- command:
- ./podinfo
- --port=9898
- --cert-path=/data/cert
- --port-metrics=9797
- --grpc-port=9999
- --grpc-service-name=podinfo
- --level=info
- --random-delay=false
- --random-error=false
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!
- name: PODINFO_UI_COLOR
value: '#34577c'
image: ghcr.io/stefanprodan/podinfo:6.6.2
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/healthz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: podinfo
ports:
- containerPort: 9898
name: http
protocol: TCP
- containerPort: 9797
name: http-metrics
protocol: TCP
- containerPort: 9999
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/readyz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits: null
requests:
cpu: 1m
memory: 16Mi
volumeMounts:
- mountPath: /data
name: data
terminationGracePeriodSeconds: 30
volumes:
- emptyDir: {}
name: data

View File

@@ -0,0 +1,23 @@
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
helm.sh/chart: podinfo-6.6.2
name: podinfo
spec:
ports:
- name: http
port: 9898
protocol: TCP
targetPort: http
- name: grpc
port: 9999
protocol: TCP
targetPort: grpc
selector:
app.kubernetes.io/name: podinfo
type: ClusterIP

View File

@@ -0,0 +1,3 @@
awk 'BEGIN {RS="---"} NR==1 {print > "service.yaml"} NR==2 {print > "deployment.yaml"}' $1
mv service.yaml $2/rendered-service.yaml
mv deployment.yaml $2/rendered-deployment.yaml

View File

@@ -0,0 +1 @@
tree -L 3 -I cue.mod .

View File

@@ -0,0 +1,17 @@
.
|-- components
| `-- podinfo
| |-- podinfo.cue
| `-- vendor
|-- deploy
| `-- components
| `-- podinfo
|-- platform
| |-- platform.gen.cue
| `-- podinfo.cue
|-- platform.metadata.json
|-- resources.cue
|-- schema.cue
`-- tags.cue
8 directories, 7 files

View File

@@ -0,0 +1,4 @@
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"

View File

@@ -0,0 +1,7 @@
exec bash -c 'bash -euo pipefail $WORK/command.sh 2>&1'
cmp stdout $WORK/output.txt
-- command.sh --
holos --version
-- output.txt --
0.104.1

View File

@@ -0,0 +1,374 @@
# Set $HOME because:
# - Helm uses it for temporary files
# - Git requires it for setting author name/email globally
env HOME=$WORK/.tmp
chmod 0755 $WORK/update.sh
# Configure git author for testscript execution
exec git config --global user.name 'Holos Docs'
exec git config --global user.email 'hello@holos.run'
exec git config --global init.defaultBranch main
# Remove the tutorial directory if it already exists
exec rm -rf holos-helm-values-tutorial
# Create and change to the tutorial directory, and then initialize the Holos platform
exec bash -c 'bash -euo pipefail mkdir-and-init.sh'
cd holos-helm-values-tutorial
# Git init and create the component directories
exec bash -c 'bash -euo pipefail $WORK/git-init.sh'
exec bash -c 'bash -euo pipefail $WORK/mkdir-components.sh'
# Combine and execute the multiline prometheus/blackbox component header/body/trailer files
exec cat $WORK/prometheus-component-header.sh ../prometheus-component-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
exec cat $WORK/blackbox-component-header.sh ../blackbox-component-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Combine and execute the multiline platform registration header/body/trailer files.
exec cat $WORK/register-components-header.sh ../register-components-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Render the platform, capture stdout, and use update.sh to gate whether the
# output file should be updated.
#
# NOTE: The [net] condition will test whether external network access is available
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/register-components-output.txt
# Commit and conditionally update the output file
exec bash -c 'bash -euo pipefail $WORK/register-components-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/register-components-git-commit-output.txt
# Import values
exec bash -c 'bash -euo pipefail $WORK/import-prometheus-values.sh'
exec bash -c 'bash -euo pipefail $WORK/import-blackbox-values.sh'
# Render, update the output file, commit, and update the commit output file.
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/import-values-render-output.txt
exec bash -c 'bash -euo pipefail $WORK/import-values-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/import-values-git-output.txt
# Create the common configuration path
exec bash -c 'bash -euo pipefail $WORK/mkdir-common-config.sh'
# Combine and execute the common configuration header/body/trailer to write the cue file.
exec cat $WORK/blackbox-common-config-header.sh ../blackbox-common-config-body.cue ../eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Git commit blackbox common config
exec bash -c 'bash -euo pipefail $WORK/blackbox-common-config-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/blackbox-common-config-git-output.txt
# Patch the common config values file and write to output file.
#
# NOTE: Using a symlink here because the patch script references values.patch
# within the same directory, but it actually lives one directory up in the
# testscript $WORK dir.
exec ln -s $WORK/values.patch values.patch
exec bash -c 'bash -euo pipefail $WORK/common-config-patch.sh'
stdin stdout
exec $WORK/update.sh $WORK/common-config-patch.txt
# Remove patch and commit changes
exec bash -c 'bash -euo pipefail $WORK/common-config-rm.sh'
exec bash -c 'bash -euo pipefail $WORK/common-config-git.sh'
stdin stdout
exec $WORK/update.sh $WORK/common-config-git-output.txt
# Final render and update of output file.
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/reviewing-changes-git-output.txt
# Git diff and write to output file.
exec bash -c 'bash -euo pipefail $WORK/git-diff.sh'
stdin stdout
exec $WORK/update.sh $WORK/git.diff
# Final commit and write to output file
exec bash -c 'bash -euo pipefail $WORK/reviewing-changes-git-commit.sh'
stdin stdout
exec $WORK/update.sh $WORK/reviewing-changes-git-output.txt
# Clean up the tutorial directory and tmp $HOME directory
cd $WORK
exec rm -rf holos-helm-values-tutorial
exec rm -rf $HOME
-- update.sh --
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"
-- mkdir-and-init.sh --
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5
-- git-init.sh --
git init . && git add . && git commit -m "initial commit"
-- mkdir-components.sh --
mkdir -p components/prometheus components/blackbox
-- prometheus-component-header.sh --
cat <<EOF > components/prometheus/prometheus.cue
-- prometheus-component-body.cue --
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
-- eof-trailer.sh --
EOF
-- blackbox-component-header.sh --
cat <<EOF > components/blackbox/blackbox.cue
-- blackbox-component-body.cue --
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
-- register-components-header.sh --
cat <<EOF > platform/prometheus.cue
-- register-components-body.cue --
package holos
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}
-- render.sh --
holos render platform
-- register-components-output.txt --
cached prometheus-blackbox-exporter 9.0.1
rendered blackbox in 3.825430417s
cached prometheus 25.27.0
rendered prometheus in 4.840089667s
rendered platform in 4.840137792s
-- register-components-git-commit.sh --
git add . && git commit -m 'add blackbox and prometheus'
-- register-components-git-commit-output.txt --
[main b5df111] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue
-- import-prometheus-values.sh --
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml
-- import-blackbox-values.sh --
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
-- import-values-render-output.txt --
rendered blackbox in 365.936792ms
rendered prometheus in 371.855875ms
rendered platform in 372.109916ms
-- import-values-git-commit.sh --
git add . && git commit -m 'import values'
-- import-values-git-output.txt --
[main 52e90ea] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue
-- mkdir-common-config.sh --
mkdir -p config/prometheus
-- blackbox-common-config-header.sh --
cat <<EOF > config/prometheus/blackbox.cue
-- blackbox-common-config-body.cue --
package prometheus
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}
-- blackbox-common-config-git-commit.sh --
git add . && git commit -m 'add blackbox configuration'
-- blackbox-common-config-git-output.txt --
[main 1adcd08] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 components/blackbox.cue
-- common-config-patch.sh --
patch -p1 < values.patch
-- values.patch --
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,11 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
+ fullnameOverride: prometheus.blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +197,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: prometheus.blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1,5 +1,8 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
// yaml-language-server: $schema=values.schema.json
// Default values for prometheus.
@@ -1083,7 +1086,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(prometheus.blackbox.host):\(prometheus.blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
-- common-config-patch.txt --
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'
-- common-config-rm.sh --
rm values.patch
-- common-config-git.sh --
git add . && git commit -m 'integrate blackbox and prometheus together'
-- common-config-git-output.txt --
[main 4221803] integrate blackbox and prometheus together
2 files changed, 4 insertions(+), 2 deletions(-)
-- reviewing-changes-render-output.txt --
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms
-- git-diff.sh --
git diff
-- git.diff --
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target
-- reviewing-changes-git-commit.sh --
git add . && git commit -m 'render integrated blackbox and prometheus manifests'
-- reviewing-changes-git-output.txt --
[main 67efe0d] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)

View File

@@ -0,0 +1 @@
holos --version

View File

@@ -0,0 +1 @@
0.104.1

View File

@@ -0,0 +1,15 @@
package prometheus
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}

View File

@@ -0,0 +1 @@
git add . && git commit -m 'add blackbox configuration'

View File

@@ -0,0 +1,3 @@
[main aa9749e] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 config/prometheus/blackbox.cue

View File

@@ -0,0 +1 @@
cat <<EOF > config/prometheus/blackbox.cue

View File

@@ -0,0 +1,15 @@
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/blackbox/blackbox.cue

View File

@@ -0,0 +1,3 @@
[main 656d4ee] integrate blackbox and prometheus together
3 files changed, 1348 insertions(+), 2 deletions(-)
create mode 100644 components/prometheus/values.cue.orig

View File

@@ -0,0 +1 @@
git add . && git commit -m 'integrate blackbox and prometheus together'

View File

@@ -0,0 +1 @@
patch -p1 < values.patch

View File

@@ -0,0 +1,2 @@
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'

View File

@@ -0,0 +1 @@
rm values.patch

View File

@@ -0,0 +1 @@
EOF

View File

@@ -0,0 +1 @@
git diff

View File

@@ -0,0 +1 @@
git init . && git add . && git commit -m "initial commit"

View File

@@ -0,0 +1,64 @@
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target

View File

@@ -0,0 +1,5 @@
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml

View File

@@ -0,0 +1,5 @@
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml

View File

@@ -0,0 +1 @@
git add . && git commit -m 'import values'

View File

@@ -0,0 +1,4 @@
[main 3788921] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue

View File

@@ -0,0 +1,3 @@
rendered blackbox in 129.142708ms
rendered prometheus in 165.260375ms
rendered platform in 165.33ms

View File

@@ -0,0 +1,3 @@
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5

View File

@@ -0,0 +1 @@
mkdir -p config/prometheus

View File

@@ -0,0 +1 @@
mkdir -p components/prometheus components/blackbox

View File

@@ -0,0 +1,15 @@
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/prometheus/prometheus.cue

View File

@@ -0,0 +1,12 @@
package holos
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}

View File

@@ -0,0 +1,7 @@
[main 654ae06] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue

View File

@@ -0,0 +1 @@
git add . && git commit -m 'add blackbox and prometheus'

View File

@@ -0,0 +1 @@
cat <<EOF > platform/prometheus.cue

View File

@@ -0,0 +1,3 @@
rendered blackbox in 1.096663084s
rendered prometheus in 1.151784875s
rendered platform in 1.151839916s

View File

@@ -0,0 +1 @@
holos render platform

View File

@@ -0,0 +1 @@
git add . && git commit -m 'render integrated blackbox and prometheus manifests'

View File

@@ -0,0 +1,2 @@
[main 9bc0126] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)

View File

@@ -0,0 +1,3 @@
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms

View File

@@ -0,0 +1,4 @@
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"

View File

@@ -0,0 +1,43 @@
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,11 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
+ fullnameOverride: prometheus.blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +197,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: prometheus.blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1,5 +1,8 @@
package holos
+// Import common blackbox configuration
+import "holos.example/config/prometheus"
+
Helm: Values: {
// yaml-language-server: $schema=values.schema.json
// Default values for prometheus.
@@ -1083,7 +1086,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(prometheus.blackbox.host):\(prometheus.blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"

View File

@@ -0,0 +1,7 @@
exec bash -c 'bash -euo pipefail $WORK/command.sh 2>&1'
cmp stdout $WORK/output.txt
-- command.sh --
holos --version
-- output.txt --
0.104.1

View File

@@ -0,0 +1,226 @@
# Set $HOME because:
# - Helm uses it for temporary files
# - Git requires it for setting author name/email globally
env HOME=$WORK/.tmp
chmod 0755 $WORK/update.sh
# Configure git author for testscript execution
exec git config --global user.name 'Holos Docs'
exec git config --global user.email 'hello@holos.run'
exec git config --global init.defaultBranch main
# Remove the tutorial directory if it already exists
exec rm -rf holos-kustomize-tutorial
# Create and change to the tutorial directory, and then initialize the Holos platform
exec bash -c 'bash -euo pipefail $WORK/mkdir-and-init.sh'
cd holos-kustomize-tutorial
# Initialize git
exec bash -c 'bash -euo pipefail $WORK/git-init.sh'
# Create the component directory
exec bash -c 'bash -euo pipefail $WORK/mkdir-component.sh'
# Combine and execute the multiline httpbin component header/body/trailer files
exec cat $WORK/httpbin-component-header.sh $WORK/httpbin-component-body.cue $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Combine and execute the multiline httpbin yaml header/body/trailer files
exec cat $WORK/httpbin-yaml-header.sh $WORK/httpbin-yaml-body.yaml $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Combine and execute the multiline registration header/body/trailer files
exec cat $WORK/register-component-header.sh $WORK/register-component-body.cue $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Render the platform, capture stdout, and use update.sh to gate whether the
# output file should be updated.
#
# NOTE: The [net] condition will test whether external network access is available
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/register-component-output.txt
# Git commit and capture output
exec bash -c 'bash -euo pipefail $WORK/git-commit-component.sh'
stdin stdout
exec $WORK/update.sh $WORK/git-commit-component-output.txt
# Export Build Plan and capture output
exec bash -c 'bash -euo pipefail $WORK/cue-export.sh'
stdin stdout
exec $WORK/update.sh $WORK/buildplan-output.cue
# Combine and execute the multiline kustomize patch header/body/trailer files
exec cat $WORK/httpbin-patch-header.sh $WORK/httpbin-patch-body.cue $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Render the platform and capture output
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/kustomize-patch-render-output.txt
# Git diff and capture output
exec bash -c 'bash -euo pipefail $WORK/git-diff.sh'
stdin stdout
exec $WORK/update.sh $WORK/git.diff
# Git commit and capture output
exec bash -c 'bash -euo pipefail $WORK/git-commit-final.sh'
stdin stdout
exec $WORK/update.sh $WORK/git-commit-final-output.txt
# Clean up the tutorial directory and tmp $HOME directory
cd $WORK
exec rm -rf holos-kustomize-tutorial
exec rm -rf $HOME
-- update.sh --
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"
-- mkdir-and-init.sh --
mkdir holos-kustomize-tutorial
cd holos-kustomize-tutorial
holos init platform v1alpha5
-- git-init.sh --
git init . && git add . && git commit -m initial
-- mkdir-component.sh --
mkdir -p components/httpbin
-- httpbin-component-header.sh --
cat <<EOF > components/httpbin/httpbin.cue
-- httpbin-component-body.cue --
package holos
// Produce a Kustomize BuildPlan for Holos
holos: Kustomize.BuildPlan
// https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/README.md
Kustomize: #Kustomize & {
KustomizeConfig: {
// Files tells Holos to copy the file from the component path to the
// temporary directory Holos uses for BuildPlan execution.
Files: {
"httpbin.yaml": _
}
CommonLabels: {
"app.kubernetes.io/name": "httpbin"
}
// Kustomization represents a kustomization.yaml file in CUE. Holos
// marshals this field into a `kustomization.yaml` while processing a
// BuildPlan. See
// https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
Kustomization: {
images: [{name: "mccutchen/go-httpbin"}]
// Use a hidden field to compose patches easily with a struct. Hidden
// fields are not included in exported structures.
_patches: {}
// Convert the hidden struct to a list.
patches: [for x in _patches {x}]
}
}
}
-- eof-trailer.sh --
EOF
-- httpbin-yaml-header.sh --
cat <<EOF > components/httpbin/httpbin.yaml
-- httpbin-yaml-body.yaml --
# https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/resources.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
template:
spec:
containers:
- name: httpbin
image: mccutchen/go-httpbin
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /status/200
port: http
readinessProbe:
httpGet:
path: /status/200
port: http
resources: {}
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
appProtocol: http
-- register-component-header.sh --
cat <<EOF > platform/httpbin.cue
-- register-component-body.cue --
package holos
Platform: Components: {
httpbin: {
name: "httpbin"
path: "components/httpbin"
}
}
-- git-commit-component.sh --
git add . && git commit -m 'add httpbin'
-- cue-export.sh --
holos cue export --expression holos --out=yaml ./components/httpbin
-- httpbin-patch-header.sh --
cat <<EOF > components/httpbin/patches.cue
-- httpbin-patch-body.cue --
package holos
import "encoding/yaml"
// Mix in a Kustomize patch to the configuration.
Kustomize: KustomizeConfig: Kustomization: _patches: {
probe: {
target: kind: "Service"
target: name: "httpbin"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/prometheus.io~1probe"
value: "true"
}])
}
}
-- httpbin-component-output.txt --
rendered httpbin in 197.030208ms
rendered platform in 197.416416ms
-- render.sh --
holos render platform
-- git-diff.sh --
git diff
-- git.diff --
diff --git a/deploy/components/httpbin/httpbin.gen.yaml b/deploy/components/httpbin/httpbin.gen.yaml
index 298b9a8..a16bd1a 100644
--- a/deploy/components/httpbin/httpbin.gen.yaml
+++ b/deploy/components/httpbin/httpbin.gen.yaml
@@ -1,6 +1,8 @@
apiVersion: v1
kind: Service
metadata:
+ annotations:
+ prometheus.io/probe: "true"
labels:
app.kubernetes.io/name: httpbin
name: httpbin
-- git-commit-final.sh --
git add . && git commit -m 'annotate httpbin for prometheus probes'

View File

@@ -0,0 +1 @@
holos --version

View File

@@ -0,0 +1 @@
0.104.1

View File

@@ -0,0 +1,36 @@
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
- kind: File
output: httpbin.yaml
file:
source: httpbin.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
- httpbin.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/name: httpbin
patches: []
images:
- name: mccutchen/go-httpbin
resources:
- resources.gen.yaml
- httpbin.yaml
validators: []

View File

@@ -0,0 +1 @@
holos cue export --expression holos --out=yaml ./components/httpbin

View File

@@ -0,0 +1 @@
EOF

View File

@@ -0,0 +1,6 @@
[main 823e136] add httpbin
4 files changed, 113 insertions(+)
create mode 100644 components/httpbin/httpbin.cue
create mode 100644 components/httpbin/httpbin.yaml
create mode 100644 deploy/components/httpbin/httpbin.gen.yaml
create mode 100644 platform/httpbin.cue

View File

@@ -0,0 +1 @@
git add . && git commit -m 'add httpbin'

View File

@@ -0,0 +1,3 @@
[main 96f5f45] annotate httpbin for prometheus probes
2 files changed, 18 insertions(+)
create mode 100644 components/httpbin/patches.cue

View File

@@ -0,0 +1 @@
git add . && git commit -m 'annotate httpbin for prometheus probes'

View File

@@ -0,0 +1 @@
git diff

View File

@@ -0,0 +1 @@
git init . && git add . && git commit -m initial

View File

@@ -0,0 +1,13 @@
diff --git a/deploy/components/httpbin/httpbin.gen.yaml b/deploy/components/httpbin/httpbin.gen.yaml
index 298b9a8..a16bd1a 100644
--- a/deploy/components/httpbin/httpbin.gen.yaml
+++ b/deploy/components/httpbin/httpbin.gen.yaml
@@ -1,6 +1,8 @@
apiVersion: v1
kind: Service
metadata:
+ annotations:
+ prometheus.io/probe: "true"
labels:
app.kubernetes.io/name: httpbin
name: httpbin

View File

@@ -0,0 +1,30 @@
package holos
// Produce a Kustomize BuildPlan for Holos
holos: Kustomize.BuildPlan
// https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/README.md
Kustomize: #Kustomize & {
KustomizeConfig: {
// Files tells Holos to copy the file from the component path to the
// temporary directory Holos uses for BuildPlan execution.
Files: {
"httpbin.yaml": _
}
CommonLabels: {
"app.kubernetes.io/name": "httpbin"
}
// Kustomization represents a kustomization.yaml file in CUE. Holos
// marshals this field into a `kustomization.yaml` while processing a
// BuildPlan. See
// https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
Kustomization: {
images: [{name: "mccutchen/go-httpbin"}]
// Use a hidden field to compose patches easily with a struct. Hidden
// fields are not included in exported structures.
_patches: {}
// Convert the hidden struct to a list.
patches: [for x in _patches {x}]
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/httpbin/httpbin.cue

View File

@@ -0,0 +1,2 @@
rendered httpbin in 197.030208ms
rendered platform in 197.416416ms

View File

@@ -0,0 +1,16 @@
package holos
import "encoding/yaml"
// Mix in a Kustomize patch to the configuration.
Kustomize: KustomizeConfig: Kustomization: _patches: {
probe: {
target: kind: "Service"
target: name: "httpbin"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/prometheus.io~1probe"
value: "true"
}])
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/httpbin/patches.cue

View File

@@ -0,0 +1,36 @@
# https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/resources.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
template:
spec:
containers:
- name: httpbin
image: mccutchen/go-httpbin
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /status/200
port: http
readinessProbe:
httpGet:
path: /status/200
port: http
resources: {}
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
appProtocol: http

View File

@@ -0,0 +1 @@
cat <<EOF > components/httpbin/httpbin.yaml

View File

@@ -0,0 +1,2 @@
rendered httpbin in 112.916375ms
rendered platform in 112.990333ms

View File

@@ -0,0 +1,3 @@
mkdir holos-kustomize-tutorial
cd holos-kustomize-tutorial
holos init platform v1alpha5

View File

@@ -0,0 +1 @@
mkdir -p components/httpbin

View File

@@ -0,0 +1,8 @@
package holos
Platform: Components: {
httpbin: {
name: "httpbin"
path: "components/httpbin"
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > platform/httpbin.cue

View File

@@ -0,0 +1,2 @@
rendered httpbin in 111.183042ms
rendered platform in 111.265792ms

View File

@@ -0,0 +1 @@
holos render platform

Some files were not shown because too many files have changed in this diff Show More