Compare commits

...

1 Commits

Author SHA1 Message Date
Gary Larizza
ae4aad4993 Update Cue tutorial to use testscript
PROBLEM:

The "Cue" tutorial has hardcoded code blocks and hasn't been
updated to use the automated testscript workflow.

NOTE: This is slightly more complex than normal because we need
to make sure Timoni is installed when we execute the testscripts
due to the fact that we need to execute `timoni mod vendor crds ...`
and capture the output.

SOLUTION:

* Add Timoni as one of the packages that are installed via `make go-deps`.
* Update the testing GH Action to install all go dependencies before executing the tests.
* Create a test for the Cue tutorial.
* Create a testscript for the Cue test.
* Update the Cue MDX file to load in data from the testscript directory.

OUTCOME:

The code content in the Cue tutorial now comes directly from the
testscript workflow.
2025-01-23 15:49:04 -08:00
26 changed files with 359 additions and 143 deletions

View File

@@ -124,6 +124,7 @@ go-deps: ## tool versions pinned in tools.go
go install golang.org/x/tools/cmd/godoc
go install github.com/princjef/gomarkdoc/cmd/gomarkdoc
go install github.com/google/ko
go install github.com/stefanprodan/timoni/cmd/timoni@v0.23.0
# curl https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | bash
.PHONY: frontend-deps

View File

@@ -0,0 +1,7 @@
exec bash -c 'bash -euo pipefail $WORK/command.sh 2>&1'
cmp stdout $WORK/output.txt
-- command.sh --
holos --version
-- output.txt --
0.103.0

View File

@@ -0,0 +1,166 @@
# Set $HOME because:
# - Helm uses it for temporary files
# - Git requires it for setting author name/email globally
env HOME=$WORK/.tmp
chmod 0755 $WORK/update.sh
# Configure git author for testscript execution
exec git config --global user.name 'Holos Docs'
exec git config --global user.email 'hello@holos.run'
exec git config --global init.defaultBranch main
# Remove the tutorial directory if it already exists
exec rm -rf holos-cue-tutorial
# Create and change to the tutorial directory, and then initialize the Holos platform
exec bash -c 'bash -euo pipefail mkdir-and-init.sh'
cd holos-cue-tutorial
# Create the components directory, then combine and execute the multiline
# podinfo component header/body/trailer files
exec bash -c 'bash -euo pipefail $WORK/mkdir-components.sh'
exec cat $WORK/podinfo-component-header.sh $WORK/podinfo-component-body.cue $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Combine and execute the multiline platform registration header/body/trailer files.
exec cat $WORK/register-components-header.sh $WORK/register-components-body.cue $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Render and capture output
# NOTE: The [net] condition will test whether external network access is available
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
[net] stdin stdout
exec $WORK/update.sh $WORK/register-components-output.txt
# Git init and commit
exec bash -c 'bash -euo pipefail $WORK/git-init.sh'
# Combine and execute the mixin component header/body/trailer files
exec cat $WORK/mixin-component-header.sh $WORK/mixin-component-body.cue $WORK/eof-trailer.sh
stdin stdout
exec bash -xeuo pipefail
# Import CRDs with Timoni
exec bash -c 'bash -euo pipefail $WORK/import-crds.sh 2>&1'
stdin stdout
exec $WORK/update.sh $WORK/timoni-vendor.txt
# Render platform
[net] exec bash -c 'bash -euo pipefail $WORK/render.sh 2>&1'
# Git diff and capture output
exec bash -c 'bash -euo pipefail $WORK/git-diff.sh 2>&1'
stdin stdout
exec $WORK/update.sh $WORK/git.diff
# Clean up the tutorial directory and tmp $HOME directory
cd $WORK
exec rm -rf holos-cue-tutorial
exec rm -rf $HOME
-- update.sh --
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"
-- mkdir-and-init.sh --
mkdir holos-cue-tutorial && cd holos-cue-tutorial
holos init platform v1alpha5
-- mkdir-components.sh --
mkdir -p components/podinfo
-- import-crds.sh --
timoni mod vendor crds -f https://raw.githubusercontent.com/external-secrets/external-secrets/v0.10.5/deploy/crds/bundle.yaml
-- podinfo-component-header.sh --
cat <<EOF > components/podinfo/podinfo.cue
-- podinfo-component-body.cue --
package holos
// export the component build plan to holos
holos: Component.BuildPlan
// Component is a Helm chart
Component: #Helm & {
Name: "podinfo"
Namespace: "default"
// Add metadata.namespace to all resources with kustomize.
KustomizeConfig: Kustomization: namespace: Namespace
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
-- eof-trailer.sh --
EOF
-- register-components-header.sh --
cat <<EOF > platform/podinfo.cue
-- register-components-body.cue --
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
-- render.sh --
holos render platform
-- git-init.sh --
git init . && git add . && git commit -m initial
-- mixin-component-header.sh --
cat <<EOF > components/podinfo/mixins.cue
-- mixin-component-body.cue --
package holos
// Component fields are unified with podinfo.cue
Component: {
// Concrete values are defined in podinfo.cue
Name: string
Namespace: string
// Resources represents mix-in resources organized as a struct.
Resources: ExternalSecret: (Name): {
// Name is consistent with the component name.
metadata: name: Name
// Namespace is consistent with the component namespace.
metadata: namespace: Namespace
spec: {
// Ensure the target secret name is consistent.
target: name: metadata.name
// Ensure the name in the SecretStore is consistent.
dataFrom: [{extract: {key: metadata.name}}]
refreshInterval: "30s"
secretStoreRef: kind: "SecretStore"
secretStoreRef: name: "default"
}
}
}
-- git-diff.sh --
git diff deploy
-- git.diff --
diff --git a/deploy/components/podinfo/podinfo.gen.yaml b/deploy/components/podinfo/podinfo.gen.yaml
index 6e4aec0..f79e9d0 100644
--- a/deploy/components/podinfo/podinfo.gen.yaml
+++ b/deploy/components/podinfo/podinfo.gen.yaml
@@ -112,3 +112,19 @@ spec:
volumes:
- emptyDir: {}
name: data
+---
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+ name: podinfo
+ namespace: default
+spec:
+ dataFrom:
+ - extract:
+ key: podinfo
+ refreshInterval: 30s
+ secretStoreRef:
+ kind: SecretStore
+ name: default
+ target:
+ name: podinfo

View File

@@ -0,0 +1 @@
holos --version

View File

@@ -0,0 +1 @@
0.103.0

View File

@@ -0,0 +1 @@
EOF

View File

@@ -0,0 +1 @@
git diff deploy

View File

@@ -0,0 +1 @@
git init . && git add . && git commit -m initial

View File

@@ -0,0 +1,24 @@
diff --git a/deploy/components/podinfo/podinfo.gen.yaml b/deploy/components/podinfo/podinfo.gen.yaml
index 6e4aec0..f79e9d0 100644
--- a/deploy/components/podinfo/podinfo.gen.yaml
+++ b/deploy/components/podinfo/podinfo.gen.yaml
@@ -112,3 +112,19 @@ spec:
volumes:
- emptyDir: {}
name: data
+---
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+ name: podinfo
+ namespace: default
+spec:
+ dataFrom:
+ - extract:
+ key: podinfo
+ refreshInterval: 30s
+ secretStoreRef:
+ kind: SecretStore
+ name: default
+ target:
+ name: podinfo

View File

@@ -0,0 +1 @@
timoni mod vendor crds -f https://raw.githubusercontent.com/external-secrets/external-secrets/v0.10.5/deploy/crds/bundle.yaml

View File

@@ -0,0 +1,25 @@
package holos
// Component fields are unified with podinfo.cue
Component: {
// Concrete values are defined in podinfo.cue
Name: string
Namespace: string
// Resources represents mix-in resources organized as a struct.
Resources: ExternalSecret: (Name): {
// Name is consistent with the component name.
metadata: name: Name
// Namespace is consistent with the component namespace.
metadata: namespace: Namespace
spec: {
// Ensure the target secret name is consistent.
target: name: metadata.name
// Ensure the name in the SecretStore is consistent.
dataFrom: [{extract: {key: metadata.name}}]
refreshInterval: "30s"
secretStoreRef: kind: "SecretStore"
secretStoreRef: name: "default"
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/podinfo/mixins.cue

View File

@@ -0,0 +1,2 @@
mkdir holos-cue-tutorial && cd holos-cue-tutorial
holos init platform v1alpha5

View File

@@ -0,0 +1 @@
mkdir -p components/podinfo

View File

@@ -0,0 +1,19 @@
package holos
// export the component build plan to holos
holos: Component.BuildPlan
// Component is a Helm chart
Component: #Helm & {
Name: "podinfo"
Namespace: "default"
// Add metadata.namespace to all resources with kustomize.
KustomizeConfig: Kustomization: namespace: Namespace
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}

View File

@@ -0,0 +1 @@
cat <<EOF > components/podinfo/podinfo.cue

View File

@@ -0,0 +1,6 @@
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}

View File

@@ -0,0 +1 @@
cat <<EOF > platform/podinfo.cue

View File

@@ -0,0 +1,3 @@
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s

View File

@@ -0,0 +1 @@
holos render platform

View File

@@ -0,0 +1,17 @@
3:20PM INF schemas vendored: external-secrets.io/clusterexternalsecret/v1beta1
3:20PM INF schemas vendored: external-secrets.io/clustersecretstore/v1alpha1
3:20PM INF schemas vendored: external-secrets.io/clustersecretstore/v1beta1
3:20PM INF schemas vendored: external-secrets.io/externalsecret/v1alpha1
3:20PM INF schemas vendored: external-secrets.io/externalsecret/v1beta1
3:20PM INF schemas vendored: external-secrets.io/pushsecret/v1alpha1
3:20PM INF schemas vendored: external-secrets.io/secretstore/v1alpha1
3:20PM INF schemas vendored: external-secrets.io/secretstore/v1beta1
3:20PM INF schemas vendored: generators.external-secrets.io/acraccesstoken/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/ecrauthorizationtoken/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/fake/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/gcraccesstoken/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/githubaccesstoken/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/password/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/uuid/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/vaultdynamicsecret/v1alpha1
3:20PM INF schemas vendored: generators.external-secrets.io/webhook/v1alpha1

View File

@@ -0,0 +1,4 @@
#! /bin/bash
set -euo pipefail
[[ -s "$1" ]] && [[ -z "${HOLOS_UPDATE_SCRIPTS:-}" ]] && exit 0
cat > "$1"

View File

@@ -7,6 +7,7 @@ sidebar_position: 50
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';
# CUE
@@ -25,92 +26,69 @@ Key concepts:
## The Code
### Holos Version
Ensure you have a current version of `holos` installed. This document was
tested with the following version.
import HolosVersionCommand from '!!raw-loader!./_cue/script-01-holos-version/command.sh';
import HolosVersionOutput from '!!raw-loader!./_cue/script-01-holos-version/output.txt';
<CodeBlock language="bash">{HolosVersionCommand}</CodeBlock>
<CodeBlock language="txt">{HolosVersionOutput}</CodeBlock>
### Generating the Structure
Use `holos` to generate a minimal platform directory structure. First, create
and navigate into a blank directory. Then, use the `holos generate platform`
command to generate a minimal platform.
```shell
mkdir holos-cue-tutorial && cd holos-cue-tutorial
holos init platform v1alpha5
```
import MkdirAndInit from '!!raw-loader!./_cue/script-02-cue/mkdir-and-init.sh';
<CodeBlock language="bash">{MkdirAndInit}</CodeBlock>
### Creating the Component
Create the directory for the `podinfo` component. Create an empty file, then add
the following CUE configuration to it.
```bash
mkdir -p components/podinfo
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
import MkdirComponents from '!!raw-loader!./_cue/script-02-cue/mkdir-components.sh';
import PodinfoHeader from '!!raw-loader!./_cue/script-02-cue/podinfo-component-header.sh';
import PodinfoBody from '!!raw-loader!./_cue/script-02-cue/podinfo-component-body.cue';
import EofTrailer from '!!raw-loader!./_cue/script-02-cue/eof-trailer.sh';
// export the component build plan to holos
holos: Component.BuildPlan
// Component is a Helm chart
Component: #Helm & {
Name: "podinfo"
Namespace: "default"
// Add metadata.namespace to all resources with kustomize.
KustomizeConfig: Kustomization: namespace: Namespace
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
```bash
EOF
```
<CodeBlock language="bash">{PodinfoHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{PodinfoBody}</CodeBlock>
<CodeBlock language="bash">{EofTrailer}</CodeBlock>
Register the component with the platform.
```bash
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
import RegisterHeader from '!!raw-loader!./_cue/script-02-cue/register-components-header.sh';
import RegisterBody from '!!raw-loader!./_cue/script-02-cue/register-components-body.cue';
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```
<CodeBlock language="bash">{RegisterHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{RegisterBody}</CodeBlock>
<CodeBlock language="bash">{EofTrailer}</CodeBlock>
Render the platform.
import RenderCommand from '!!raw-loader!./_cue/script-02-cue/render.sh';
import RegisterOutput from '!!raw-loader!./_cue/script-02-cue/register-components-output.txt';
<Tabs groupId="tutorial-hello-render-manifests">
<TabItem value="command" label="Command">
```bash
holos render platform
```
<CodeBlock language="bash">{RenderCommand}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
<CodeBlock language="txt">{RegisterOutput}</CodeBlock>
</TabItem>
</Tabs>
Add and commit the initial configuration.
```bash
git init . && git add . && git commit -m initial
```
import GitInit from '!!raw-loader!./_cue/script-02-cue/git-init.sh';
<CodeBlock language="bash">{GitInit}</CodeBlock>
### Mixing in Resources
@@ -120,39 +98,12 @@ component kind. This field is a convenient wrapper around the core [BuildPlan]
Create the mixins.cue file.
```bash
cat <<EOF > components/podinfo/mixins.cue
```
```cue showLineNumbers
package holos
import MixinHeader from '!!raw-loader!./_cue/script-02-cue/mixin-component-header.sh';
import MixinBody from '!!raw-loader!./_cue/script-02-cue/mixin-component-body.cue';
// Component fields are unified with podinfo.cue
Component: {
// Concrete values are defined in podinfo.cue
Name: string
Namespace: string
// Resources represents mix-in resources organized as a struct.
Resources: ExternalSecret: (Name): {
// Name is consistent with the component name.
metadata: name: Name
// Namespace is consistent with the component namespace.
metadata: namespace: Namespace
spec: {
// Ensure the target secret name is consistent.
target: name: metadata.name
// Ensure the name in the SecretStore is consistent.
dataFrom: [{extract: {key: metadata.name}}]
refreshInterval: "30s"
secretStoreRef: kind: "SecretStore"
secretStoreRef: name: "default"
}
}
}
```
```bash
EOF
```
<CodeBlock language="bash">{MixinHeader}</CodeBlock>
<CodeBlock language="cue" showLineNumbers>{MixinBody}</CodeBlock>
<CodeBlock language="bash">{EofTrailer}</CodeBlock>
:::important
Holos uses CUE to validate mixed in resources against a schema. The `Resources`
@@ -169,32 +120,15 @@ tutorial.
To import your own custom resource definitions, use [Timoni]. We imported the
ExternalSecret CRDs embedded in `holos` using the following command.
import ImportCRDs from '!!raw-loader!./_cue/script-02-cue/import-crds.sh';
import ImportOutput from '!!raw-loader!./_cue/script-02-cue/timoni-vendor.txt';
<Tabs groupId="35B1A1A1-D7DF-4D27-A575-28556E182096">
<TabItem value="command" label="Command">
```bash
timoni mod vendor crds -f https://raw.githubusercontent.com/external-secrets/external-secrets/v0.10.5/deploy/crds/bundle.yaml
```
<CodeBlock language="bash">{ImportCRDs}</CodeBlock>
</TabItem>
<TabItem value="output" label="Output">
```txt
2:22PM INF schemas vendored: external-secrets.io/clusterexternalsecret/v1beta1
2:22PM INF schemas vendored: external-secrets.io/clustersecretstore/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/clustersecretstore/v1beta1
2:22PM INF schemas vendored: external-secrets.io/externalsecret/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/externalsecret/v1beta1
2:22PM INF schemas vendored: external-secrets.io/pushsecret/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/secretstore/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/secretstore/v1beta1
2:22PM INF schemas vendored: generators.external-secrets.io/acraccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/ecrauthorizationtoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/fake/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/gcraccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/githubaccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/password/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/uuid/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/vaultdynamicsecret/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/webhook/v1alpha1
```
<CodeBlock language="txt">{ImportOutput}</CodeBlock>
</TabItem>
</Tabs>
@@ -212,42 +146,15 @@ existing [resources.cue] file.
Render the platform with the `ExternalSecret` mixed into the podinfo component.
```shell
holos render platform
```
<CodeBlock language="bash">{RenderCommand}</CodeBlock>
Take a look at the diff to see the mixed in `ExternalSecret`.
```shell
git diff deploy
```
import GitDiff from '!!raw-loader!./_cue/script-02-cue/git-diff.sh';
import DiffOutput from '!!raw-loader!./_cue/script-02-cue/git.diff';
```diff
diff --git a/deploy/components/podinfo/podinfo.gen.yaml b/deploy/components/podinfo/podinfo.gen.yaml
index 6e4aec0..f79e9d0 100644
--- a/deploy/components/podinfo/podinfo.gen.yaml
+++ b/deploy/components/podinfo/podinfo.gen.yaml
@@ -112,3 +112,19 @@ spec:
volumes:
- emptyDir: {}
name: data
+---
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+ name: podinfo
+ namespace: default
+spec:
+ dataFrom:
+ - extract:
+ key: podinfo
+ refreshInterval: 30s
+ secretStoreRef:
+ kind: SecretStore
+ name: default
+ target:
+ name: podinfo
```
<CodeBlock language="bash">{GitDiff}</CodeBlock>
<CodeBlock language="diff">{DiffOutput}</CodeBlock>
We saw how to mix in resources using the `Resources` field of the
[ComponentConfig]. This approach works for every kind of component in Holos,

View File

@@ -0,0 +1,19 @@
package main
import (
"path/filepath"
"testing"
)
// Run these with go test -v to see the verbose names
func TestCue(t *testing.T) {
t.Run("TestCue", func(t *testing.T) {
// Get an ordered list of test script files.
dir := "_cue"
for _, file := range sortedTestScripts(t, filepath.Join(dir, "examples")) {
t.Run("examples", func(t *testing.T) {
runOneScript(t, dir, file)
})
}
})
}

View File

@@ -1,3 +1,7 @@
#! /bin/bash
set -xeuo pipefail
# Necessary to install timoni for testscript execution
make go-deps
go test -coverprofile=coverage.out ./...

View File

@@ -16,4 +16,5 @@ import (
_ "golang.org/x/tools/cmd/godoc"
_ "google.golang.org/protobuf/cmd/protoc-gen-go"
_ "honnef.co/go/tools/cmd/staticcheck"
- "github.com/stefanprodan/timoni/cmd/timoni"
)