Compare commits

...

126 Commits

Author SHA1 Message Date
Jeff McCune
386fb89cc6 ci: replace lint workflow with cspell
The lint workflow was slow and we don't often change buf or angular
these days so they're not necessary.

The remaining valuable task is cspell, which we can speed up with a
dedicated step.
2024-12-20 13:52:54 -08:00
Jeff McCune
c5401d6b02 ci: speed up tests by killing steps 2024-12-20 11:57:05 -08:00
Jeff McCune
f215405643 docs: fix links in readme 2024-12-20 07:28:04 -08:00
Jeff McCune
2c79982bd3 cue: enable @embed for loading yaml (#385)
mpvl suggests @embed is a more ideal solution than our implementation of
core.Component.Instances for the use case of unifying YAML data updated
by Kargo Stage resources.

See the issue for a link to the discussion.
2024-12-20 07:14:01 -08:00
Jeff McCune
e5e4de3073 cue: update to 0.11.1
go get cuelang.org/go/cmd/cue@latest

    go: downloading cuelang.org/go v0.11.1
    go: upgraded cuelang.org/go v0.11.0 => v0.11.1
2024-12-20 07:09:39 -08:00
Jeff McCune
ec462f5f0b docs: redirect /docs/support 2024-12-19 22:13:04 -08:00
Jeff McCune
0e95a2812e cmd: expose MakeMain() for testing
I'd like to add the kargo-demo repository to Unity to test evalv3, but
can't get a handle on the main function to wire up to testscript.

This patch fixes the problem by moving the MakeMain function to a public
package so the kargo-demo go module can import and call it using the go
mod tools technique.
2024-12-19 15:19:46 -08:00
Jeff McCune
54efe3e24a core: pass --extract-yaml flag from platform to component (#376)
Previously holos render platform was not setting the --extract-yaml file
when calling holos render component, causing data file instances defined
in the Platform spec to be discarded.

This patch passes the value along using the flag.
2024-12-19 08:39:55 -08:00
Jeff McCune
f693f049f4 core: refactor --instance to --extract-yaml (#376)
Extract YAML is more clear and aligns with the schema docs for the
Component Instance field which has an extractYAML kind.  This also
leaves the door open for additional kinds of data extractors which are
almost certainly going to be needed.
2024-12-19 08:34:05 -08:00
Jeff McCune
85238710ac core: unify data files into config (#376)
Previously there isn't a good way to unify json and yaml files with the
cue configuration.  This is a problem for use cases where data can be
generated idempotentialy prior to rendering the platform configuration.

The first use case is to explore unifying configuration with decrypted
sops values, which isn't typical since Holos is designed to handle
secrets with ExternalSecret resources, but does fit into the use case of
executing a command to produce data idempotently, then make the data
available to the platform configuration.

Other use cases this feature is intended to support are the prior
experiment where we fetch top level platform configuration from an rpc
service, and the future goal of integrating with data provided by
Terraform.
2024-12-19 08:34:05 -08:00
Jeff McCune
3ec62d272e v1alpha5: update kargo crds to 1.1.1 2024-12-19 08:34:04 -08:00
Jeff McCune
49afb44fd4 docs: redirect /docs/comparison/ 2024-12-18 14:37:36 -08:00
Gary Larizza
a023f135ab Add a Comparisons page
PROBLEM:

We've noticed that Holos almost immediately gets compared to Timoni, and
we frequently get asked for specifics in how they're similar/different.

SOLUTION:

* Add a `Comparison` page.
* Include a section that compares Holos to Timoni

OUTCOME:

Fewer questions about how Holos compares to Timoni because people are
able to find that answer themselves on our docs page.
2024-12-18 14:33:52 -08:00
Jeff McCune
c6a3a5d689 docs: redirect /docs/kargo/ 2024-12-17 06:30:20 -08:00
Jeff McCune
3f1eed3f06 platform: add kargo.akuity.io custom resource definitions
Needed for Kargo integration.  Imported with timoni from v1.0.3 Kargo
CRD's.
2024-12-16 13:19:39 -08:00
Jeff McCune
7fb7df1441 docs: make the linter happy 2024-12-16 11:04:35 -05:00
Jeff McCune
a798111d4d docs: add oci helm charts example
Question came up in chat, there isn't a good example and it's a pain to
piece together from the reference docs.
2024-12-16 10:56:50 -05:00
Jeff McCune
3ddb823341 docs: add note about compinit
Andy ran into issues enabling completion without first figuring out how
to initialize the completion system.
2024-12-16 08:15:45 -05:00
Jeff McCune
70d48592c4 docs: fix environments topic
It didn't work, failed with:

  ❯ holos show buildplans --selector app.holos.run/city=ams
  could not run: Component.Name: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/instance.go:66
  Component.Name: 2 errors in empty disjunction:
  Component.Name: conflicting values "no-name" and "podinfo-ams":
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:6:12
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:6:13
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:35:2
      /Users/jeff/Holos/foo/holos-environments-tutorial/tags.cue:13:19
  Component.Name: conflicting values "podinfo" and "podinfo-ams":
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:6:12
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:7:8
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:6:13
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:35:2

This was likely because the podinfo component was used in different ways
in different topics.  Don't use the shared component to fix the problem.
2024-12-13 09:20:52 -05:00
Jeff McCune
006f08df93 docs: add kargo place holder (#378) 2024-12-11 09:58:54 -08:00
Jeff McCune
39e2db5d37 docs: remove related content from youtube embed
Except stuff in our own channel.
2024-12-08 19:43:12 -08:00
Jeff McCune
ceb293fd8a docs: fix typescript className not class check error 2024-12-08 19:36:36 -08:00
Jeff McCune
188ff95015 docs: enable youtube fullscreen
Without this patch the fullscreen button is disabled.
2024-12-08 19:33:06 -08:00
Jeff McCune
5f658e0ba0 docs: add flux kustomization example (#374)
Almost identical to the ArgoCD Application example.
2024-12-08 19:20:12 -08:00
Jeff McCune
18b2850d3c platform: import flux custom resources
kustomize build https://github.com/fluxcd/flux2/manifests/crds\?ref=v2.4.0 \
      timoni mod vendor crds -f-
2024-12-08 19:03:18 -08:00
Jeff McCune
366a7fe93d docs: private helm repos need updated schemas (#370)
Document the need to run holos init platform v1alpha5 --force to use the
private helm repository feature.
2024-12-08 17:13:56 -08:00
Jeff McCune
f71d6d5bd9 helm: support private helm repositories (#370)
Previously holos unconditionally executed helm repo add which failed for
private repositories requiring basic authentication.

This patch addresses the problem by using the Helm SDK to pull and cache
charts without adding them as repositories.  New fields for the
core.Helm type allow basic auth credentials to be read from environment
variables.

Multiple repositories are supported by using different env vars for
different repositories.
2024-12-06 15:38:46 -08:00
Gary Larizza
4529673e93 Embed YouTube Video (#369)
PROBLEM:

We've created a YouTube video walking people through Holos and the Helm
Values tutorial, but now we need to embed it on the site for visitors to
watch.

SOLUTION:

* Create a `YouTube` MDX plugin
* Use that plugin on Overview and Helm Values
* Tune the video size/attributes using CSS

OUTCOME:

The Helm Values YouTube video is embedded on our site for visitors to
watch.
2024-12-06 15:33:03 -08:00
Jeff McCune
16a6447926 helm: support oci images in chart name
Without this patch we do not support installing Kargo from an OCI helm
chart.  We want to support:

```
Component: #Helm & {
	Name:      "kargo"
	Namespace: Kargo.Namespace

	Chart: {
		name:    "oci://ghcr.io/akuity/kargo-charts/kargo"
		version: "1.0.3"
		release: Name
	}
	EnableHooks: true

	Values: Kargo.Values
}
```

This patch fixes the problem by using the base name for filesystem cache
operations.
2024-12-03 12:15:06 -08:00
Jeff McCune
111a5944ff cue: bump to 0.11.0
go get cuelang.org/go/cmd/cue@latest
2024-12-02 12:37:19 -08:00
Jeff McCune
ff1446dc93 docs: redirect /docs/guides/helm/
This shows up in the Unity tests I'm working on with mvdan and goes to a
blank page without the redirect in place.

	--- FAIL: TestGuides_v1alpha5 (0.00s)
	   --- FAIL: TestGuides_v1alpha5/helm (0.60s)
	       testscript.go:584: # Helm Guide https://holos.run/docs/guides/helm/
2024-12-02 09:05:13 -08:00
Jeff McCune
67ef990c37 v0.101.2 build tags 2024-12-02 08:09:23 -08:00
Jeff McCune
6bd54ab856 render: pass build tags from platform to component (#366)
Previously, build tags were not propagated from `holos render platform
-t validate` through to the underlying `holos render component` command.
This is a problem because validators need to be selectively enabled as a
work around until we have an audit mode field.

This patch fixes the problem by propagating command line tags from the
render platform command to the underlying commands.  This patch also
propagates tags for the show command.
2024-11-30 20:56:11 -08:00
Jeff McCune
89a23a10fd docs: remove DIRECTORY from holos render platform --help
The directory argument is deprecated now, use the --platform flag
instead.
2024-11-30 13:08:01 -08:00
Jeff McCune
5a939bb6fe render: support cue build tags e.g. -t foo for @if(foo) (#366)
Previously Holos only supported tags in the form of key=value.  CUE
supports boolean style tags in the form of `key [ "=" value ]` which we
want to use to conditionally use to register components with the
platform.

This patch modifies the flag parsing to support -t foo like cue does,
for use with the @if(foo) build tag.
2024-11-30 12:50:31 -08:00
Jeff McCune
ee16f14e03 refactor build plan pipeline
Previously the BuildPlan pipeline didn't execute generators and
transformers concurrently.  All steps were sequentially executed.  Holos
was primarily concurrent by executing multiple BuildPlans at once.

This patch changes the Build implementation for each BuildPlan to
execute a GoRoutine pipeline.  One producer fans out to a group of
routines each executing the pipeline for one artifact in the build plan.
The pipeline has 3 stages:

1: Fan-out to build each Generator concurrently.
2: Fan-in to build each Transformer sequentially.
3: Fan-out again to run each validator concurrently.

When the artifact pipelines return, the producer closes the tasks
channel causing the worker tasks to return.

Note the overall runtime for 8 BuildPlans is roughly equivalent to
previously at 160ms with --concurrency=8 on my M3 Max.  I expect this to
perform better than previously when multiple artifacts are rendered for
each BuildPlan.
2024-11-29 14:52:06 -08:00
Jeff McCune
7530345620 main: add tracing and profiling
Writes files based on parent pid and process pid to avoid collisions.

Analyze with:

export HOLOS_TRACE=trace.%d.%d.out
go tool trace trace.999.1000.out

export HOLOS_CPU_PROFILE=cpu.%d.%d.prof
go tool pprof cpu.999.1000.prof

export HOLOS_MEM_PROFILE=mem.%d.%d.prof
go tool pprof mem.999.1000.prof
2024-11-29 14:52:06 -08:00
Jeff McCune
47d60ef86d docs: fix cue vet path in validators post (#357)
Without this patch the validator fails if a component manages two of the
same kind of resource, which is common.

This patch updates the example to use the metadata namespace and name as
lookup keys.  This works for most components, but may not for
ClusterResources.  Use the kind top level field in that case and pass
the field name of the validator as a tag value to vary by component.
2024-11-25 19:35:02 -08:00
Jeff McCune
9e9f6efd04 docs: fix typos in validators blog post (#357) 2024-11-25 15:46:54 -08:00
Jeff McCune
fb4a043823 docs: add card for validators blog post (#357) 2024-11-25 15:11:11 -08:00
Jeff McCune
d718ab1910 docs: redirect /docs/local-cluster to the v1alpha5 topic 2024-11-25 10:53:31 -08:00
Jeff McCune
c649db18a9 docs: redirect /docs/quickstart to the overview 2024-11-25 10:43:16 -08:00
Jeff McCune
b3bddf3ee3 docs: add validators blog post (#357) 2024-11-25 08:49:27 -08:00
Jeff McCune
77836be250 docs: update readme 2024-11-25 08:10:11 -08:00
Jeff McCune
4db670b854 docs: add validators tutorial (#357)
Add a tutorial page on validators.
2024-11-24 21:34:09 -08:00
Jeff McCune
d87c919519 docs: redirect /docs/topics to structures
/docs/v1alpha5/api/author/ links to it in the opening paragraphs.
2024-11-24 19:38:30 -08:00
Jeff McCune
2184bda2a1 v1alpha5: add validators (#357) 2024-11-24 17:40:41 -08:00
Jeff McCune
a8ab4dabaa cue: fix holos cue vet exit code (#358)
Without this patch `holos cue vet` always returns exit code 0, even when
there are errors.

This patch fixes the problem by catching the error and returning it to
our own top level error handler.  Note the final error, "could not run:
terminating because of errors" which wraps the generic error reported by
cue in the presence of multiple errors.

Result:

```
❯ holos cue vet ./policy --path 'strings.ToLower(kind)' /tmp/podinfo.gen.yaml
deployment.kind: conflicting values "Forbidden" and "Deployment":
    ./policy/validations.cue:18:8
    ../../../../../tmp/podinfo.gen.yaml:25:7
deployment.spec.template.spec.containers.0.resources.limits: conflicting values null and {[string]:"k8s.io/apimachinery/pkg/api/resource".#Quantity} (mismatched types null and struct):
    ./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:355:9
    ./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:376:12
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2840:11
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2968:14
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:15
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:18
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:5027:9
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:6407:16
    ./policy/validations.cue:17:13
    ../../../../../tmp/podinfo.gen.yaml:104:19
could not run: terminating because of errors
```
2024-11-24 16:39:56 -08:00
Jeff McCune
7175950ce0 docs: redirect /docs/overview permalink 2024-11-24 08:58:08 -08:00
Jeff McCune
e186c1be37 docs: edits for grammar 2024-11-21 15:55:14 -08:00
Jeff McCune
a998513e34 docs: edit for grammar and make concise 2024-11-21 15:26:20 -08:00
Jeff McCune
2f821ec33c docs: add environments topic page (#354)
Similar to the Clusters topic, add a topic about configuring multiple
environments.  This likely needs some work, the example is a bit
contrivied but at least shows how we can look up attributes, then use
those attributes to look up additional configuration from platform-wide
configuration data.
2024-11-21 15:07:46 -08:00
Jeff McCune
4d54190f2e docs: remove empty topic pages
We'll add them back in as we write them.
2024-11-21 14:40:39 -08:00
Gary Larizza
b466ec3457 Merge pull request #353 from holos-run/gl/fix-platform-args
Remove deprecated argument to 'holos render platform'
2024-11-21 13:04:17 -08:00
Gary Larizza
45120797b9 Remove deprecated argument to 'holos render platform'
This commit removes the extra `./platform` argument from any of the
current tutorial/topic docs to reflect the change that was made to
`holos render platform` in version `0.100.0`.
2024-11-21 11:49:44 -08:00
Jeff McCune
e6d25bf5eb holos: improve OrderedEncoder error handling
If someone accidentally provides the same index multiple times, or
indexes less than the next expected, the program would silently discard
the data.  This would be difficult to troubleshoot since an
OrderedEncoder is usually used with concurrent go routines, which would
likely mislead the investigator.

Better to just fail hard with an error indicating the caller in these
situations.
2024-11-20 17:02:16 -08:00
Jeff McCune
3ad6e69336 docs: update report issue template
Show how we can copy and paste directly from the issue into testscript.
2024-11-20 15:58:02 -08:00
Gary Larizza
9b10e23e43 Redirect docs to current version (#352)
* Redirect docs to current version

PROBLEM:

See #350 for context. There's a GitHub issue open for this on the
facebook docusaurus repo:
https://github.com/facebook/docusaurus/issues/9049

SOLUTION:

Use the redirect workaround for the time being.

OUTCOME:

https://holos.run/docs will link to the most recent version of our docs
site.

* Update doc/website/static/_redirects

Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>

---------

Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>
Co-authored-by: Jeff McCune <jeff@openinfrastructure.co>
2024-11-20 15:33:14 -08:00
Jeff McCune
b19022d3ff tests: fill in the buildplan.yaml for better coverage (#348)
Assert against the complete build plan so we know if we change the
output format in the future.

It's easy to update if so:

  HOLOS_UPDATE_SCRIPTS=1 go test github.com/holos-run/holos/cmd/holos
2024-11-20 15:21:41 -08:00
Jeff McCune
3fd06f594b schemas: fix kustomize patch name field (#348)
Without this patch trying to use a Kustomize patch with the optional
name field omitted results in the following error:

  could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
  holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
      $WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2

This patch fixes the problem by providing a default value for the name
field matching the Go zero value for a string.
2024-11-20 15:08:44 -08:00
Jeff McCune
a75338f21c docs: add bug report issue template, disable dev deploy 2024-11-20 14:50:25 -08:00
Jeff McCune
6002040360 version 0.100.0 2024-11-20 11:34:04 -08:00
Jeff McCune
864d7d442b build: pass platform component labels and annotations to BuildPlan
Without this patch the BuildPlan resulting from a Platform that has
components with labels and annotations does not have the labels or
annotations of the source component.

Holos should copy the labels and annotations defined on each of the
Platform.spec.components to the resulting BuildPlan so end users can see
clearly where a BuildPlan originated from, and filter with selectors the
intermediate output BuildPlan the same way we filter with selectors the
original Platform spec components list.

Result:

```
holos init platform v1alpha5 --force
holos show buildplans  | head
```

```yaml
kind: BuildPlan
apiVersion: v1alpha5
metadata:
  name: podinfo
  labels:
    app.holos.run/cluster: local
    app.holos.run/name: podinfo
  annotations:
    app.holos.run/description: podinfo for cluster local
```
2024-11-20 11:34:03 -08:00
Jeff McCune
791c0a9ffd util: fix misleading RunCmd error message source location
The error message misleads the reader to the utility function.  It
should lead to the caller.
2024-11-20 11:23:54 -08:00
Jeff McCune
bfe4a8d7c4 remove unmaintained embedded platforms
v1alpha5 is current and maintained.
2024-11-20 11:23:54 -08:00
Jeff McCune
eaa508a6f8 remove v1alpha1 schemas and support 2024-11-20 11:23:53 -08:00
Jeff McCune
63256a2845 remove embedded k3d platform
No longer used, v1alpha5 is used in the docs and maintained.
2024-11-20 11:23:53 -08:00
Jeff McCune
937c1dc953 show: fix inconsistent ordering of output
Without this patch the holos show buildplans command returns results in
an inconsistent order.  This is a problem because the output should be
idempotent.

This patch fixes the problem by adding an EncodeSeq(idx int, v any) method to
the encoder interface.  idx represents the index position of the
Platform.spec.components list after selector filtering has been applied.

This patch modifies the json and yaml encoders to buffer out of order
results from the concurrent go routines.

Result:

Concurrent execution is preserved. The buffer is kept to a reasonable
size, entries are deleted once they're encoded in the correct order.

Most importantly the output is consistent and idempotent so we can write
effective integration tests.
2024-11-20 11:23:52 -08:00
Jeff McCune
11bd50e2eb show: fix buildplans selector inconsistency
Sometimes, but not always, the holos show buildplans command produces no
output.

```
❯ holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug
finalized config from flags
rendered platform in 13.458µs
```

It only happens when there's a selector.  It doesn't happen without the
selector flag.  It only happens with ==, not with =.

This test fails quickly.

```
while [[ $(holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug | wc -l) -eq 39 ]]; do true; done
```

This test runs until killed.

```
while [[ $(holos show buildplans --log-level=debug | wc -l) -eq 279 ]]; do true; done
```

Solution:

The problem is the use of the map.  Iterating over the keys happens in a
random order.  With the fix we check in an explicit order.
2024-11-20 11:23:52 -08:00
Jeff McCune
7cfcf55565 add yaml struct tags to core v1alpha5 schemas
Without this patch the `holos show buildplans` BuildPlan output has
incorrect yaml with the v3 encoder.  For example apiversion: v1alpha5
instead of apiVersion.
2024-11-20 11:23:52 -08:00
Jeff McCune
8b22ba04e1 cli: add show command and refactor interfaces (#331)
Show subcommand:

This is large change that accomplishes a number of goals.  First, there
was no convenient way to show a build plan without using the debug logs
to indentify the tags to inject, then calling the cue command with the
right incantation to inspect the BuildPlan.

This patch addresses the problem by adding a `holos show buildplans`
command.  The command loads the Platform spec from the platform
directory, then iterates over all Components to produce the BuildPlan.

This patch adds labels and annotations to the platform Components
collection in order to select and filter the output.

Result:

```
❯ holos show components --selector app.holos.run/cluster=local --format=yaml | head
kind: BuildPlan
apiversion: v1alpha5
metadata:
  name: podinfo
spec:
  artifacts:
    - artifact: clusters/local/components/podinfo/podinfo.gen.yaml
      generators:
        - kind: Helm
          output: helm.gen.yaml
```

---

Interface refactor:

This refactors the interface between the `holos` Go CLI layer and the
various core schema data structures.  We now use a proper Go interface.
Concurrent execution over platform components has been improved to
accept a closure function so we can use the same interface method to
process the components.  We use this to show each component and render
each component from different subcommands using the same interface
embedded in the builder.Platform struct.

The embedded interface allows us to easily swap in different versions,
e.g. v1beta1 and eventually v1.  The number of interface methods are
quite small.  14 methods across 4 interfaces in holos/interface.go.

---

Remove old versions:

This patch removes support for versions prior to v1alpha5 in an effort
to clean up cruft.
2024-11-20 11:23:51 -08:00
Nate McCurdy
7d5187873b docs: Remove non-existent CLI completions, add pwsh
Cobra provides completions for bash, zsh, fish, and powershell, not
ksh.
2024-11-18 10:27:33 -08:00
Jeff McCune
03b796312a cli: gate grpc client and auth flags behind feature flag
Previously the holos render platform and component subcommands had flags
for oidc authentication and client access to the gRPC service.  These
flags aren't currently used, they're remnants from the json powered form
prototype.

This patch gates the flags behind a feature flag which is disabled by
default.

Result:

  holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int   number of components to render concurrently (default 8)
  -v, --version           version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")

---

  HOLOS_FEATURE_CLIENT=1 holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int             number of components to render concurrently (default 8)
      --oidc-client-id string       oidc client id. (default "270319630705329162@holos_platform")
      --oidc-extra-scopes strings   optional oidc scopes
      --oidc-force-refresh          force refresh
      --oidc-issuer string          oidc token issuer url. (default "https://login.holos.run")
      --oidc-scopes strings         required oidc scopes (default openid,email,profile,groups,offline_access)
      --server string               server to connect to (default "https://app.holos.run:443")
  -v, --version                     version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")
2024-11-17 16:06:57 -08:00
Jeff McCune
20fb39e49b docs: add clusters topic (#343)
Previously we didn't have good documentation on how to manage multiple
sets of clusters.

This patch adds a clusters topic in the structures category.  Each one
of the environments, projects, owners, etc... structures follow the same
pattern as #Clusters and #ClusterSets, so it makes sense to put them
into a dedicated sidebar category for specific CUE structures.
2024-11-17 14:45:32 -08:00
Jeff McCune
c9c8c13810 docs: replace touch with cat 2024-11-15 09:54:09 -07:00
Jeff McCune
374cd872e9 docs: CUE not cue and typo fix in hello holos 2024-11-15 09:40:02 -07:00
Jeff McCune
8db06dd0e1 releaser: fix brew test command (#327)
holos version isn't a valid command but holos --version is.
2024-11-14 16:14:26 -07:00
Jeff McCune
66acadf86d docs: support brew install (#327) 2024-11-14 16:07:13 -07:00
Jeff McCune
032f72b435 render: log helm pull errors (#332)
Previously errors were not logged, giving no indication what went wrong.
This patch changes the error handler to log errors from helm.
2024-11-14 09:44:27 -07:00
Jeff McCune
2380223794 docs: add argocd application example (#340)
When we moved from v1alpha4 to v1alpha5 we removed ArgoConfig from the
author schema.  There was no longer a clear example of how to configure
an ArgoCD Application for every component in v1alpha5.

This patch adds a topic document with an example of how to add an
Application along side the resources by mixing an additional Artifact
into the BuildPlan.
2024-11-13 16:30:59 -07:00
Jeff McCune
e6892c3b16 v0.99.0 2024-11-13 12:49:28 -07:00
Jeff McCune
847fd2958e helm: add support for helm template --kube-version capabilities (#330)
Previously the Helm generator had no support for the --kube-version
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:43:01 -07:00
Jeff McCune
cf622835db helm: add support for helm template --api-versions capabilities (#330)
Previously the Helm generator had no support for the --api-versions
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:42:50 -07:00
Jeff McCune
1f5dc3a082 docs: add note about tested helm version (#335)
To help users understand what should definitely work.
2024-11-13 09:45:56 -07:00
Jeff McCune
9f4da68dc9 v0.98.2 2024-11-13 09:19:30 -07:00
Jeff McCune
2ee056be9f cue: fix panic with no args (#334)
Fixes:

```
❯ holos
panic: runtime error: slice bounds out of range [2:1]

goroutine 1 [running]:
github.com/holos-run/holos/internal/cli.newCueCmd(...)
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:121
github.com/holos-run/holos/internal/cli.New(0xc0002837c0, {0x3826e00, 0x4f60860})
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:102 +0x772
main.main.MakeMain.func1()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/main.go:22 +0x5b
main.main()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/cmd/holos/main.go:10 +0x3e
```
2024-11-13 09:04:37 -07:00
Jeff McCune
394e2cb0b2 docs: add cue tutorial (#318)
Show how to use the ComponentConfig Resources field to mix in resources.
2024-11-13 08:00:37 -07:00
Jeff McCune
cf95c9664d docs: change hello holos parameters to greeting (#328)
Version doesn't make as much sense since we're doing a hello world
tutorial.

Also consolidate the values information into one step, and consolidate
the breaking it down section to make it shorter and clearer.
2024-11-12 09:46:19 -07:00
Jeff McCune
0192eeeb7e docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest

This time in the correct directory.
2024-11-11 17:25:17 -07:00
Jeff McCune
ed54bcc58f docs: rename cue-generator to cue
The main use case is to manage resources from CUE, but CUE has many uses
in Holos such as validation and driving Kustomize.
2024-11-11 17:16:53 -07:00
Jeff McCune
9ac7f185f9 docs: fix broken validators link in diagram 2024-11-11 17:11:35 -07:00
Jeff McCune
7de72d3dab docs: add component parameters example to hello holos (#328)
The important note was weird because we didn't show an example of how to
use component parameters.  This patch replaces the note with an example.
2024-11-11 16:56:16 -07:00
Jeff McCune
2e3c998454 docs: add directory tree to hello holos doc (#324)
Feedback from Zack, give a tree so people skimming know where to figure
out the lay of the land.
2024-11-11 16:19:48 -07:00
Jeff McCune
580afffa7f 0.98.1 - holos init platform 2024-11-11 14:44:00 -07:00
Jeff McCune
67535e1e1d doc: remove init platform from setup guide
We have it in the hello guide, setup should just be install only and not
how to use the tool yet.
2024-11-11 14:41:58 -07:00
Nate McCurdy
767ea69d2e docs: Add a tree view to Hello Holos
A tree view of the `holos-tutorial/` directory should give readers a
quick, high-level understanding of the folder structure of a typical
Holos platform project.
2024-11-11 14:04:40 -07:00
Jeff McCune
21e1a116e4 cli: hide help flags and command (#325)
They're unnecessary.
2024-11-11 14:02:25 -07:00
Jeff McCune
65fe7779be cli: rename generate to init (#325)
This patch changes the `holos generate` command to `holos init` to match
other tools like `go mod init`.
2024-11-11 14:02:25 -07:00
Jeff McCune
0e7abf0173 docs: consolidate diagrams to @site/src/diagrams/
So we don't have two different copies in two different places.
2024-11-11 13:40:46 -07:00
Jeff McCune
cca022ac99 docs: move architecture diagrams (#323) 2024-11-11 12:03:21 -07:00
Jeff McCune
43e939d06a doc: refactor breaking it down table in hello holos
So it displays nicely on mobile.
2024-11-09 14:52:11 -08:00
Jeff McCune
8096268826 docs: fix diagram urls again 2024-11-09 14:40:08 -08:00
Jeff McCune
631b23091d docs: fix rendering overview diagram on blog 2024-11-09 14:33:54 -08:00
Jeff McCune
09c6476282 docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-09 14:30:44 -08:00
Jeff McCune
a768d16c5f docs: set current version to v1alpha5
Previously the current version would always be unreleased at /docs/next
and we'd have to copy the doc/md/ folder into the
doc/website/versioned_docs/version-v1alpha5/ every time we made a
change.

We're going to be working on v1alpha5 for awhile so it makes sense to
just have the current version published at /docs/v1alpha5/ and we can
start /docs/v1alpha6/ whenever we're ready.

This also has the nice effect of giving us permalinks if we change the
structure again.  /docs/v1alpha5/ will remain over time.
2024-11-09 14:29:27 -08:00
Jeff McCune
3834a7ef85 docs: add missing link to kustomize tutorial 2024-11-08 22:25:34 -08:00
Jeff McCune
606a1aae73 docs: add nav bar title back 2024-11-08 19:29:05 -08:00
Jeff McCune
340d07ee7a docs: fix announcing holos blog (#321) 2024-11-08 19:24:46 -08:00
Jeff McCune
12d2cec4d5 docs: fix rendering overview diagram links (#321) 2024-11-08 19:12:23 -08:00
Jeff McCune
e93feb49b7 docs: add version drop down to nav bar (#321) 2024-11-08 17:05:37 -08:00
Jeff McCune
dcf8602a0b docs: release v1alpha5 (#321) 2024-11-08 17:00:34 -08:00
Jeff McCune
e07c4d11c8 docs: revise helm values and kustomize tutorials (#316)
These are now where I'd like them to be.
2024-11-08 15:12:22 -08:00
Jeff McCune
b7e1c14192 docs: kustomize tutorial (#316)
Add httpbin using kustomize and patch the result, all from CUE.  The is
the second half of the v1alpha4 helm guide split into a dedicated
tutorial.
2024-11-08 14:08:48 -08:00
Jeff McCune
29f44cdac9 docs: helm values (#316)
Add a helm values tutorial which is a cut down version of the v1alpha4
helm guide.  The httpbin kustomize will immediately follow building on
the prometheus and blackbox charts.
2024-11-08 12:03:14 -08:00
Jeff McCune
96be7a4ae3 docs: add generate platform to hello holos (#311) 2024-11-07 17:59:07 -08:00
Jeff McCune
d6bd030a72 docs: fixup overview 2024-11-07 17:59:07 -08:00
Jeff McCune
75047b590f docs: hello holos edits (#311) 2024-11-07 17:19:53 -08:00
Gary Larizza
a05881df0f Add the Hello Holos tutorial 2024-11-07 16:47:36 -08:00
Jeff McCune
5f406fce5c docs: organize docs for new structure (#301) 2024-11-07 15:10:15 -08:00
Jeff McCune
49c945a037 docs: setup tutorial with diagrams (#301) 2024-11-07 14:13:49 -08:00
Gary Larizza
54de20f0b8 docs: setup tutorial (#301) 2024-11-07 10:24:49 -08:00
Gary Larizza
80b4ab9852 Merge pull request #310 from holos-run/jeff/308-holos-field
docs: rewrite technical overview for v1alpha5
2024-11-07 10:15:01 -08:00
Jeff McCune
acd98aa63c docs: rewrite technical overview for v1alpha5
Attribution: following the structure and length of the tokio docs, with
some more diagrams.
2024-11-07 10:07:45 -08:00
Jeff McCune
0afaab8f2b render: nest output under the holos top level field (#308)
Previously the holos command line expected a Platform and BuildPlan
resource at the top level of the exported data from CUE.  This forced us
to use hidden fields for everything else.

This patch modifies the BuildData struct to first look for a holos top
level field and use it if present.  This opens up other top level fields
for use by end users.

Our intent is to reserve any top level field prefixed with holos.

Note this follows how Timoni works as well.
2024-11-07 07:00:26 -08:00
Jeff McCune
7ded38bc3f v1alpha5: strip down the core and author schemas (#306)
This patch strips down the v1alpha4 core and author schemas to only with
is absolutely necessary for all holos users.  Aspects of platform
configuration applicable to some, even most, but not all users will be
moved into documentation topics organized as a recipe book.

The functionality removed from the v1alpha4 author schemas in v1alpha5
will move into self contained examples documented as topics on the docs
site.

The overall purpose is to have a focused, composeable, maintainable
author schema to help people get started and ideally we can support for
years with making breaking changes.

With this patch the v1alpha5 helm guide test passes.  We're not going to
have this guide anymore but it demonstrates we're back to where we were
with v1alpha4.
2024-11-06 15:22:17 -08:00
Jeff McCune
840676709a docs: partially write the overview doc for v1alpha5
Partial, switching gears to v1alpha5 to unblock others working on
discrete topics.
2024-11-05 10:19:59 -08:00
731 changed files with 38845 additions and 191705 deletions

View File

@@ -6,10 +6,12 @@
],
"words": [
"acmesolver",
"acraccesstoken",
"acraccesstokens",
"admissionregistration",
"alertmanager",
"alertmanagers",
"anchore",
"anthos",
"apiextensions",
"apimachinery",
@@ -27,12 +29,14 @@
"authpolicy",
"authproxy",
"authroutes",
"autoload",
"automount",
"automounting",
"autoscaler",
"balancereader",
"blackbox",
"buildplan",
"buildplans",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
@@ -41,6 +45,7 @@
"certificaterequest",
"certificaterequests",
"certificatesigningrequests",
"chartmuseum",
"clientset",
"clsx",
"clusterexternalsecret",
@@ -55,6 +60,8 @@
"Cmds",
"CNCF",
"CODEOWNERS",
"compinit",
"componentconfig",
"configdir",
"configmap",
"configmapargs",
@@ -67,6 +74,7 @@
"creds",
"crossplane",
"crunchydata",
"ctxt",
"cuecontext",
"cuelang",
"customresourcedefinition",
@@ -74,9 +82,11 @@
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devel",
"devicecode",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtoken",
"ecrauthorizationtokens",
"edns",
"endpointslices",
@@ -92,9 +102,11 @@
"fieldmaskpb",
"fieldspec",
"flushcache",
"fluxcd",
"fullname",
"gatewayclass",
"gatewayclasses",
"gcraccesstoken",
"gcraccesstokens",
"gendoc",
"generationbehavior",
@@ -103,6 +115,7 @@
"genproto",
"ggnpl",
"ghaction",
"githubaccesstoken",
"githubaccesstokens",
"gitops",
"GOBIN",
@@ -133,6 +146,7 @@
"httproute",
"httproutes",
"iampolicygenerator",
"incpatch",
"Infima",
"intstr",
"isatty",
@@ -142,6 +156,7 @@
"jetstack",
"jiralert",
"Jsonnet",
"Kargo",
"kfbh",
"killall",
"kubeadm",
@@ -149,6 +164,7 @@
"kubelet",
"kubelogin",
"kubernetesobjects",
"kubeversion",
"Kustomization",
"Kustomizations",
"kustomize",
@@ -165,6 +181,7 @@
"loadbalancer",
"loadrestrictions",
"logfmt",
"lxnl",
"mattn",
"mccutchen",
"metav",
@@ -177,6 +194,7 @@
"mutatingwebhookconfigurations",
"mvdan",
"mxcl",
"mychart",
"myhostname",
"myRegistrKeySecretName",
"mysecret",
@@ -190,6 +208,7 @@
"oauthproxy",
"objectmap",
"objectmeta",
"omitempty",
"organizationconnect",
"orgid",
"otelconnect",
@@ -249,6 +268,7 @@
"rolebinding",
"rootfs",
"ropc",
"sboms",
"seccomp",
"secretargs",
"SECRETKEY",
@@ -291,13 +311,16 @@
"tokencache",
"Tokener",
"tolerations",
"TOPLEVEL",
"Traceid",
"traefik",
"transactionhistory",
"tsdb",
"txtar",
"typemeta",
"udev",
"uibutton",
"Unmarshal",
"unstage",
"untar",
"upbound",
@@ -309,6 +332,7 @@
"userservice",
"validatingwebhookconfiguration",
"validatingwebhookconfigurations",
"vaultdynamicsecret",
"vaultdynamicsecrets",
"virtualservice",
"virtualservices",

131
.github/ISSUE_TEMPLATE/bug-report.md vendored Normal file
View File

@@ -0,0 +1,131 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: NeedsInvestigation, Triage
assignees: ''
---
<!--
Please answer these questions before submitting your issue. Thanks!
To ask questions, see https://github.com/holos-run/holos/discussions
-->
### What version of holos are you using (`holos --version`)?
```
0.0.0
```
### Does this issue reproduce with the latest release?
<!--
Get the latest release with:
brew install holos-run/tap/holos
Or see https://holos.run/docs/v1alpha5/tutorial/setup/
-->
### What did you do?
<!--
Please provide a testscript that should pass, but does not because of the bug.
See the below example.
You can create a txtar from a directory with:
holos txtar ./path/to/dir
Refer to: https://github.com/rogpeppe/go-internal/tree/master/cmd/testscript
-->
Steps to reproduce:
```shell
testscript -v -continue <<EOF
```
```txtar
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work.
exec holos --version
exec holos init platform v1alpha5 --force
# remove the fix to trigger the bug
rm cue.mod/pkg/sigs.k8s.io/kustomize/api/types/var.cue
# want a BuildPlan shown
exec holos show buildplans
cmp stdout buildplan.yaml
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- buildplan.yaml --
kind: BuildPlan
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
```
```shell
EOF
```
### What did you expect to see?
The testscript should pass.
### What did you see instead?
The testscript fails because of the bug.
```txt
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work. (0.168s)
> exec holos --version
[stdout]
0.100.1-2-g9b10e23-dirty
> exec holos init platform v1alpha5 --force
# remove the fix to trigger the bug (0.000s)
> rm cue.mod/pkg/sigs.k8s.io/kustomize/api/types/var.cue
# want a BuildPlan shown (0.091s)
> exec holos show buildplans
[stderr]
could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
$WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2
[exit status 1]
FAIL: <stdin>:8: unexpected command failure
> cmp stdout buildplan.yaml
diff stdout buildplan.yaml
--- stdout
+++ buildplan.yaml
@@ -0,0 +1,1 @@
+kind: BuildPlan
FAIL: <stdin>:9: stdout and buildplan.yaml differ
# want this error to go away (0.000s)
> ! stderr 'cannot convert non-concrete value string'
FAIL: <stdin>:11: unexpected match for `cannot convert non-concrete value string` found in stderr: cannot convert non-concrete value string
failed run
```

View File

@@ -2,7 +2,7 @@ name: Dev Deploy
on:
push:
branches: ['main', 'dev-deploy']
branches: ['dev-deploy']
jobs:
deploy:

View File

@@ -1,6 +1,5 @@
---
# https://github.com/golangci/golangci-lint-action?tab=readme-ov-file#how-to-use
name: Lint
name: Spelling
"on":
push:
branches:
@@ -8,35 +7,11 @@ name: Lint
- test
pull_request:
types: [opened, synchronize]
permissions:
contents: read
jobs:
lint:
name: lint
cspell:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: stable
## Not needed on ubuntu-latest
# - name: Install Packages
# run: sudo apt update && sudo apt -qq -y install git curl zip unzip tar bzip2 make
- name: Install Tools
run: make tools
- name: Lint
# golangci-lint runs in a separate workflow.
run: make lint -o golangci-lint
- uses: actions/checkout@v4
- run: ./hack/cspell

View File

@@ -35,6 +35,9 @@ jobs:
with:
go-version: stable
- name: Setup Syft
uses: anchore/sbom-action/download-syft@1ca97d9028b51809cf6d3c934c3e160716e1b605 # v0.17.5
# Necessary to run these outside of goreleaser, otherwise
# /home/runner/_work/holos/holos/internal/frontend/node_modules/.bin/protoc-gen-connect-query is not in PATH
- name: Install Tools
@@ -54,11 +57,19 @@ jobs:
- name: Git diff
run: git diff
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
version: '~> v2'
args: release --clean
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -28,19 +28,11 @@ jobs:
with:
go-version: stable
- name: Install Packages
run: sudo apt update && sudo apt -qq -y install git curl zip unzip tar bzip2 make
- name: Set up Helm
uses: azure/setup-helm@v4
- name: Set up Kubectl
uses: azure/setup-kubectl@v4
- name: Install Tools
run: |
set -x
make tools
- name: Test
run: ./scripts/test

View File

@@ -6,7 +6,7 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
version: 2
before:
hooks:
@@ -50,3 +50,39 @@ changelog:
exclude:
- "^docs:"
- "^test:"
source:
enabled: true
name_template: '{{ .ProjectName }}_{{ .Version }}_source_code'
sboms:
- id: source
artifacts: source
documents:
- "{{ .ProjectName }}_{{ .Version }}_sbom.spdx.json"
brews:
- name: holos
repository:
owner: holos-run
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
homepage: "https://holos.run"
description: "Holos CLI"
dependencies:
- name: helm
type: optional
- name: kubectl
type: optional
install: |
bin.install "holos"
bash_output = Utils.safe_popen_read(bin/"holos", "completion", "bash")
(bash_completion/"holos").write bash_output
zsh_output = Utils.safe_popen_read(bin/"holos", "completion", "zsh")
(zsh_completion/"_holos").write zsh_output
fish_output = Utils.safe_popen_read(bin/"holos", "completion", "fish")
(fish_completion/"holos.fish").write fish_output
test: |
system "#{bin}/holos --version"

129
README.md
View File

@@ -1,35 +1,130 @@
## Holos - A Holistic Development Platform
# Holos
<img width="50%"
align="right"
style="display: block; margin: 40px auto;"
src="https://openinfrastructure.co/blog/2016/02/27/logo/logorectangle.png">
Building and maintaining a software development platform is a complex and time
consuming endeavour. Organizations often dedicate a team of 3-4 who need 6-12
months to build the platform.
[Holos] is a configuration management tool for Kubernetes implementing the
[rendered manifests pattern]. It handles configurations ranging from single
resources to multi-cluster platforms across regions.
Holos is a tool and a reference platform to reduce the complexity and speed up
the process of building a modern, cloud native software development platform.
Key components:
- Platform schemas defining component integration
- Building blocks unifying Helm, Kustomize and Kubernetes configs with CUE
- BuildPlan pipeline for generating, transforming and validating manifests
- **Accelerate new projects** - Reduce time to market and operational complexity by starting your new project on top of the Holos reference platform.
- **Modernize existing projects** - Incrementally incorporate your existing platform services into Holos for simpler integration.
- **Unified configuration model** - Increase safety and reduce the risk of config changes with CUE.
- **First class Helm and Kustomize support** - Leverage and reuse your existing investment in existing configuration tools such as Helm and Kustomize.
- **Modern Authentication and Authorization** - Holos seamlessly integrates platform identity and access management with zero-trust beyond corp style authorization policy.
```mermaid
---
title: Rendering Overview
---
graph LR
Platform[<a href="https://holos.run/docs/v1alpha5/api/author/#Platform">Platform</a>]
Component[<a href="https://holos.run/docs/v1alpha5/api/author/#ComponentConfig">Components</a>]
## Quick Installation
Helm[<a href="https://holos.run/docs/v1alpha5/api/author/#Helm">Helm</a>]
Kustomize[<a href="https://holos.run/docs/v1alpha5/api/author/#Kustomize">Kustomize</a>]
Kubernetes[<a href="https://holos.run/docs/v1alpha5/api/author/#Kubernetes">Kubernetes</a>]
```console
go install github.com/holos-run/holos/cmd/holos@latest
BuildPlan[<a href="https://holos.run/docs/v1alpha5/api/core/#BuildPlan">BuildPlan</a>]
ResourcesArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">GitOps<br/>Artifact</a>]
Generators[<a href="https://holos.run/docs/v1alpha5/api/core/#Generator">Generators</a>]
Transformers[<a href="https://holos.run/docs/v1alpha5/api/core/#Transformer">Transformers</a>]
Validators[<a href="https://holos.run/docs/v1alpha5/api/core/#Validator">Validators</a>]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Validators --> Files
```
## Docs and Support
## Setup
The documentation for developing and using Holos is available at: https://holos.run
```shell
brew install holos-run/tap/holos
```
For discussion and support, [open a discussion](https://github.com/orgs/holos-run/discussions/new/choose).
Refer to [setup] for other installation methods and dependencies.
## Example
See our [tutorial] for a complete hello world example.
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
## Organizational Role
Platform engineers use Holos to generate Kubernetes manifests, both locally and
in CI pipelines. The manifests are committed to version control and deployed via
GitOps tools like ArgoCD or Flux.
Holos integrates seamlessly with existing Helm charts, Kustomize bases, and
other version-controlled configurations.
## Advantages of Holos
### Safe
Holos leverages [CUE] for strong typing and validation of configuration data,
ensuring consistent output from Helm and other tools.
### Consistent
A unified pipeline processes all configurations - whether from CUE, Helm, or
Kustomize - through the same well-defined stages.
### Flexible
Composable building blocks for generation, transformation, validation and
integration let teams assemble workflows that match their needs.
The core is intentionally unopinionated about platform configuration patterns.
Common needs like environments and clusters are provided as customizable
[topics] recipes rather than enforced structures.
## Getting Help
Get support through our [Discord] channel or [GitHub discussions]. Configuration
challenges arise at all experience levels - we welcome your questions and are
here to help.
## License
Holos is licensed under Apache 2.0 as found in the [LICENSE file](LICENSE).
[Holos]: https://holos.run/docs/overview/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[CUE]: https://cuelang.org/
[Discord]: https://discord.gg/JgDVbNpye7
[GitHub discussions]: https://github.com/holos-run/holos/discussions
[Why CUE for Configuration]: https://holos.run/blog/why-cue-for-configuration/
[tutorial]: https://holos.run/docs/overview/
[setup]: https://holos.run/docs/setup/
[topics]: https://holos.run/docs/topics/

View File

@@ -0,0 +1,152 @@
// Package author contains a standard set of schemas for component authors to
// generate common [core] BuildPlans.
//
// Holos values stability, flexibility, and composition. This package
// intentionally defines only the minimal necessary set of structures.
// Component authors are encouraged to define their own structures building on
// our example [topics].
//
// The Holos Maintainers may add definitions to this package if the community
// identifies nearly all users must define the exact same structure. Otherwise,
// definitions should be added as a customizable example in [topics].
//
// For example, structures representing a cluster and environment almost always
// need to be defined. Their definition varies from one organization to the
// next. Therefore, customizable definitions for a cluster and environment are
// best maintained in [topics], not standardized in this package.
//
// [core]: https://holos.run/docs/api/core/
// [topics]: https://holos.run/docs/topics/
package author
import core "github.com/holos-run/holos/api/core/v1alpha5"
//go:generate ../../../hack/gendoc
// Platform assembles a core Platform in the Resource field for the holos render
// platform command. Use the Components field to register components with the
// platform.
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// ComponentConfig represents the configuration common to all kinds of
// components for use with the holos render component command. All component
// kinds may be transformed with [kustomize] configured with the
// [KustomizeConfig] field.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the kustomize configuration.
KustomizeConfig KustomizeConfig
// Validators represent checks that must pass for output to be written.
Validators map[NameLabel]core.Validator
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
// Helm assembles a BuildPlan rendering a helm chart. Useful to mix in
// additional resources from CUE and transform the helm output with kustomize.
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kubernetes assembles a BuildPlan containing inline resources exported from
// CUE.
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize assembles a BuildPlan rendering manifests from a [kustomize]
// kustomization.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// KustomizeConfig represents the configuration for [kustomize] post processing.
// Use the Files field to mix in plain manifest files located in the component
// directory. Use the Resources field to mix in manifests from network urls.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
// NameLabel represents the common use case of converting a struct to a list
// where the name field of each value unifies with the field name of the outer
// struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,5 @@
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---

View File

@@ -0,0 +1,5 @@
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---

349
api/core/v1alpha5/types.go Normal file
View File

@@ -0,0 +1,349 @@
// Package core contains schemas for a [Platform] and [BuildPlan]. Holos takes
// a [Platform] as input, then iterates over each [Component] to produce a
// [BuildPlan]. Holos processes the [BuildPlan] to produce fully rendered
// manifests, each an [Artifact].
package core
//go:generate ../../../hack/gendoc
// BuildPlan represents an implementation of the [rendered manifest pattern].
// Holos processes a BuildPlan to produce one or more [Artifact] output files.
// BuildPlan artifact files usually contain Kubernetes manifests, but they may
// have any content.
//
// A BuildPlan usually produces two artifacts. One artifact contains a manifest
// of resources. A second artifact contains a GitOps resource to manage the
// first, usually an ArgoCD Application resource.
//
// Holos uses CUE to construct a BuildPlan. A future enhancement will support
// user defined executables providing a BuildPlan to Holos in the style of an
// [external credential provider].
//
// [rendered manifest pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Validators []Validator `json:"validators,omitempty" yaml:"validators,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// Each Generator in an [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents Kubernetes resources. Most commonly used to mix
// resources into the [BuildPlan] generated from CUE, but may be generated from
// elsewhere.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source" yaml:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
//
// The Auth field is useful to configure http basic authentication to the Helm
// repository. Holos gets the username and password from the environment
// variables represented by the Auth field.
type Repository struct {
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
Auth Auth `json:"auth,omitempty" yaml:"auth,omitempty"`
}
// Auth represents environment variable names containing auth credentials.
type Auth struct {
Username AuthSource `json:"username" yaml:"username"`
Password AuthSource `json:"password" yaml:"password"`
}
// AuthSource represents a source for the value of an [Auth] field.
type AuthSource struct {
Value string `json:"value,omitempty" yaml:"value,omitempty"`
FromEnv string `json:"fromEnv,omitempty" yaml:"fromEnv,omitempty"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
// outputs into one output. [Kustomize] is the most commonly used transformer.
// A simple [Join] is also supported for use with plain manifest files.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// FileContent represents file contents.
type FileContent string
// Validator validates files. Useful to validate an [Artifact] prior to writing
// it out to the final destination. Holos may execute validators concurrently.
// See the [validators] tutorial for an end to end example.
//
// [validators]: https://holos.run/docs/v1alpha5/tutorial/validators/
type Validator struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Command\""`
// Inputs represents the files to validate. Usually the final Artifact.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Command represents a validation command. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Command represents a command vetting one or more artifacts. Holos appends
// fully qualified input file paths to the end of the args list, then executes
// the command. Inputs are written into a temporary directory prior to
// executing the command and removed afterwards.
type Command struct {
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
}
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
// collection and integrates the components together into a holistic platform.
// Holos iterates over the [Component] collection producing a [BuildPlan] for
// each, which holos then executes to render manifests.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components" yaml:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
// from a path containing parameterized CUE configuration.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path" yaml:"path"`
// Instances represents additional cue instance paths to unify with Path.
// Useful to unify data files into a component BuildPlan. Added in holos
// 0.101.7.
Instances []Instance `json:"instances,omitempty" yaml:"instances,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Instance represents a data instance to unify with the configuration.
//
// Useful to unify json and yaml files with cue configuration files for
// integration with other tools. For example, executing holos render platform
// from a pull request workflow after [Kargo] executes the [yaml update] and
// [git wait for pr] promotion steps.
//
// [Kargo]: https://docs.kargo.io/
// [yaml update]: https://docs.kargo.io/references/promotion-steps#yaml-update
// [git wait for pr]: https://docs.kargo.io/references/promotion-steps#git-wait-for-pr
type Instance struct {
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"ExtractYAML\""`
// Ignored unless kind is ExtractYAML.
ExtractYAML ExtractYAML `json:"extractYAML,omitempty" yaml:"extractYAML,omitempty"`
}
// ExtractYAML represents a cue data instance encoded as yaml or json. If Path
// refers to a directory all files in the directory are extracted
// non-recursively. Otherwise, path must refer to a file.
type ExtractYAML struct {
Path string `json:"path" yaml:"path"`
}

View File

@@ -1,56 +0,0 @@
package v1alpha1
import (
"errors"
"fmt"
"strings"
)
// BuildPlan is the primary interface between CUE and the Holos cli.
type BuildPlan struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
Spec BuildPlanSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
}
type BuildPlanSpec struct {
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
Components BuildPlanComponents `json:"components,omitempty" yaml:"components,omitempty"`
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
type BuildPlanComponents struct {
HelmChartList []HelmChart `json:"helmChartList,omitempty" yaml:"helmChartList,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty" yaml:"kubernetesObjectsList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty" yaml:"kustomizeBuildList,omitempty"`
Resources map[string]KubernetesObjects `json:"resources,omitempty" yaml:"resources,omitempty"`
}
func (bp *BuildPlan) Validate() error {
errs := make([]string, 0, 2)
if bp.Kind != BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", BuildPlanKind, bp.Kind))
}
if bp.APIVersion != APIVersion {
errs = append(errs, fmt.Sprintf("apiVersion invalid: want: %s have: %s", APIVersion, bp.APIVersion))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (bp *BuildPlan) ResultCapacity() (count int) {
if bp == nil {
return 0
}
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}

View File

@@ -1,30 +0,0 @@
package v1alpha1
// HolosComponent defines the fields common to all holos component kinds including the Render Result.
type HolosComponent struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
// APIObjectMap holds the marshalled representation of api objects. Think of
// these as resources overlaid at the back of the render pipeline.
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty" yaml:"apiObjectMap,omitempty"`
// Kustomization holds the marshalled representation of the flux kustomization
// which reconciles resources in git with the api server.
Kustomization `json:",inline" yaml:",inline"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:",inline" yaml:",inline"`
// Skip causes holos to take no action regarding the component.
Skip bool
}
func (hc *HolosComponent) NewResult() *Result {
return &Result{HolosComponent: *hc}
}
func (hc *HolosComponent) GetAPIVersion() string {
return hc.APIVersion
}
func (hc *HolosComponent) GetKind() string {
return hc.Kind
}

View File

@@ -1,11 +0,0 @@
package v1alpha1
const (
APIVersion = "holos.run/v1alpha1"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,2 +0,0 @@
// Package v1alpha1 defines the api boundary between CUE and Holos.
package v1alpha1

View File

@@ -1,13 +0,0 @@
package v1alpha1
import object "github.com/holos-run/holos/service/gen/holos/object/v1alpha1"
// Form represents a collection of Formly json powered form.
type Form struct {
TypeMeta `json:",inline" yaml:",inline"`
Spec FormSpec `json:"spec" yaml:"spec"`
}
type FormSpec struct {
Form object.Form `json:"form" yaml:"form"`
}

View File

@@ -1,184 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// A HelmChart represents a helm command to provide chart values in order to render kubernetes api objects.
type HelmChart struct {
HolosComponent `json:",inline" yaml:",inline"`
// Namespace is the namespace to install into. TODO: Use metadata.namespace instead.
Namespace string `json:"namespace"`
Chart Chart `json:"chart"`
ValuesContent string `json:"valuesContent"`
EnableHooks bool `json:"enableHooks"`
}
type Chart struct {
Name string `json:"name"`
Version string `json:"version"`
Release string `json:"release"`
Repository Repository `json:"repository,omitempty"`
}
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
func (hc *HelmChart) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: hc.HolosComponent}
if err := hc.helm(ctx, &result, path); err != nil {
return nil, err
}
result.addObjectMap(ctx, hc.APIObjectMap)
if err := result.kustomize(ctx); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not kustomize: %w", err))
}
return &result, nil
}
// runHelm provides the values produced by CUE to helm template and returns
// the rendered kubernetes api objects in the result.
func (hc *HelmChart) helm(ctx context.Context, r *Result, path holos.InstancePath) error {
log := logger.FromContext(ctx).With("chart", hc.Chart.Name)
if hc.Chart.Name == "" {
log.WarnContext(ctx, "skipping helm: no chart name specified, use a different component type")
return nil
}
cachedChartPath := filepath.Join(string(path), ChartDir, filepath.Base(hc.Chart.Name))
if isNotExist(cachedChartPath) {
// Add repositories
repo := hc.Chart.Repository
if repo.URL != "" {
out, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo add: %w", err))
}
// Update repository
out, err = util.RunCmd(ctx, "helm", "repo", "update", repo.Name)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo update: %w", err))
}
} else {
log.DebugContext(ctx, "no chart repository url proceeding assuming oci chart")
}
// Cache the chart
if err := cacheChart(ctx, path, ChartDir, hc.Chart); err != nil {
return fmt.Errorf("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos")
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, tempDir)
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, []byte(hc.ValuesContent), 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "helm: wrote values", "path", valuesPath, "bytes", len(hc.ValuesContent))
// Run charts
chart := hc.Chart
args := []string{"template"}
if !hc.EnableHooks {
args = append(args, "--no-hooks")
}
namespace := hc.Namespace
args = append(args, "--include-crds", "--values", valuesPath, "--namespace", namespace, "--kubeconfig", "/dev/null", "--version", chart.Version, chart.Release, cachedChartPath)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Wrap(fmt.Errorf("could not run helm template: %w", err))
}
r.accumulatedOutput = helmOut.Stdout.String()
return nil
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// It is assumed that the only method responsible for writing to chartDir is
// cacheChart itself.
//
// This relies on the atomicity of moving temporary directories into place on
// the same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is an
// expected scenario when this function is called concurrently.
func cacheChart(ctx context.Context, path holos.InstancePath, chartDir string, chart Chart) error {
log := logger.FromContext(ctx)
cacheTemp, err := os.MkdirTemp(string(path), chartDir)
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, cacheTemp)
chartName := chart.Name
if chart.Repository.Name != "" {
chartName = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, chartName)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
cachePath := filepath.Join(string(path), chartDir)
if err := os.MkdirAll(cachePath, 0777); err != nil {
return errors.Wrap(fmt.Errorf("could not mkdir: %w", err))
}
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
for _, item := range items {
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cachePath, item.Name())
log.DebugContext(ctx, "rename", "src", src, "dst", dst)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
}
}
log.InfoContext(ctx, "cached", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
return nil
}
func isNotExist(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}

View File

@@ -1,21 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents CUE output which directly provides Kubernetes api objects to holos.
type KubernetesObjects struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces kubernetes api objects from the APIObjectMap
func (o *KubernetesObjects) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: o.HolosComponent}
result.addObjectMap(ctx, o.APIObjectMap)
return &result, nil
}

View File

@@ -1,7 +0,0 @@
package v1alpha1
// Kustomization holds the rendered flux kustomization api object content for git ops.
type Kustomization struct {
// KsContent is the yaml representation of the flux kustomization for gitops.
KsContent string `json:"ksContent,omitempty" yaml:"ksContent,omitempty"`
}

View File

@@ -1,47 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
const KustomizeBuildKind = "KustomizeBuild"
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process raw yaml file resources in a holos component directory.
// 2. Post process a HelmChart to inject istio, add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty" yaml:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty" yaml:"resourcesFile,omitempty"`
}
// KustomizeBuild renders plain yaml files in the holos component directory using kubectl kustomize build.
type KustomizeBuild struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces a Result by executing kubectl kustomize on the holos
// component path. Useful for processing raw yaml files.
func (kb *KustomizeBuild) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
log := logger.FromContext(ctx)
result := Result{HolosComponent: kb.HolosComponent}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", string(path))
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return nil, errors.Wrap(err)
}
// Replace the accumulated output
result.accumulatedOutput = kOut.Stdout.String()
// Add CUE based api objects.
result.addObjectMap(ctx, kb.APIObjectMap)
return &result, nil
}

View File

@@ -1,14 +0,0 @@
package v1alpha1
// Label is an arbitrary unique identifier. Defined as a type for clarity and type checking.
type Label string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
type Kind string
// APIObjectMap is the shape of marshalled api objects returned from cue to the
// holos cli. A map is used to improve the clarity of error messages from cue.
type APIObjectMap map[Kind]map[Label]string
// FileContentMap is a map of file names to file contents.
type FileContentMap map[string]string

View File

@@ -1,15 +0,0 @@
package v1alpha1
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are by holos objects distinct
// from kubernetes api objects.
type ObjectMeta struct {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
Name string `json:"name,omitempty" yaml:"name,omitempty"`
// Namespace confines a holos component to a single namespace via kustomize if set.
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Labels are not used but are copied from api machinery ObjectMeta for completeness.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations are not used but are copied from api machinery ObjectMeta for completeness.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,32 +0,0 @@
package v1alpha1
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
TypeMeta `json:",inline" yaml:",inline"`
Metadata ObjectMeta `json:"metadata" yaml:"metadata"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform build plan specification.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// holos.platform.v1alpha1.PlatformService.GetPlatform method and provides to
// CUE using a tag.
Model structpb.Struct `json:"model" yaml:"model"`
Components []PlatformSpecComponent `json:"components" yaml:"components"`
}
// PlatformSpecComponent represents a component to build or render with flags to
// pass, for example the cluster name.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path" yaml:"path"`
// Cluster is the cluster name to use when building the component.
Cluster string `json:"cluster" yaml:"cluster"`
}

View File

@@ -1,22 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
type Renderer interface {
GetKind() string
Render(ctx context.Context, path holos.InstancePath) (*Result, error)
}
// Render produces a Result representing the kubernetes api objects to
// configure. Each of the various holos component types, e.g. Helm, Kustomize,
// et al, should implement the Renderer interface. This process is best
// conceptualized as a data pipeline, for example a component may render a
// result by first calling helm template, then passing the result through
// kustomize, then mixing in overlay api objects.
func Render(ctx context.Context, r Renderer, path holos.InstancePath) (*Result, error) {
return r.Render(ctx, path)
}

View File

@@ -1,165 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"slices"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// Result is the build result for display or writing. Holos components Render the Result as a data pipeline.
type Result struct {
HolosComponent
// accumulatedOutput accumulates rendered api objects.
accumulatedOutput string
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
// Continue returns true if Skip is true indicating the result is to be skipped over.
func (r *Result) Continue() bool {
if r == nil {
return false
}
return r.Skip
}
func (r *Result) Name() string {
return r.Metadata.Name
}
func (r *Result) Filename(writeTo string, cluster string) string {
name := r.Metadata.Name
return filepath.Join(writeTo, "clusters", cluster, "components", name, name+".gen.yaml")
}
func (r *Result) KustomizationFilename(writeTo string, cluster string) string {
return filepath.Join(writeTo, "clusters", cluster, "holos", "components", r.Metadata.Name+"-kustomization.gen.yaml")
}
// AccumulatedOutput returns the accumulated rendered output.
func (r *Result) AccumulatedOutput() string {
return r.accumulatedOutput
}
// addObjectMap renders the provided APIObjectMap into the accumulated output.
func (r *Result) addObjectMap(ctx context.Context, objectMap APIObjectMap) {
log := logger.FromContext(ctx)
b := []byte(r.AccumulatedOutput())
kinds := make([]Kind, 0, len(objectMap))
// Sort the keys
for kind := range objectMap {
kinds = append(kinds, kind)
}
slices.Sort(kinds)
for _, kind := range kinds {
v := objectMap[kind]
// Sort the keys
names := make([]Label, 0, len(v))
for name := range v {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
yamlString := v[name]
log.Debug(fmt.Sprintf("%s/%s", kind, name), "kind", kind, "name", name)
b = util.EnsureNewline(b)
header := fmt.Sprintf("---\n# Source: CUE apiObjects.%s.%s\n", kind, name)
b = append(b, []byte(header+yamlString)...)
b = util.EnsureNewline(b)
}
}
r.accumulatedOutput = string(b)
}
// kustomize replaces the accumulated output with the output of kustomize build
func (r *Result) kustomize(ctx context.Context) error {
log := logger.FromContext(ctx)
if r.ResourcesFile == "" {
log.DebugContext(ctx, "skipping kustomize: no resourcesFile")
return nil
}
if len(r.KustomizeFiles) < 1 {
log.DebugContext(ctx, "skipping kustomize: no kustomizeFiles")
return nil
}
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
// Write the main api object resources file for kustomize.
target := filepath.Join(tempDir, r.ResourcesFile)
b := []byte(r.AccumulatedOutput())
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write resources: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
// Write the kustomization tree, kustomization.yaml must be in this map for kustomize to work.
for file, content := range r.KustomizeFiles {
target := filepath.Join(tempDir, file)
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return errors.Wrap(err)
}
b := []byte(content)
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return errors.Wrap(err)
}
// Replace the accumulated output
r.accumulatedOutput = kOut.Stdout.String()
return nil
}
func (r *Result) WriteDeployFiles(ctx context.Context, path string) error {
log := logger.FromContext(ctx)
if len(r.DeployFiles) == 0 {
return nil
}
for k, content := range r.DeployFiles {
path := filepath.Join(path, k)
if err := r.Save(ctx, path, content); err != nil {
return errors.Wrap(err)
}
log.InfoContext(ctx, "wrote deploy file", "path", path, "bytes", len(content))
}
return nil
}
// Save writes the content to the filesystem for git ops.
func (r *Result) Save(ctx context.Context, path string, content string) error {
log := logger.FromContext(ctx)
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, os.FileMode(0775)); err != nil {
log.WarnContext(ctx, "could not mkdir", "path", dir, "err", err)
return errors.Wrap(err)
}
// Write the file content
if err := os.WriteFile(path, []byte(content), os.FileMode(0644)); err != nil {
log.WarnContext(ctx, "could not write", "path", path, "err", err)
return errors.Wrap(err)
}
log.DebugContext(ctx, "out: wrote "+path, "action", "write", "path", path, "status", "ok")
return nil
}

View File

@@ -1,20 +0,0 @@
package v1alpha1
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
func (tm *TypeMeta) GetKind() string {
return tm.Kind
}
func (tm *TypeMeta) GetAPIVersion() string {
return tm.APIVersion
}
// Discriminator is an interface to discriminate the kind api object.
type Discriminator interface {
GetKind() string
GetAPIVersion() string
}

63
cmd/cmd.go Normal file
View File

@@ -0,0 +1,63 @@
package cmd
import (
"context"
"fmt"
"log/slog"
"os"
"runtime/pprof"
"runtime/trace"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/internal/holos"
)
// MakeMain makes a main function for the cli or tests.
func MakeMain(options ...holos.Option) func() int {
return func() (exitCode int) {
cfg := holos.New(options...)
slog.SetDefault(cfg.Logger())
ctx := context.Background()
if format := os.Getenv("HOLOS_CPU_PROFILE"); format != "" {
f, _ := os.Create(fmt.Sprintf(format, os.Getppid(), os.Getpid()))
err := pprof.StartCPUProfile(f)
defer func() {
pprof.StopCPUProfile()
f.Close()
}()
if err != nil {
return cli.HandleError(ctx, err, cfg)
}
}
defer memProfile(ctx, cfg)
if format := os.Getenv("HOLOS_TRACE"); format != "" {
f, _ := os.Create(fmt.Sprintf(format, os.Getppid(), os.Getpid()))
err := trace.Start(f)
defer func() {
trace.Stop()
f.Close()
}()
if err != nil {
return cli.HandleError(ctx, err, cfg)
}
}
feature := &holos.EnvFlagger{}
if err := cli.New(cfg, feature).ExecuteContext(ctx); err != nil {
return cli.HandleError(ctx, err, cfg)
}
return 0
}
}
func memProfile(ctx context.Context, cfg *holos.Config) {
if format := os.Getenv("HOLOS_MEM_PROFILE"); format != "" {
f, _ := os.Create(fmt.Sprintf(format, os.Getppid(), os.Getpid()))
defer f.Close()
if err := pprof.WriteHeapProfile(f); err != nil {
_ = cli.HandleError(ctx, err, cfg)
}
}
}

View File

@@ -3,9 +3,9 @@ package main
import (
"os"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/cmd"
)
func main() {
os.Exit(cli.MakeMain()())
os.Exit(cmd.MakeMain()())
}

View File

@@ -6,19 +6,27 @@ import (
"testing"
cue "cuelang.org/go/cmd/cue/cmd"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/cmd"
"github.com/rogpeppe/go-internal/testscript"
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cli.MakeMain(),
"holos": cmd.MakeMain(),
"cue": cue.Main,
}))
}
func TestGuides(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha4", "guides")))
func TestGuides_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "guides")))
}
func TestSchemas_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "schemas")))
}
func TestIssues_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "issues")))
}
func TestCLI(t *testing.T) {
@@ -29,7 +37,9 @@ func params(dir string) testscript.Params {
return testscript.Params{
Dir: filepath.Join("tests", dir),
RequireExplicitExec: true,
RequireUniqueNames: true,
RequireUniqueNames: os.Getenv("HOLOS_WORKDIR_ROOT") == "",
WorkdirRoot: os.Getenv("HOLOS_WORKDIR_ROOT"),
UpdateScripts: os.Getenv("HOLOS_UPDATE_SCRIPTS") != "",
Setup: func(env *testscript.Env) error {
// Just like cmd/cue/cmd.TestScript, set up separate cache and config dirs per test.
env.Setenv("CUE_CACHE_DIR", filepath.Join(env.WorkDir, "tmp/cachedir"))

View File

@@ -0,0 +1,12 @@
# https://github.com/holos-run/holos/issues/358
# holos cue vet should fail verifications with exit code 1
! exec holos cue vet ./policy --path strings.ToLower(kind) ./data/secret.yaml
# holos cue vet should report validation errors to stderr
stderr 'Forbidden. Use an ExternalSecret instead.'
-- data/secret.yaml --
kind: Secret
-- policy/validators.cue --
package policy
secret: kind: "Forbidden. Use an ExternalSecret instead."

View File

@@ -0,0 +1,2 @@
# https://github.com/holos-run/holos/issues/334
exec holos

View File

@@ -4,14 +4,14 @@
cd $WORK
# Generate the directory structure we're going to work in.
exec holos generate platform v1alpha4
exec holos generate platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform ./platform
exec holos render platform
stderr -count=1 '^rendered platform'
# Holos uses CUE to build a platform specification.
exec cue export --out=yaml ./platform
exec holos show platform
cmp stdout want/1.platform_spec.yaml
# Define the host and port in projects/blackbox.schema.cue
@@ -22,10 +22,10 @@ mv projects/platform/components/prometheus/prometheus.cue.disabled projects/plat
mv platform/prometheus.cue.disabled platform/prometheus.cue
# Render the platform to render the prometheus chart.
exec holos render platform ./platform
stderr -count=1 '^rendered prometheus for cluster local'
exec holos render platform
stderr -count=1 '^rendered prometheus'
stderr -count=1 '^rendered platform'
cmp deploy/clusters/local/components/prometheus/prometheus.gen.yaml want/1.prometheus.gen.yaml
cmp deploy/components/prometheus/prometheus.gen.yaml want/1.prometheus.gen.yaml
# Add the CUE configuration to manage the blackbox Helm Chart component.
mv projects/platform/components/blackbox/blackbox.cue.disabled projects/platform/components/blackbox/blackbox.cue
@@ -34,14 +34,14 @@ mv platform/blackbox.cue.disabled platform/blackbox.cue
# Render the platform to render both the prometheus and blackbox charts.
exec holos render platform ./platform
stderr -count=1 '^rendered prometheus for cluster local'
stderr -count=1 '^rendered blackbox for cluster local'
stderr -count=1 '^rendered prometheus'
stderr -count=1 '^rendered blackbox'
stderr -count=1 '^rendered platform'
cmp deploy/clusters/local/components/blackbox/blackbox.gen.yaml want/1.blackbox.gen.yaml
cmp deploy/components/blackbox/blackbox.gen.yaml want/1.blackbox.gen.yaml
# Import helm values
exec cue import --package holos --path '_Helm: Values:' --outfile projects/platform/components/prometheus/values.cue projects/platform/components/prometheus/vendor/25.27.0/prometheus/values.yaml
exec cue import --package holos --path '_Helm: Values:' --outfile projects/platform/components/blackbox/values.cue projects/platform/components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
exec cue import --package holos --path 'Helm: Values:' --outfile projects/platform/components/prometheus/values.cue projects/platform/components/prometheus/vendor/25.27.0/prometheus/values.yaml
exec cue import --package holos --path 'Helm: Values:' --outfile projects/platform/components/blackbox/values.cue projects/platform/components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
# Patch imported values
stdin values.patch
exec patch -p1
@@ -54,12 +54,27 @@ mv platform/httpbin.cue.disabled platform/httpbin.cue
# Render the platform with httpbin
exec holos render platform ./platform
stderr -count=1 '^rendered httpbin for cluster'
cmp deploy/clusters/local/components/httpbin/httpbin.gen.yaml want/1.httpbin.gen.yaml
stderr -count=1 '^rendered httpbin'
cmp deploy/components/httpbin/httpbin.gen.yaml want/1.httpbin.gen.yaml
# Verify the labels are correct
grep 'replacement: blackbox:9115' deploy/components/prometheus/prometheus.gen.yaml
# Check the blackbox fullnameOverride
grep -count=4 '^ name: blackbox$' deploy/components/blackbox/blackbox.gen.yaml
# Check the blackbox Service port
grep -count=1 '^ port: 9115$' deploy/components/blackbox/blackbox.gen.yaml
-- projects/output.cue --
package holos
import "github.com/holos-run/holos/api/core/v1alpha5:core"
core.#BuildPlan & {
metadata: name: _Tags.component.name
}
-- want/1.platform_spec.yaml --
apiVersion: v1alpha5
kind: Platform
apiVersion: v1alpha4
metadata:
name: default
spec:
@@ -74,9 +89,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: alertmanager
app.kubernetes.io/version: v0.27.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: alertmanager-1.12.0
holos.run/component.name: prometheus
name: prometheus-alertmanager
namespace: default
---
@@ -91,9 +104,7 @@ metadata:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/part-of: kube-state-metrics
app.kubernetes.io/version: 2.13.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: kube-state-metrics-5.25.1
holos.run/component.name: prometheus
name: prometheus-kube-state-metrics
namespace: default
---
@@ -108,9 +119,7 @@ metadata:
app.kubernetes.io/name: prometheus-node-exporter
app.kubernetes.io/part-of: prometheus-node-exporter
app.kubernetes.io/version: 1.8.2
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-node-exporter-4.39.0
holos.run/component.name: prometheus
name: prometheus-prometheus-node-exporter
namespace: default
---
@@ -123,9 +132,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-pushgateway
app.kubernetes.io/version: v1.9.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-pushgateway-2.14.0
holos.run/component.name: prometheus
name: prometheus-prometheus-pushgateway
namespace: default
---
@@ -139,9 +146,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
namespace: default
---
@@ -155,9 +160,7 @@ metadata:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/part-of: kube-state-metrics
app.kubernetes.io/version: 2.13.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: kube-state-metrics-5.25.1
holos.run/component.name: prometheus
name: prometheus-kube-state-metrics
rules:
- apiGroups:
@@ -371,9 +374,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
rules:
- apiGroups:
@@ -424,9 +425,7 @@ metadata:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/part-of: kube-state-metrics
app.kubernetes.io/version: 2.13.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: kube-state-metrics-5.25.1
holos.run/component.name: prometheus
name: prometheus-kube-state-metrics
roleRef:
apiGroup: rbac.authorization.k8s.io
@@ -447,9 +446,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
roleRef:
apiGroup: rbac.authorization.k8s.io
@@ -480,9 +477,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: alertmanager
app.kubernetes.io/version: v0.27.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: alertmanager-1.12.0
holos.run/component.name: prometheus
name: prometheus-alertmanager
namespace: default
---
@@ -836,9 +831,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
namespace: default
---
@@ -850,9 +843,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: alertmanager
app.kubernetes.io/version: v0.27.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: alertmanager-1.12.0
holos.run/component.name: prometheus
name: prometheus-alertmanager
namespace: default
spec:
@@ -864,8 +855,6 @@ spec:
selector:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: alertmanager
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
type: ClusterIP
---
apiVersion: v1
@@ -876,9 +865,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: alertmanager
app.kubernetes.io/version: v0.27.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: alertmanager-1.12.0
holos.run/component.name: prometheus
name: prometheus-alertmanager-headless
namespace: default
spec:
@@ -891,8 +878,6 @@ spec:
selector:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: alertmanager
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
---
apiVersion: v1
kind: Service
@@ -906,9 +891,7 @@ metadata:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/part-of: kube-state-metrics
app.kubernetes.io/version: 2.13.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: kube-state-metrics-5.25.1
holos.run/component.name: prometheus
name: prometheus-kube-state-metrics
namespace: default
spec:
@@ -920,8 +903,6 @@ spec:
selector:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: kube-state-metrics
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
type: ClusterIP
---
apiVersion: v1
@@ -936,9 +917,7 @@ metadata:
app.kubernetes.io/name: prometheus-node-exporter
app.kubernetes.io/part-of: prometheus-node-exporter
app.kubernetes.io/version: 1.8.2
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-node-exporter-4.39.0
holos.run/component.name: prometheus
name: prometheus-prometheus-node-exporter
namespace: default
spec:
@@ -950,8 +929,6 @@ spec:
selector:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: prometheus-node-exporter
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
type: ClusterIP
---
apiVersion: v1
@@ -964,9 +941,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-pushgateway
app.kubernetes.io/version: v1.9.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-pushgateway-2.14.0
holos.run/component.name: prometheus
name: prometheus-prometheus-pushgateway
namespace: default
spec:
@@ -978,8 +953,6 @@ spec:
selector:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: prometheus-pushgateway
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
type: ClusterIP
---
apiVersion: v1
@@ -992,9 +965,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
namespace: default
spec:
@@ -1007,8 +978,6 @@ spec:
app.kubernetes.io/component: server
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: prometheus
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
sessionAffinity: None
type: ClusterIP
---
@@ -1022,9 +991,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
namespace: default
spec:
@@ -1044,9 +1011,7 @@ metadata:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/part-of: kube-state-metrics
app.kubernetes.io/version: 2.13.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: kube-state-metrics-5.25.1
holos.run/component.name: prometheus
name: prometheus-kube-state-metrics
namespace: default
spec:
@@ -1056,8 +1021,6 @@ spec:
matchLabels:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: kube-state-metrics
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
strategy:
type: RollingUpdate
template:
@@ -1069,9 +1032,7 @@ spec:
app.kubernetes.io/name: kube-state-metrics
app.kubernetes.io/part-of: kube-state-metrics
app.kubernetes.io/version: 2.13.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: kube-state-metrics-5.25.1
holos.run/component.name: prometheus
spec:
automountServiceAccountToken: true
containers:
@@ -1131,9 +1092,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-pushgateway
app.kubernetes.io/version: v1.9.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-pushgateway-2.14.0
holos.run/component.name: prometheus
name: prometheus-prometheus-pushgateway
namespace: default
spec:
@@ -1142,8 +1101,6 @@ spec:
matchLabels:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: prometheus-pushgateway
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
strategy:
type: Recreate
template:
@@ -1153,9 +1110,7 @@ spec:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-pushgateway
app.kubernetes.io/version: v1.9.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-pushgateway-2.14.0
holos.run/component.name: prometheus
spec:
automountServiceAccountToken: true
containers:
@@ -1201,9 +1156,7 @@ metadata:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
name: prometheus-server
namespace: default
spec:
@@ -1214,8 +1167,6 @@ spec:
app.kubernetes.io/component: server
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: prometheus
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
strategy:
rollingUpdate: null
type: Recreate
@@ -1228,9 +1179,7 @@ spec:
app.kubernetes.io/name: prometheus
app.kubernetes.io/part-of: prometheus
app.kubernetes.io/version: v2.54.1
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-25.27.0
holos.run/component.name: prometheus
spec:
containers:
- args:
@@ -1323,9 +1272,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: alertmanager
app.kubernetes.io/version: v0.27.0
argocd.argoproj.io/instance: prometheus
helm.sh/chart: alertmanager-1.12.0
holos.run/component.name: prometheus
name: prometheus-alertmanager
namespace: default
spec:
@@ -1336,8 +1283,6 @@ spec:
matchLabels:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: alertmanager
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
serviceName: prometheus-alertmanager-headless
template:
metadata:
@@ -1346,8 +1291,6 @@ spec:
labels:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: alertmanager
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
spec:
automountServiceAccountToken: true
containers:
@@ -1397,9 +1340,6 @@ spec:
name: config
volumeClaimTemplates:
- metadata:
labels:
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
name: storage
spec:
accessModes:
@@ -1418,9 +1358,7 @@ metadata:
app.kubernetes.io/name: prometheus-node-exporter
app.kubernetes.io/part-of: prometheus-node-exporter
app.kubernetes.io/version: 1.8.2
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-node-exporter-4.39.0
holos.run/component.name: prometheus
name: prometheus-prometheus-node-exporter
namespace: default
spec:
@@ -1429,8 +1367,6 @@ spec:
matchLabels:
app.kubernetes.io/instance: prometheus
app.kubernetes.io/name: prometheus-node-exporter
argocd.argoproj.io/instance: prometheus
holos.run/component.name: prometheus
template:
metadata:
annotations:
@@ -1442,9 +1378,7 @@ spec:
app.kubernetes.io/name: prometheus-node-exporter
app.kubernetes.io/part-of: prometheus-node-exporter
app.kubernetes.io/version: 1.8.2
argocd.argoproj.io/instance: prometheus
helm.sh/chart: prometheus-node-exporter-4.39.0
holos.run/component.name: prometheus
spec:
automountServiceAccountToken: false
containers:
@@ -1536,8 +1470,6 @@ metadata:
prometheus.io/probe: "true"
labels:
app.kubernetes.io/name: httpbin
argocd.argoproj.io/instance: httpbin
holos.run/component.name: httpbin
name: httpbin
spec:
ports:
@@ -1548,29 +1480,21 @@ spec:
targetPort: http
selector:
app.kubernetes.io/name: httpbin
argocd.argoproj.io/instance: httpbin
holos.run/component.name: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/name: httpbin
argocd.argoproj.io/instance: httpbin
holos.run/component.name: httpbin
name: httpbin
spec:
selector:
matchLabels:
app.kubernetes.io/name: httpbin
argocd.argoproj.io/instance: httpbin
holos.run/component.name: httpbin
template:
metadata:
labels:
app.kubernetes.io/name: httpbin
argocd.argoproj.io/instance: httpbin
holos.run/component.name: httpbin
spec:
containers:
- image: mccutchen/go-httpbin
@@ -1597,9 +1521,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
argocd.argoproj.io/instance: blackbox
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
holos.run/component.name: blackbox
name: prometheus-blackbox-exporter
namespace: default
---
@@ -1623,9 +1545,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
argocd.argoproj.io/instance: blackbox
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
holos.run/component.name: blackbox
name: prometheus-blackbox-exporter
namespace: default
---
@@ -1637,9 +1557,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
argocd.argoproj.io/instance: blackbox
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
holos.run/component.name: blackbox
name: prometheus-blackbox-exporter
namespace: default
spec:
@@ -1651,8 +1569,6 @@ spec:
selector:
app.kubernetes.io/instance: prometheus-blackbox-exporter
app.kubernetes.io/name: prometheus-blackbox-exporter
argocd.argoproj.io/instance: blackbox
holos.run/component.name: blackbox
type: ClusterIP
---
apiVersion: apps/v1
@@ -1663,9 +1579,7 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
argocd.argoproj.io/instance: blackbox
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
holos.run/component.name: blackbox
name: prometheus-blackbox-exporter
namespace: default
spec:
@@ -1674,8 +1588,6 @@ spec:
matchLabels:
app.kubernetes.io/instance: prometheus-blackbox-exporter
app.kubernetes.io/name: prometheus-blackbox-exporter
argocd.argoproj.io/instance: blackbox
holos.run/component.name: blackbox
strategy:
rollingUpdate:
maxSurge: 1
@@ -1688,8 +1600,6 @@ spec:
labels:
app.kubernetes.io/instance: prometheus-blackbox-exporter
app.kubernetes.io/name: prometheus-blackbox-exporter
argocd.argoproj.io/instance: blackbox
holos.run/component.name: blackbox
spec:
automountServiceAccountToken: false
containers:
@@ -1730,61 +1640,60 @@ spec:
name: prometheus-blackbox-exporter
name: config
-- values.patch --
diff --git a/projects/platform/components/blackbox/values.cue b/projects/platform/components/blackbox/values.cue
index 6e29043..e3fa9ce 100644
--- a/projects/platform/components/blackbox/values.cue
+++ b/projects/platform/components/blackbox/values.cue
@@ -2,6 +2,9 @@ package holos
@@ -1,6 +1,8 @@
package holos
_Helm: {
Values: {
+ // the upstream chart authors forgot to add the fullnameOverride value used in the service template.
+ fullnameOverride: _blackbox.host
Helm: Values: {
+ fullnameOverride: _blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -193,7 +196,7 @@ _Helm: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: _blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +194,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: _blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
diff --git a/projects/platform/components/prometheus/values.cue b/projects/platform/components/prometheus/values.cue
index 869b11d..119c608 100644
--- a/projects/platform/components/prometheus/values.cue
+++ b/projects/platform/components/prometheus/values.cue
@@ -1084,7 +1084,7 @@ _Helm: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(_blackbox.host):\(_blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
@@ -1083,7 +1083,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(_blackbox.host):\(_blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
-- platform/httpbin.cue.disabled --
package holos
_Platform: Components: httpbin: {
name: "httpbin"
component: "projects/platform/components/httpbin"
cluster: "local"
Platform: Components: httpbin: {
name: "httpbin"
path: "projects/platform/components/httpbin"
}
-- platform/blackbox.cue.disabled --
package holos
_Platform: Components: blackbox: {
name: "blackbox"
component: "projects/platform/components/blackbox"
cluster: "local"
Platform: Components: blackbox: {
name: "blackbox"
path: "projects/platform/components/blackbox"
}
-- projects/platform/components/blackbox/blackbox.cue.disabled --
package holos
// Produce a helm chart build plan.
_Helm.BuildPlan
_Helm: #Helm & {
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
@@ -1794,6 +1703,9 @@ _Helm: #Helm & {
}
}
}
// Produce a helm chart build plan.
holos: Helm.BuildPlan
-- projects/blackbox.schema.cue.disabled --
package holos
@@ -1813,10 +1725,7 @@ _blackbox: #blackbox & {
-- projects/platform/components/prometheus/prometheus.cue.disabled --
package holos
// Produce a helm chart build plan.
_Helm.BuildPlan
_Helm: #Helm & {
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
@@ -1826,13 +1735,15 @@ _Helm: #Helm & {
}
}
}
// Produce a helm chart build plan.
holos: Helm.BuildPlan
-- platform/prometheus.cue.disabled --
package holos
_Platform: Components: prometheus: {
name: "prometheus"
component: "projects/platform/components/prometheus"
cluster: "local"
Platform: Components: prometheus: {
name: "prometheus"
path: "projects/platform/components/prometheus"
}
-- projects/platform/components/prometheus/vendor/25.27.0/prometheus/.helmignore --
# Patterns to ignore when building packages.
@@ -14983,11 +14894,8 @@ package holos
import "encoding/yaml"
// Produce a Kustomize BuildPlan for Holos
_Kustomize.BuildPlan
// https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/README.md
_Kustomize: #Kustomize & {
Kustomize: #Kustomize & {
KustomizeConfig: Files: "resources.yaml": _
KustomizeConfig: Kustomization: {
commonLabels: "app.kubernetes.io/name": "httpbin"
@@ -15004,6 +14912,9 @@ _Kustomize: #Kustomize & {
patches: [for x in _patches {x}]
}
}
// Produce a Kustomize BuildPlan for Holos
holos: Kustomize.BuildPlan
-- projects/platform/components/httpbin/resources.yaml --
apiVersion: apps/v1
kind: Deployment

View File

@@ -0,0 +1,422 @@
# https://github.com/holos-run/holos/issues/331
# ensure holos show components --labels selects correctly.
# ensure BuildPlan includes labels and annotations from the platform component.
# ensure holos render platform injects the holos_component_labels and
# holos_component_annotations tags.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cmp stdout want/platform.yaml
# all buildplans are selected by default
exec holos show buildplans
cmp stdout want/all-buildplans.yaml
# one = works in the selector
exec holos show buildplans --selector app.holos.run/name=empty1-label
cmp stdout want/buildplans.1.yaml
# double == works in the selector
exec holos show buildplans --selector app.holos.run/name==empty2-label
cmp stdout want/buildplans.2.yaml
# not equal != negates the selection
exec holos show buildplans --selector app.holos.run/name!=empty3-label
cmp stdout want/buildplans.3.yaml
exec holos show buildplans --selector app.holos.run/name!=something-else
cmp stdout want/buildplans.4.yaml
-- platform/empty.cue --
package holos
Platform: Components: {
empty1: _
empty2: _
empty3: _
empty4: _
}
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": "\(name)-label"
annotations: "app.holos.run/description": "\(name)-annotation empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/platform.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components:
- annotations:
app.holos.run/description: empty1-annotation empty test case
labels:
app.holos.run/name: empty1-label
name: empty1
path: components/empty
- annotations:
app.holos.run/description: empty2-annotation empty test case
labels:
app.holos.run/name: empty2-label
name: empty2
path: components/empty
- annotations:
app.holos.run/description: empty3-annotation empty test case
labels:
app.holos.run/name: empty3-label
name: empty3
path: components/empty
- annotations:
app.holos.run/description: empty4-annotation empty test case
labels:
app.holos.run/name: empty4-label
name: empty4
path: components/empty
-- want/empty.yaml --
-- want/all-buildplans.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.1.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.2.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.3.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.4.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,64 @@
# https://github.com/holos-run/holos/issues/348
# when the optional kustomize patch name field is omitted
exec holos init platform v1alpha5 --force
# want a buildplan shown
exec holos show buildplans
cmp stdout buildplan.yaml
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
-- buildplan.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: example
spec:
artifacts:
- artifact: components/example/example.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/example/example.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
patches:
- patch: |
- op: add
path: /metadata/annotations/example
value: example-value
target:
kind: CustomResourceDefinition
name: ""
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,50 @@
# https://github.com/holos-run/holos/issues/366
# Build tags conditionally include CUE files.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cmp stdout want/empty.yaml
exec holos show platform -t foo
cmp stdout want/foo.yaml
-- platform/empty.cue --
@if(foo)
package holos
Platform: Components: foo: _
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": NAME
annotations: "app.holos.run/description": "\(NAME) empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/empty.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components: []
-- want/foo.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components:
- annotations:
app.holos.run/description: foo empty test case
labels:
app.holos.run/name: foo
name: foo
path: components/empty

View File

@@ -0,0 +1,144 @@
# https://github.com/holos-run/holos/issues/330
exec holos init platform v1alpha5 --force
exec helm template ./components/capabilities/vendor/0.1.0/capabilities
cmp stdout want/helm-template.yaml
exec holos render platform ./platform
# When no capabilities are specified
cmp deploy/components/capabilities/capabilities.gen.yaml want/when-no-capabilities-specified.yaml
# With APIVersions specified
cmp deploy/components/specified/specified.gen.yaml want/with-capabilities-specified.yaml
# With KubeVersion specified
cmp deploy/components/kubeversion1/kubeversion1.gen.yaml want/with-kubeversion-specified.yaml
# With both APIVersions and KubeVersion specified
cmp deploy/components/kubeversion2/kubeversion2.gen.yaml want/with-both-specified.yaml
-- want/with-both-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-kubeversion-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/when-no-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- platform/capabilities.cue --
package holos
import "encoding/json"
Platform: Components: capabilities: {
name: "capabilities"
path: "components/capabilities"
}
Platform: Components: specified: {
name: "specified"
path: "components/capabilities"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
Platform: Components: kubeversion1: {
name: "kubeversion1"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
}
Platform: Components: kubeversion2: {
name: "kubeversion2"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
-- components/capabilities/capabilities.cue --
package holos
import "encoding/json"
holos: Component.BuildPlan
Component: #Helm & {
Name: string @tag(holos_component_name, type=string)
Chart: name: "capabilities"
Chart: version: "0.1.0"
_APIVersions: string | *"[]" @tag(apiVersions, type=string)
APIVersions: json.Unmarshal(_APIVersions)
KubeVersion: string | *"v1.31.0" @tag(kubeVersion, type=string)
}
-- components/capabilities/vendor/0.1.0/capabilities/Chart.yaml --
apiVersion: v2
name: capabilities
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: "1.16.0"
-- components/capabilities/vendor/0.1.0/capabilities/templates/service.yaml --
apiVersion: v1
kind: Service
metadata:
{{- if .Capabilities.APIVersions.Has "foo/v1" }}
name: has-foo-v1
{{- else }}
name: has-foo-v1beta1
{{- end }}
annotations:
kubeVersion: {{ .Capabilities.KubeVersion }}
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
-- want/helm-template.yaml --
---
# Source: capabilities/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: has-foo-v1beta1
annotations:
kubeVersion: v1.31.0
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http

View File

@@ -0,0 +1,45 @@
# author.#Kubernetes
# Start in an empty directory.
cd $WORK
# Generate the directory structure we're going to work in.
exec holos init platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform
stderr -count=1 '^rendered platform'
# When author.#Kubernetes is empty
exec holos cue export --expression holos --out=yaml ./components/empty
cmp stdout want.txt
-- components/empty/empty.cue --
package holos
Kubernetes: #Kubernetes & {}
holos: Kubernetes.BuildPlan
-- want.txt --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
validators: []
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
resources:
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1

View File

@@ -0,0 +1,37 @@
# https://github.com/holos-run/holos/issues/357
exec holos init platform v1alpha5 --force
! exec holos render platform
stderr 'secret.kind: conflicting values "Forbidden. Use an ExternalSecret instead." and "Secret"'
-- validators.cue --
package holos
import "github.com/holos-run/holos/api/author/v1alpha5:author"
#ComponentConfig: author.#ComponentConfig & {
Validators: cue: {
kind: "Command"
command: args: ["holos", "cue", "vet", "./policy", "--path", "strings.ToLower(kind)"]
}
}
-- policy/validations.cue --
package validations
secret: kind: "Forbidden. Use an ExternalSecret instead."
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/secret.cue --
package holos
holos: Component.BuildPlan
Component: #Kubernetes & {
Resources: Secret: test: {
metadata: name: "test"
}
}

View File

@@ -23,7 +23,7 @@ improves cross team collaboration through well defined, typed structures at the
integration layer. These definitions provide golden paths for other teams to
easily integrate their own services into the platform.
<!-- truncate -->
{/* truncate */}
## The Problem

View File

@@ -1,5 +0,0 @@
import DocCardList from '@theme/DocCardList';
# API Reference
<DocCardList />

10
doc/md/api.mdx Normal file
View File

@@ -0,0 +1,10 @@
---
slug: api
description: Schema Reference
---
import DocCardList from '@theme/DocCardList';
# Schema Reference
<DocCardList />

View File

@@ -1,3 +1,186 @@
# Author Schema
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
v1alpha5
```go
import "github.com/holos-run/holos/api/author/v1alpha5"
```
Package author contains a standard set of schemas for component authors to generate common [core](<https://holos.run/docs/api/core/>) BuildPlans.
Holos values stability, flexibility, and composition. This package intentionally defines only the minimal necessary set of structures. Component authors are encouraged to define their own structures building on our example [topics](<https://holos.run/docs/topics/>).
The Holos Maintainers may add definitions to this package if the community identifies nearly all users must define the exact same structure. Otherwise, definitions should be added as a customizable example in [topics](<https://holos.run/docs/topics/>).
For example, structures representing a cluster and environment almost always need to be defined. Their definition varies from one organization to the next. Therefore, customizable definitions for a cluster and environment are best maintained in [topics](<https://holos.run/docs/topics/>), not standardized in this package.
## Index
- [type ComponentConfig](<#ComponentConfig>)
- [type Helm](<#Helm>)
- [type Kubernetes](<#Kubernetes>)
- [type Kustomize](<#Kustomize>)
- [type KustomizeConfig](<#KustomizeConfig>)
- [type NameLabel](<#NameLabel>)
- [type Platform](<#Platform>)
<a name="ComponentConfig"></a>
## type ComponentConfig {#ComponentConfig}
ComponentConfig represents the configuration common to all kinds of components for use with the holos render component command. All component kinds may be transformed with [kustomize](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>) configured with the [KustomizeConfig](<#KustomizeConfig>) field.
- [Helm](<#Helm>) charts.
- [Kubernetes](<#Kubernetes>) resources generated from CUE.
- [Kustomize](<#Kustomize>) bases.
```go
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the kustomize configuration.
KustomizeConfig KustomizeConfig
// Validators represent checks that must pass for output to be written.
Validators map[NameLabel]core.Validator
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm assembles a BuildPlan rendering a helm chart. Useful to mix in additional resources from CUE and transform the helm output with kustomize.
```go
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="Kubernetes"></a>
## type Kubernetes {#Kubernetes}
Kubernetes assembles a BuildPlan containing inline resources exported from CUE.
```go
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize assembles a BuildPlan rendering manifests from a [kustomize](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>) kustomization.
```go
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="KustomizeConfig"></a>
## type KustomizeConfig {#KustomizeConfig}
KustomizeConfig represents the configuration for [kustomize](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>) post processing. Use the Files field to mix in plain manifest files located in the component directory. Use the Resources field to mix in manifests from network urls.
```go
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
```
<a name="NameLabel"></a>
## type NameLabel {#NameLabel}
NameLabel represents the common use case of converting a struct to a list where the name field of each value unifies with the field name of the outer struct.
For example:
```
S: [NameLabel=string]: name: NameLabel
S: jeff: _
S: gary: _
S: nate: _
L: [for x in S {x}]
// L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
```
```go
type NameLabel string
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform assembles a core Platform in the Resource field for the holos render platform command. Use the Components field to register components with the platform.
```go
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -1,3 +1,505 @@
# Core Schema
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
v1alpha5
```go
import "github.com/holos-run/holos/api/core/v1alpha5"
```
Package core contains schemas for a [Platform](<#Platform>) and [BuildPlan](<#BuildPlan>). Holos takes a [Platform](<#Platform>) as input, then iterates over each [Component](<#Component>) to produce a [BuildPlan](<#BuildPlan>). Holos processes the [BuildPlan](<#BuildPlan>) to produce fully rendered manifests, each an [Artifact](<#Artifact>).
## Index
- [type Artifact](<#Artifact>)
- [type Auth](<#Auth>)
- [type AuthSource](<#AuthSource>)
- [type BuildPlan](<#BuildPlan>)
- [type BuildPlanSpec](<#BuildPlanSpec>)
- [type Chart](<#Chart>)
- [type Command](<#Command>)
- [type Component](<#Component>)
- [type ExtractYAML](<#ExtractYAML>)
- [type File](<#File>)
- [type FileContent](<#FileContent>)
- [type FileContentMap](<#FileContentMap>)
- [type FilePath](<#FilePath>)
- [type Generator](<#Generator>)
- [type Helm](<#Helm>)
- [type Instance](<#Instance>)
- [type InternalLabel](<#InternalLabel>)
- [type Join](<#Join>)
- [type Kind](<#Kind>)
- [type Kustomization](<#Kustomization>)
- [type Kustomize](<#Kustomize>)
- [type Metadata](<#Metadata>)
- [type Platform](<#Platform>)
- [type PlatformSpec](<#PlatformSpec>)
- [type Repository](<#Repository>)
- [type Resource](<#Resource>)
- [type Resources](<#Resources>)
- [type Transformer](<#Transformer>)
- [type Validator](<#Validator>)
- [type Values](<#Values>)
<a name="Artifact"></a>
## type Artifact {#Artifact}
Artifact represents one fully rendered manifest produced by a [Transformer](<#Transformer>) sequence, which transforms a [Generator](<#Generator>) collection. A [BuildPlan](<#BuildPlan>) produces an [Artifact](<#Artifact>) collection.
Each Artifact produces one manifest file artifact. Generator Output values are used as Transformer Inputs. The Output field of the final [Transformer](<#Transformer>) should have the same value as the Artifact field.
When there is more than one [Generator](<#Generator>) there must be at least one [Transformer](<#Transformer>) to combine outputs into one Artifact. If there is a single Generator, it may directly produce the Artifact output.
An Artifact is processed concurrently with other artifacts in the same [BuildPlan](<#BuildPlan>). An Artifact should not use an output from another Artifact as an input. Each [Generator](<#Generator>) may also run concurrently. Each [Transformer](<#Transformer>) is executed sequentially starting after all generators have completed.
Output fields are write\-once. It is an error for multiple Generators or Transformers to produce the same Output value within the context of a [BuildPlan](<#BuildPlan>).
```go
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Validators []Validator `json:"validators,omitempty" yaml:"validators,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
```
<a name="Auth"></a>
## type Auth {#Auth}
Auth represents environment variable names containing auth credentials.
```go
type Auth struct {
Username AuthSource `json:"username" yaml:"username"`
Password AuthSource `json:"password" yaml:"password"`
}
```
<a name="AuthSource"></a>
## type AuthSource {#AuthSource}
AuthSource represents a source for the value of an [Auth](<#Auth>) field.
```go
type AuthSource struct {
Value string `json:"value,omitempty" yaml:"value,omitempty"`
FromEnv string `json:"fromEnv,omitempty" yaml:"fromEnv,omitempty"`
}
```
<a name="BuildPlan"></a>
## type BuildPlan {#BuildPlan}
BuildPlan represents an implementation of the [rendered manifest pattern](<https://akuity.io/blog/the-rendered-manifests-pattern>). Holos processes a BuildPlan to produce one or more [Artifact](<#Artifact>) output files. BuildPlan artifact files usually contain Kubernetes manifests, but they may have any content.
A BuildPlan usually produces two artifacts. One artifact contains a manifest of resources. A second artifact contains a GitOps resource to manage the first, usually an ArgoCD Application resource.
Holos uses CUE to construct a BuildPlan. A future enhancement will support user defined executables providing a BuildPlan to Holos in the style of an [external credential provider](<https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md>).
```go
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
```
<a name="BuildPlanSpec"></a>
## type BuildPlanSpec {#BuildPlanSpec}
BuildPlanSpec represents the specification of the [BuildPlan](<#BuildPlan>).
```go
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
```
<a name="Chart"></a>
## type Chart {#Chart}
Chart represents a [Helm](<#Helm>) Chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
```
<a name="Command"></a>
## type Command {#Command}
Command represents a command vetting one or more artifacts. Holos appends fully qualified input file paths to the end of the args list, then executes the command. Inputs are written into a temporary directory prior to executing the command and removed afterwards.
```go
type Command struct {
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
}
```
<a name="Component"></a>
## type Component {#Component}
Component represents the complete context necessary to produce a [BuildPlan](<#BuildPlan>) from a path containing parameterized CUE configuration.
```go
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path" yaml:"path"`
// Instances represents additional cue instance paths to unify with Path.
// Useful to unify data files into a component BuildPlan. Added in holos
// 0.101.7.
Instances []Instance `json:"instances,omitempty" yaml:"instances,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
<a name="ExtractYAML"></a>
## type ExtractYAML {#ExtractYAML}
ExtractYAML represents a cue data instance encoded as yaml or json. If Path refers to a directory all files in the directory are extracted non\-recursively. Otherwise, path must refer to a file.
```go
type ExtractYAML struct {
Path string `json:"path" yaml:"path"`
}
```
<a name="File"></a>
## type File {#File}
File represents a simple single file copy [Generator](<#Generator>). Useful with a [Kustomize](<#Kustomize>) [Transformer](<#Transformer>) to process plain manifest files stored in the component directory. Multiple File generators may be used to transform multiple resources.
```go
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source" yaml:"source"`
}
```
<a name="FileContent"></a>
## type FileContent {#FileContent}
FileContent represents file contents.
```go
type FileContent string
```
<a name="FileContentMap"></a>
## type FileContentMap {#FileContentMap}
FileContentMap represents a mapping of file paths to file contents.
```go
type FileContentMap map[FilePath]FileContent
```
<a name="FilePath"></a>
## type FilePath {#FilePath}
FilePath represents a file path.
```go
type FilePath string
```
<a name="Generator"></a>
## type Generator {#Generator}
Generator generates Kubernetes resources. [Helm](<#Helm>) and [Resources](<#Resources>) are the most commonly used, often paired together to mix\-in resources to an unmodified Helm chart. A simple [File](<#File>) generator is also available for use with the [Kustomize](<#Kustomize>) transformer.
Each Generator in an [Artifact](<#Artifact>) must have a distinct Output value for a [Transformer](<#Transformer>) to reference.
1. [Resources](<#Resources>) \- Generates resources from CUE code.
2. [Helm](<#Helm>) \- Generates rendered yaml from a [Chart](<#Chart>).
3. [File](<#File>) \- Generates data by reading a file from the component directory.
```go
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm represents a [Chart](<#Chart>) manifest [Generator](<#Generator>).
```go
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
```
<a name="Instance"></a>
## type Instance {#Instance}
Instance represents a data instance to unify with the configuration.
Useful to unify json and yaml files with cue configuration files for integration with other tools. For example, executing holos render platform from a pull request workflow after [Kargo](<https://docs.kargo.io/>) executes the [yaml update](<https://docs.kargo.io/references/promotion-steps#yaml-update>) and [git wait for pr](<https://docs.kargo.io/references/promotion-steps#git-wait-for-pr>) promotion steps.
```go
type Instance struct {
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"ExtractYAML\""`
// Ignored unless kind is ExtractYAML.
ExtractYAML ExtractYAML `json:"extractYAML,omitempty" yaml:"extractYAML,omitempty"`
}
```
<a name="InternalLabel"></a>
## type InternalLabel {#InternalLabel}
InternalLabel is an arbitrary unique identifier internal to holos itself. The holos cli is expected to never write a InternalLabel value to rendered output files, therefore use a InternalLabel when the identifier must be unique and internal. Defined as a type for clarity and type checking.
```go
type InternalLabel string
```
<a name="Join"></a>
## type Join {#Join}
Join represents a [Transformer](<#Transformer>) using [bytes.Join](<https://pkg.go.dev/bytes#Join>) to concatenate multiple inputs into one output with a separator. Useful for combining output from [Helm](<#Helm>) and [Resources](<#Resources>) together into one [Artifact](<#Artifact>) when [Kustomize](<#Kustomize>) is otherwise unnecessary.
```go
type Join struct {
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
```
<a name="Kind"></a>
## type Kind {#Kind}
Kind is a discriminator. Defined as a type for clarity and type checking.
```go
type Kind string
```
<a name="Kustomization"></a>
## type Kustomization {#Kustomization}
Kustomization represents a kustomization.yaml file for use with the [Kustomize](<#Kustomize>) [Transformer](<#Transformer>). Untyped to avoid tightly coupling holos to kubectl versions which was a problem for the Flux maintainers. Type checking is expected to happen in CUE against the kubectl version the user prefers.
```go
type Kustomization map[string]any
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize represents a kustomization [Transformer](<#Transformer>).
```go
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
```
<a name="Metadata"></a>
## type Metadata {#Metadata}
Metadata represents data about the resource such as the Name.
```go
type Metadata struct {
// Name represents the resource name.
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform represents a platform to manage. A Platform specifies a [Component](<#Component>) collection and integrates the components together into a holistic platform. Holos iterates over the [Component](<#Component>) collection producing a [BuildPlan](<#BuildPlan>) for each, which holos then executes to render manifests.
Inspect a Platform resource holos would process by executing:
```
cue export --out yaml ./platform
```
```go
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
```
<a name="PlatformSpec"></a>
## type PlatformSpec {#PlatformSpec}
PlatformSpec represents the platform specification.
```go
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components" yaml:"components"`
}
```
<a name="Repository"></a>
## type Repository {#Repository}
Repository represents a [Helm](<#Helm>) [Chart](<#Chart>) repository.
The Auth field is useful to configure http basic authentication to the Helm repository. Holos gets the username and password from the environment variables represented by the Auth field.
```go
type Repository struct {
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
Auth Auth `json:"auth,omitempty" yaml:"auth,omitempty"`
}
```
<a name="Resource"></a>
## type Resource {#Resource}
Resource represents one kubernetes api object.
```go
type Resource map[string]any
```
<a name="Resources"></a>
## type Resources {#Resources}
Resources represents Kubernetes resources. Most commonly used to mix resources into the [BuildPlan](<#BuildPlan>) generated from CUE, but may be generated from elsewhere.
```go
type Resources map[Kind]map[InternalLabel]Resource
```
<a name="Transformer"></a>
## type Transformer {#Transformer}
Transformer combines multiple inputs from prior [Generator](<#Generator>) or [Transformer](<#Transformer>) outputs into one output. [Kustomize](<#Kustomize>) is the most commonly used transformer. A simple [Join](<#Join>) is also supported for use with plain manifest files.
1. [Kustomize](<#Kustomize>) \- Patch and transform the output from prior generators or transformers. See [Introduction to Kustomize](<https://kubectl.docs.kubernetes.io/guides/config_management/introduction/>).
2. [Join](<#Join>) \- Concatenate multiple prior outputs into one output.
```go
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
```
<a name="Validator"></a>
## type Validator {#Validator}
Validator validates files. Useful to validate an [Artifact](<#Artifact>) prior to writing it out to the final destination. Holos may execute validators concurrently. See the [validators](<https://holos.run/docs/v1alpha5/tutorial/validators/>) tutorial for an end to end example.
```go
type Validator struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Command\""`
// Inputs represents the files to validate. Usually the final Artifact.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Command represents a validation command. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
```
<a name="Values"></a>
## type Values {#Values}
Values represents [Helm](<#Helm>) Chart values generated from CUE.
```go
type Values map[string]any
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -0,0 +1,16 @@
Integrate the `podinfo` component into the platform.
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```

View File

@@ -0,0 +1,34 @@
Create a directory for the example `podinfo` component we'll use to render
platform manifests.
```bash
mkdir -p components/podinfo
```
Create the CUE configuration for the example `podinfo` component.
```bash
cat <<EOF >components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
```bash
EOF
```

View File

@@ -1,14 +1,17 @@
---
description: Get Support for Holos
slug: /support
sidebar_position: 900
slug: support
sidebar_position: 2000
---
# Support
## Community Support
You can ask questions in our community forums in [GitHub Discussions](https://github.com/holos-run/holos/discussions), [Discord](https://discord.gg/JgDVbNpye7), or [Google Groups](https://groups.google.com/g/holos-discuss).
You can ask questions in our community forums in [GitHub
Discussions](https://github.com/holos-run/holos/discussions),
[Discord](https://discord.gg/JgDVbNpye7), or [Google
Groups](https://groups.google.com/g/holos-discuss).
## Commercial Support and Services

View File

@@ -1 +0,0 @@
# Multi Cluster

View File

@@ -0,0 +1,18 @@
---
description: Architecture diagrams.
slug: architecture
sidebar_position: 100
---
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
import RenderComponentDiagram from '@site/src/diagrams/render-component-sequence.mdx';
# Architecture
## Platform Rendering Sequence
<RenderPlatformDiagram />
## Component Rendering Sequence
<RenderComponentDiagram />

View File

@@ -0,0 +1,57 @@
---
description: Holos compared to other tools
sidebar_label: Comparison
slug: comparison
sidebar_position: 40
---
{/* cspell:ignore Prodan, rollouts */}
# Holos compared to other tools
## Timoni
Holos and Timoni both aim to solve similar problems but approach them at
different levels of the stack.
Timoni focuses on managing applications by evaluating [CUE] stored in OCI
containers. Its creator, Stephan Prodan, envisions a controller that applies the
resulting manifests. In this process, Timoni defers to [Flux] for managing Helm
charts within the cluster.
In contrast, Holos implements the [Rendered Manifests Pattern] and takes a
different approach, particularly in how it handles [Helm] charts. Like
[ArgoCD], Holos renders Helm charts into manifests using the `helm template`
command in its rendering pipeline. Holos differs from Timoni in several important
ways:
1. **Separation of Responsibilities:** Holos stops short of applying
rendered manifests to a cluster, leaving that task to existing tools like
[ArgoCD], [Flux], or even basic `kubectl apply` commands.
2. **Ecosystem Integration:** By focusing solely on rendering Kubernetes
manifests, Holos creates space for other tools to handle deployment and
management. For instance, Holos integrates seamlessly with [Kargo] for
progressive rollouts, as [Kargo] operates between Holos and the Kubernetes API.
This approach ensures that you're not locked into any specific tool and can
choose the best solution for each task.
3. **Platform Integration:** Holos focuses on integrating multiple Components
into a larger Platform. In Holos terminology, a Component refers to a wrapper
for [Helm] charts, [Kustomize] bases, or raw YAML files, integrated into the
rendering pipeline through [CUE]. A Platform represents the full combination of
these components.
4. **Explicit Rendering Pipeline:** Holos emphasizes flexibility in its
rendering pipeline. The system allows any tool that generates Kubernetes
manifests to be wrapped in a Generator, which can then feed into existing
transformers like [Kustomize]. This explicit separation makes Holos highly
adaptable for different workflows.
[Kargo]: https://kargo.io/
[Flux]: https://fluxcd.io
[Helm]: https://helm.sh
[ArgoCD]: https://argoproj.github.io/cd/
[Kustomize]: https://kustomize.io/
[CUE]: https://cuelang.org/
[Rendered Manifests Pattern]: https://akuity.io/blog/the-rendered-manifests-pattern

View File

@@ -0,0 +1,223 @@
---
slug: argocd-application
title: ArgoCD Application
description: Configuring an Application for each Component.
sidebar_position: 110
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
import CommonComponentIntegrate from '../../common/example-component-integrate.mdx';
# ArgoCD Application
## Overview
This topic covers how to mix in an ArgoCD Application to all components. We'll
use the `Artifacts` field of [ComponentConfig] defined by the author schema.
## The Code
### Generating the structure
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-argocd-application && cd holos-argocd-application
```
```shell
holos init platform v1alpha5
```
### Creating an example Component
<CommonComponent />
<CommonComponentIntegrate />
## Adding ArgoCD Application
Configure Holos to render an [Application] by defining an [Artifact] for it in
every BuildPlan holos produces. We're unifying our custom configuration with
the existing `#ComponentConfig` defined in `schema.cue`.
```bash
cat <<EOF >argocd-application.cue
```
```cue showLineNumbers
package holos
import (
"path"
app "argoproj.io/application/v1alpha1"
)
#ComponentConfig: {
Name: _
OutputBaseDir: _
let ArtifactPath = path.Join([OutputBaseDir, "gitops", "\(Name).application.gen.yaml"], path.Unix)
let ResourcesPath = path.Join(["deploy", OutputBaseDir, "components", Name], path.Unix)
Artifacts: "\(Name)-application": {
artifact: ArtifactPath
generators: [{
kind: "Resources"
output: artifact
resources: Application: (Name): app.#Application & {
metadata: name: Name
metadata: namespace: "argocd"
spec: {
destination: server: "https://kubernetes.default.svc"
project: "default"
source: {
path: ResourcesPath
repoURL: "https://example.com/example.git"
targetRevision: "main"
}
}
}
}]
}
}
```
```bash
EOF
```
## Inspecting the BuildPlan
Our customized `#ComponentConfig` results in the following `BuildPlan`.
:::note
The second artifact around line 40 contains the configured `Application`
resource.
:::
<Tabs groupId="55075C71-02E8-4222-88C0-2D52C82D18FC">
<TabItem value="command" label="Command">
```bash
holos cue export --expression holos --out=yaml ./components/podinfo
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: podinfo
spec:
artifacts:
- artifact: components/podinfo/podinfo.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: podinfo
version: 6.6.2
release: podinfo
repository:
name: podinfo
url: https://stefanprodan.github.io/podinfo
values: {}
enableHooks: false
- kind: Resources
output: resources.gen.yaml
resources: {}
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: components/podinfo/podinfo.gen.yaml
kustomize:
kustomization:
labels:
- includeSelectors: false
pairs: {}
resources:
- helm.gen.yaml
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
- artifact: gitops/podinfo.application.gen.yaml
generators:
- kind: Resources
output: gitops/podinfo.application.gen.yaml
resources:
Application:
podinfo:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/components/podinfo
repoURL: https://example.com/example.git
targetRevision: main
source:
component:
name: podinfo
path: no-path
parameters: {}
```
</TabItem>
</Tabs>
## Rendering manifests
<Tabs groupId="E150C802-7162-4FBF-82A7-77D9ADAEE847">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
</Tabs>
## Reviewing the Application
The Artifact we added to `#ComponentConfig` will produce an ArgoCD Application
resource for every component in the platform. The output in this example is
located at:
```txt
deploy/gitops/podinfo.application.gen.yaml
```
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/components/podinfo
repoURL: https://example.com/example.git
targetRevision: main
```
[podinfo]: https://github.com/stefanprodan/podinfo
[CUE Module]: https://cuelang.org/docs/reference/modules/
[CUE Tags]: https://cuelang.org/docs/howto/inject-value-into-evaluation-using-tag-attribute/
[Application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[Platform]: ../../api/author.md#Platform
[ComponentConfig]: ../../api/author.md#ComponentConfig
[Artifact]: ../../api/core.md#Artifact

View File

@@ -0,0 +1,218 @@
---
slug: flux-kustomization
title: Flux Kustomization
description: Configuring a Kustomization for each Component.
sidebar_position: 120
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
import CommonComponentIntegrate from '../../common/example-component-integrate.mdx';
# Flux Kustomization
## Overview
This topic covers how to mix in a Flux Kustomization to all components. We'll
use the `Artifacts` field of [ComponentConfig] defined by the author schema.
## The Code
### Generating the structure
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-flux-kustomization && cd holos-flux-kustomization
```
```shell
holos init platform v1alpha5
```
### Creating an example Component
<CommonComponent />
<CommonComponentIntegrate />
## Adding Flux Kustomizations
Configure Holos to render a [Kustomization] by defining an [Artifact] for it in
every BuildPlan holos produces. We're unifying our custom configuration with
the existing `#ComponentConfig` defined in `schema.cue`.
```bash
cat <<EOF >flux-kustomization.cue
```
```cue showLineNumbers
package holos
import (
"path"
flux "kustomize.toolkit.fluxcd.io/kustomization/v1"
)
#ComponentConfig: {
Name: _
OutputBaseDir: _
let ArtifactPath = path.Join([OutputBaseDir, "gitops", "\(Name).kustomization.gen.yaml"], path.Unix)
let ResourcesPath = path.Join(["deploy", OutputBaseDir, "components", Name], path.Unix)
Artifacts: "\(Name)-kustomization": {
artifact: ArtifactPath
generators: [{
kind: "Resources"
output: artifact
resources: Kustomization: (Name): flux.#Kustomization & {
metadata: name: Name
metadata: namespace: "default"
spec: {
interval: "5m"
timeout: "1m"
prune: true
path: ResourcesPath
sourceRef: {
kind: "GitRepository"
name: "webapp"
}
}
}
}]
}
}
```
```bash
EOF
```
## Inspecting the BuildPlan
Our customized `#ComponentConfig` results in the following `BuildPlan`.
:::note
The second artifact around line 40 contains the configured `Kustomization`
resource.
:::
<Tabs groupId="55075C71-02E8-4222-88C0-2D52C82D18FC">
<TabItem value="command" label="Command">
```bash
holos cue export --expression holos --out=yaml ./components/podinfo
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: podinfo
spec:
artifacts:
- artifact: components/podinfo/podinfo.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: podinfo
version: 6.6.2
release: podinfo
repository:
name: podinfo
url: https://stefanprodan.github.io/podinfo
values:
ui:
message: Hello World
enableHooks: false
- kind: Resources
output: resources.gen.yaml
resources: {}
validators: []
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: components/podinfo/podinfo.gen.yaml
kustomize:
kustomization:
resources:
- helm.gen.yaml
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
- artifact: gitops/podinfo.kustomization.gen.yaml
generators:
- kind: Resources
output: gitops/podinfo.kustomization.gen.yaml
resources:
Kustomization:
podinfo:
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: podinfo
namespace: default
spec:
interval: 5m
path: deploy/components/podinfo
prune: true
sourceRef:
kind: GitRepository
name: webapp
timeout: 1m
```
</TabItem>
</Tabs>
## Rendering manifests
<Tabs groupId="E150C802-7162-4FBF-82A7-77D9ADAEE847">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```
rendered podinfo in 140.341417ms
rendered platform in 140.441333ms
```
</TabItem>
</Tabs>
## Reviewing the Kustomization
The Artifact we added to `#ComponentConfig` will produce a Flux Kustomization
resource for every component in the platform. The output in this example is
located at:
```txt
deploy/gitops/podinfo.kustomization.gen.yaml
```
```yaml showLineNumbers
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: podinfo
namespace: default
spec:
interval: 5m
path: deploy/components/podinfo
prune: true
sourceRef:
kind: GitRepository
name: webapp
timeout: 1m
```
[podinfo]: https://github.com/stefanprodan/podinfo
[CUE Module]: https://cuelang.org/docs/reference/modules/
[CUE Tags]: https://cuelang.org/docs/howto/inject-value-into-evaluation-using-tag-attribute/
[Kustomization]: https://fluxcd.io/flux/components/kustomize/kustomizations/
[Platform]: ../../api/author.md#Platform
[ComponentConfig]: ../../api/author.md#ComponentConfig
[Artifact]: ../../api/core.md#Artifact

View File

@@ -0,0 +1,19 @@
---
slug: .
title: GitOps
description: Managing resources with GitOps.
sidebar_position: 120
---
import DocCardList from '@theme/DocCardList';
# GitOps
This section has self contained articles covering how to manage resources using
GitOps tooling like [ArgoCD] and [Flux].
---
<DocCardList />
[ArgoCD]: https://argo-cd.readthedocs.io/en/stable/
[Flux]: https://fluxcd.io/

20
doc/md/topics/kargo.mdx Normal file
View File

@@ -0,0 +1,20 @@
---
description: Kargo
slug: kargo
sidebar_position: 110
---
# Kargo
Holos pairs nicely with [Kargo], offering a holistic solution for code
promotion across stages.
Watch this space for a more detailed write up of the integration being
developed.
If you're interested in this topic, please thumbs up the [Kargo
Topic](https://github.com/holos-run/holos/issues/378) issue, or drop into
[Discord] and let us know about your use case.
[Kargo]: https://kargo.io/
[Discord]: https://discord.gg/JgDVbNpye7

View File

@@ -1,7 +1,7 @@
---
description: Build a local Cluster to use with these guides.
slug: /guides/local-cluster
sidebar_position: 999
description: Build a local cluster for use with Holos.
slug: local-cluster
sidebar_position: 50
---
import Tabs from '@theme/Tabs';
@@ -16,8 +16,7 @@ have a standard Kubernetes API server with proper DNS and TLS certificates.
You'll be able to easily reset the cluster to a known good state to iterate on
your own Platform.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
The [Glossary] page defines capitalized terms such as Platform and Component.
## Reset the Cluster
@@ -105,7 +104,7 @@ You're back to the same state as the first time you completed this guide.
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
1. [holos](../tutorial/setup.mdx) - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a k8s api server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
@@ -274,4 +273,7 @@ k3d cluster delete workload
## Next Steps
Now that you have a real cluster, apply and explore the manifests Holos renders
in the [Quickstart](/docs/quickstart) guide.
in the [Tutorial].
[Glossary]: ../glossary.mdx
[Tutorial]: ../tutorial.mdx

View File

@@ -0,0 +1,65 @@
---
description: OCI Helm Charts
slug: oci-helm-charts
sidebar_position: 710
---
# OCI Helm Charts
Holos supports OCI Helm charts. Use the following example to get started.
```bash
mkdir -p oci-helm && cd oci-helm
holos init platform v1alpha5
```
```bash
mkdir -p components/podinfo-oci
cat <<EOF > components/podinfo-oci/podinfo-oci.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Chart: {
name: "oci://ghcr.io/stefanprodan/charts/podinfo"
release: "podinfo"
version: "6.6.2"
}
}
```
```bash
EOF
```
Register the component with the platform.
```bash
cat <<EOF >platform/podinfo-oci.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo-oci"
path: "components/podinfo-oci"
}
```
```bash
EOF
```
The OCI chart is cached in the vendor directory and rendered.
```bash
holos render platform
```
```txt
Pulled: ghcr.io/stefanprodan/charts/podinfo:6.6.2
Digest: sha256:83295d47de6d6ca634ed4b952a7572fc176bcc38854d0c11ca0fa197bc5f1154
rendered podinfo-oci in 7.21581325s
rendered platform in 7.216199167s
```

View File

@@ -0,0 +1,183 @@
---
description: Private Helm Repositories
slug: private-helm
sidebar_position: 700
---
# Private Helm
Holos supports private Helm repositories accessed with http basic authentication
since `v0.101.4`. Use the following command to update your author and core
schemas to support this configuration.
```bash
holos init platform v1alpha5 --force
```
## Configuration
Holos uses the Helm SDK and defers to it for authentication to private
repositories. Each Helm Generator supports providing http basic authentication
credentials from environment variables.
For example, the following BuildPlan causes `holos` to get the admin username
password from the `HOLOS_TEST_PASS` environment variable.
```bash
mkdir -p projects/holos/components/private-chart
cat <<EOF > projects/holos/components/private-chart/private-chart.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
// Test holos can access a private repository with basic auth.
// https://github.com/holos-run/holos/issues/370
Component: #Helm & {
Chart: {
name: "mychart"
version: "0.1.0"
repository: {
name: "holos-test"
url: "https://charts.holos.localhost"
// auth: username: fromEnv: "HOLOS_TEST_USER"
auth: username: value: "admin"
auth: password: fromEnv: "HOLOS_TEST_PASS"
}
}
}
```
```bash
EOF
```
## Verification
Verify `holos` can access a private Helm repository by setting [ChartMuseum] up
on a [Local Cluster]. We'll use https with basic auth to authenticate to the
chart repository.
Using the [bank of holos] repository, deploy chart museum:
```bash
holos render platform -t ChartMuseum
```
Apply the manifests:
```bash
kubectl apply --server-side=true -f deploy/clusters/workload/projects/holos/components/chart-museum
kubectl apply --server-side=true -f deploy/clusters/workload/projects/network/components/httproutes
```
Get the admin password:
```bash
kubectl get secret -n holos chartmuseum-auth -o json \
| jq --exit-status -r '.data.password | @base64d'
```
Add a local repo:
```bash
helm repo add holos-test https://charts.holos.localhost --username admin
```
```txt
Password:
"holos-test" has been added to your repositories
```
:::note
Helm by default stores this password in `~/Library/Preferences/helm/repositories.yaml`
:::
Create a chart:
```bash
helm create mychart
```
```txt
Creating mychart
```
Package it up.
```bash
helm package mychart
```
```txt
Successfully packaged chart and saved it to: mychart-0.1.0.tgz
```
Publish it.
```bash
curl --user "admin:$(pbpaste)" --data-binary "@mychart-0.1.0.tgz" https://charts.holos.localhost/api/charts
```
```json
{"saved":true}
```
Remove all cached charts:
```bash
find . -name vendor | xargs rm -rf
```
Render the chart:
```bash
cat <<EOF > test-private-repo.cue
```
```cue showLineNumbers
@if(TestPrivateRepo)
package holos
// Test holos can access a private repository with basic auth.
// https://github.com/holos-run/holos/issues/370
Projects: holos: #ProjectBuilder & {
team: "holos-authors"
namespaces: holos: _
_components: "private-chart": _
}
```
```bash
EOF
```
```
time holos render platform -t TestPrivateRepo
```
Check the chart was pulled and cached:
```shell
tree ./projects/holos/components/private-chart/vendor
```
```txt
./projects/holos/components/private-chart/vendor
└── 0.1.0
├── mychart
│   ├── Chart.yaml
│   ├── mychart-0.1.0.tgz
│   ├── templates
│   │   ├── NOTES.txt
│   │   ├── _helpers.tpl
│   │   ├── deployment.yaml
│   │   ├── hpa.yaml
│   │   ├── ingress.yaml
│   │   ├── service.yaml
│   │   ├── serviceaccount.yaml
│   │   └── tests
│   │   └── test-connection.yaml
│   └── values.yaml
└── mychart-0.1.0.tgz
6 directories, 11 files
```
[Local Cluster]: ./local-cluster.mdx
[ChartMuseum]: https://chartmuseum.com/docs/
[bank of holos]: https://github.com/holos-run/bank-of-holos

View File

@@ -0,0 +1,425 @@
---
slug: clusters
title: Clusters
description: Managing clusters - management and workload sets.
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
# Clusters
## Overview
This topic covers one common method to manage multiple clusters with Holos. We'll
define two schemas to hold cluster attributes. First, a single `#Cluster` then
a `#Clusters` collection. We'll use a `Clusters: #Clusters` struct to look up
configuration data using a key. We'll use the cluster name as the lookup key
identifying the cluster.
We'll also organize sets of similar clusters by defining `#ClusterSet` and
`#ClusterSets`. We'll use a `ClusterSets:
#ClusterSets` struct to configure a management cluster and iterate over all
workload clusters.
## The Code
### Initializing the structure
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-multiple-clusters && cd holos-multiple-clusters
```
```shell
holos init platform v1alpha5
```
### Using an example Component
<CommonComponent />
We'll integrate the component with the platform after we define the
configuration structures.
## Defining Clusters
We'll define a `#Cluster` schema and a `#Clusters` collection in this section.
We'll use these schemas to define a `Clusters` structure we use to manage
multiple clusters.
### Assumptions
We'll make the following assumptions, which hold true for many real world
environments.
1. There are two sets of clusters, workload clusters and management clusters.
2. There is one management cluster.
3. There are multiple workload clusters.
4. Each workload cluster is configured similarly, but not identically, to the
others.
### Prototyping the data
Before we define the schema, let's prototype the data structure we want to work
with. We want a structure that makes it easy to iterate over each cluster in
two distinct sets of clusters, management clusters and workload clusters. The
following `ClusterSets` struct accomplishes this goal.
```yaml showLineNumbers
management:
name: management
clusters:
management:
name: management
region: us-central1
set: management
workload:
name: workload
clusters:
e1:
name: e1
region: us-east1
set: workload
w1:
name: w1
region: us-west1
set: workload
```
:::tip
The `ClusterSets` data structure supports iterating over each cluster in each
cluster set.
:::
:::important
You're free to define your own fields and structures like we define `region` in
this topic.
:::
### Defining the schema
Armed with a concrete example of the structure, we can write a schema to define
and validate the data.
In CUE, schema definitions are usually defined at the root so they're accessible
in all subdirectories. The following is one example schema, you're free to
modify it to your situation. Holos is flexible, supporting schemas that match
your unique use case.
```bash
cat <<EOF > clusters.schema.cue
```
```cue showLineNumbers
package holos
import "strings"
// #Cluster represents one cluster
#Cluster: {
// name represents the cluster name.
name: string & =~"[a-z][a-z0-9]+" & strings.MinRunes(2) & strings.MaxRunes(63)
// Constrain the regions. No default, the region must be specified.
region: "us-east1" | "us-central1" | "us-west1"
// Each cluster must be in only one set of clusters. All but one cluster are
// workload clusters, so make it the default.
set: "management" | *"workload"
}
// #Clusters represents a cluster collection structure
#Clusters: {
// name is the lookup key for the collection.
[NAME=string]: #Cluster & {
// name must match the struct field name.
name: NAME
}
}
// #ClusterSet represents a set of clusters.
#ClusterSet: {
// name represents the cluster set name.
name: string & =~"[a-z][a-z0-9]+" & strings.MinRunes(2) & strings.MaxRunes(63)
clusters: #Clusters & {
// Constrain the cluster set to clusters having the same set. Ensures
// clusters are never mis-categorized.
[_]: set: name
}
}
// #ClusterSets represents a cluster set collection.
#ClusterSets: {
// name is the lookup key for the collection.
[NAME=string]: #ClusterSet & {
// name must match the struct field name.
name: NAME
}
}
```
```bash
EOF
```
### Defining the data
With a schema defined, we also define the data close to the root so it's
accessible through the unified configuration tree.
```bash
cat <<EOF > clusters.cue
```
```cue showLineNumbers
package holos
Clusters: #Clusters & {
// Management Cluster
management: region: "us-central1"
management: set: "management"
// Local Cluster
local: region: "us-west1"
// Some example clusters. Add new clusters to the Clusters struct like this.
e1: region: "us-east1"
e2: region: "us-east1"
e3: region: "us-east1"
w1: region: "us-west1"
w2: region: "us-west1"
w3: region: "us-west1"
}
// ClusterSets is dynamically built from the Clusters structure.
ClusterSets: #ClusterSets & {
// Map every cluster into the correct set.
for CLUSTER in Clusters {
(CLUSTER.set): clusters: (CLUSTER.name): CLUSTER
}
}
```
```bash
EOF
```
### Inspecting the data
We'll use the `holos cue` command to inspect the `ClusterSets` data structure we
just defined.
<Tabs groupId="9190BDAD-B4C5-4386-9C94-8E178AA6178A">
<TabItem value="command" label="Command">
```bash
holos cue export --expression ClusterSets --out=yaml ./
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
management:
name: management
clusters:
management:
name: management
region: us-central1
set: management
workload:
name: workload
clusters:
local:
name: local
region: us-west1
set: workload
e1:
name: e1
region: us-east1
set: workload
e2:
name: e2
region: us-east1
set: workload
e3:
name: e3
region: us-east1
set: workload
w1:
name: w1
region: us-west1
set: workload
w2:
name: w2
region: us-west1
set: workload
w3:
name: w3
region: us-west1
set: workload
```
</TabItem>
</Tabs>
This looks like our prototype, we're confident we can iterate over each cluster
in each set.
## Integrating Components
The `ClusterSets` data structure unlocks the capability to iterate over each
cluster in each cluster set. We'll use this capability to integrate the
`podinfo` component with each cluster in the platform.
### Configuring the Output directory
We need to configure `holos` to write output manifests into a cluster specific
output directory. We'll use the [ComponentConfig] `OutputBaseDir` field for
this purpose. We'll pass the value of this field as a component parameter.
```bash
cat <<EOF > componentconfig.cue
```
```cue showLineNumbers
package holos
#ComponentConfig: {
// Inject the output base directory from platform component parameters.
OutputBaseDir: string @tag(outputBaseDir, type=string)
}
```
```bash
EOF
```
### Integrating Podinfo
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
// Manage podinfo on all workload clusters.
for CLUSTER in ClusterSets.workload.clusters {
// We use the cluster name to disambiguate different podinfo build plans.
Platform: Components: "\(CLUSTER.name)-podinfo": {
name: "podinfo"
// Reuse the same component across multiple workload clusters.
path: "components/podinfo"
// Configure a cluster-unique message in the podinfo UI.
parameters: message: "Hello, I am cluster \(CLUSTER.name) in region \(CLUSTER.region)"
// Write to deploy/{outputBaseDir}/components/{name}/{name}.gen.yaml
parameters: outputBaseDir: "clusters/\(CLUSTER.name)"
}
}
```
```bash
EOF
```
## Rendering manifests
### Rendering the Platform
Render the platform to configure `podinfo` on each cluster.
<Tabs groupId="34A2D80B-0E86-4142-B65B-7DF70C47E1D2">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached podinfo 6.6.2
rendered podinfo in 164.278583ms
rendered podinfo in 165.48525ms
rendered podinfo in 165.186208ms
rendered podinfo in 165.831792ms
rendered podinfo in 166.845208ms
rendered podinfo in 167.000208ms
rendered podinfo in 167.012208ms
rendered platform in 167.06525ms
```
</TabItem>
</Tabs>
### Inspecting the Tree
Rendering the platform produces the following rendered manifests.
```bash
tree deploy
```
```txt showLineNumbers
deploy
└── clusters
├── e1
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── e2
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── e3
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── local
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── w1
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── w2
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── w3
└── components
└── podinfo
└── podinfo.gen.yaml
23 directories, 7 files
```
### Inspecting the Variation
Note how each component has slight variation using the component parameters.
```bash
diff -U2 deploy/clusters/{e,w}1/components/podinfo/podinfo.gen.yaml
```
```diff
--- deploy/clusters/e1/components/podinfo/podinfo.gen.yaml 2024-11-17 14:20:17
+++ deploy/clusters/w1/components/podinfo/podinfo.gen.yaml 2024-11-17 14:20:17
@@ -61,5 +61,5 @@
env:
- name: PODINFO_UI_MESSAGE
- value: Hello, I am cluster e1 in region us-east1
+ value: Hello, I am cluster w1 in region us-west1
- name: PODINFO_UI_COLOR
value: '#34577c'
```
## Concluding Remarks
In this topic we covered how to use CUE structures to organize multiple clusters
into various sets.
1. Clusters are defined in one place at the root of the configuration.
2. Clusters may be organized into sets by their purpose.
3. Most organizations have at least two sets, a set of workload clusters and a
set of management clusters.
4. Holos uses CUE, a super set of JSON. New clusters may be added by dropping a
JSON file into the root of the repository.
5. The pattern of defining a `#Cluster` and a `#Clusters` collection is a
general pattern. We'll see the same pattern for environments, projects, owners,
and more.
6. Component parameters are a flexible way to inject user defined configuration
from the platform level into a reusable component.
[ClusterSet]: https://multicluster.sigs.k8s.io/api-types/cluster-set/
[Environments]: ./environments.mdx
[Namespace Sameness - SIG Multicluster Position Statement]: https://github.com/kubernetes/community/blob/master/sig-multicluster/namespace-sameness-position-statement.md
[ComponentConfig]: ../../api/author.md#ComponentConfig

View File

@@ -0,0 +1,521 @@
---
slug: environments
title: Environments
description: Managing Environments - dev, test, stage, prod.
sidebar_position: 130
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
# Environments
## Overview
This topic covers how to model environments in Holos. We'll define schemas for
`#Environment` and `#Environments` to represent one environment and a
collection. The `Environments: #Environments` struct maps environment names to
configurations.
:::note
This approach unifies the component definition with the overall platform
configuration, creating a tight coupling between the two.
:::
This tight coupling is appropriate when you're configuring your own platform.
For example:
1. When you're integrating third party software into your own platform.
2. When you're configuring first party in-house software into your own platform.
This approach is not well suited to writing a component to share outside of your
own organization, which we can think of as configuring someone else's platform.
## The Code
### Generating the structure
Use `holos init platform` to generate a minimal platform structure:
```shell
mkdir holos-environments-tutorial && cd holos-environments-tutorial
holos init platform v1alpha5
```
### Using an example Component
Create a directory for the example `podinfo` component we'll use to render
platform manifests.
```bash
mkdir -p components/podinfo
```
Create the CUE configuration for the example `podinfo` component.
```bash
cat <<EOF >components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Chart: {
name: "podinfo"
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
```bash
EOF
```
We'll integrate the component with the platform after we define the
configuration structures.
## Defining Environments
We'll define an `#Environment` schema `#Environments` collection. We'll use
these schemas to define an `Environments` struct of concrete configuration
values.
### Assumptions
There are two tiers of environments, prod and nonprod. Prod environments
organized along broad jurisdictions, for example US and EU. Nonprod
environments are organized by purpose, dev, test, and stage.
### Prototyping the data
Before we define the schema, let's prototype the data structure we want to work
with from the perspective of each component.
Let's imagine we're configuring `podinfo` to comply with regulations. When
podinfo is deployed to production in the EU, we'll configure opt-in behavior.
In the US we'll configure opt-out behavior.
We'll pass the environment name as a component parameter. The component
definition can then look up the jurisdiction to determine the appropriate
configuration values.
```shell
holos cue export --out=yaml --expression Environments
```
```yaml showLineNumbers
prod-pdx:
name: prod-pdx
tier: prod
jurisdiction: us
state: oregon
prod-cmh:
name: prod-cmh
tier: prod
jurisdiction: us
state: ohio
prod-ams:
name: prod-ams
tier: prod
jurisdiction: eu
state: netherlands
dev:
name: dev
tier: nonprod
jurisdiction: us
state: oregon
test:
name: test
tier: nonprod
jurisdiction: us
state: oregon
stage:
name: stage
tier: nonprod
jurisdiction: us
state: oregon
```
### Defining the schema
Given the example structure, we can write a schema to define and validate the
data.
```shell
cat <<EOF > environments.schema.cue
```
```cue showLineNumbers
package holos
#Environment: {
name: string
tier: "prod" | "nonprod"
jurisdiction: "us" | "eu" | "uk" | "global"
state: "oregon" | "ohio" | "germany" | "netherlands" | "england" | "global"
// Prod environment names must be prefixed with prod for clarity.
if tier == "prod" {
name: "prod" | =~"^prod-"
}
}
#Environments: {
[NAME=string]: #Environment & {
name: NAME
}
}
```
```shell
EOF
```
### Adding configuration
With a schema defined, we can fill in the concrete values.
```shell
cat <<EOF > environments.cue
```
```cue showLineNumbers
package holos
// Injected from Platform.spec.components.parameters.EnvironmentName
EnvironmentName: string @tag(EnvironmentName)
Environments: #Environments & {
"prod-pdx": {
tier: "prod"
jurisdiction: "us"
state: "oregon"
}
"prod-cmh": {
tier: "prod"
jurisdiction: "us"
state: "ohio"
}
"prod-ams": {
tier: "prod"
jurisdiction: "eu"
state: "netherlands"
}
// Nonprod environments are colocated together.
_nonprod: {
tier: "nonprod"
jurisdiction: "us"
state: "oregon"
}
dev: _nonprod
test: _nonprod
stage: _nonprod
}
```
```shell
EOF
```
### Inspecting the configuration
Inspect the `Environments` data structure to verify the schema and concrete
values are what we want.
<Tabs groupId="FF820F5A-A85F-464D-B299-39CAAFFCE5C6">
<TabItem value="command" label="Command">
```bash
holos cue export --out=yaml --expression Environments
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
prod-pdx:
name: prod-pdx
tier: prod
jurisdiction: us
state: oregon
prod-cmh:
name: prod-cmh
tier: prod
jurisdiction: us
state: ohio
prod-ams:
name: prod-ams
tier: prod
jurisdiction: eu
state: netherlands
dev:
name: dev
tier: nonprod
jurisdiction: us
state: oregon
test:
name: test
tier: nonprod
jurisdiction: us
state: oregon
stage:
name: stage
tier: nonprod
jurisdiction: us
state: oregon
```
</TabItem>
</Tabs>
This looks like our prototype, we're confident we can iterate over each
environment and get a handle on the configuration values we need.
## Integrating components
The `Environments` data structure unlocks the capability to look up concrete
values specific to a named environment. We'll use this capability to configure
the `podinfo` component in compliance with the regulations of the jurisdiction.
### Configuring the environment
Inject the environment name when we integrate `podinfo` with the platform.
```shell
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: {
podinfoPDX: ProdPodinfo & {_city: "pdx"}
podinfoCMH: ProdPodinfo & {_city: "cmh"}
podinfoAMS: ProdPodinfo & {_city: "ams"}
podinfoDEV: {
name: "podinfo-dev"
path: "components/podinfo"
labels: "app.holos.run/component": "podinfo"
parameters: EnvironmentName: "dev"
}
}
let ProdPodinfo = {
_city: string
name: "podinfo-\(_city)"
path: "components/podinfo"
labels: "app.holos.run/component": "podinfo"
labels: "app.holos.run/tier": "prod"
labels: "app.holos.run/city": _city
parameters: EnvironmentName: "prod-\(_city)"
}
```
```
EOF
```
### Using the environment
Now we can configure `podinfo` based on the jurisdiction of the environment.
```shell
cat <<EOF > components/podinfo/cookie-consent.cue
```
```cue showLineNumbers
package holos
// Schema definition for our configuration.
#Values: {
ui: enableCookieConsent: *true | false
ui: message: string
}
// Map jurisdiction to helm values
JurisdictionValues: {
// Enable cookie consent by default in any jurisdiction.
[_]: #Values
// Disable in the US.
us: ui: enableCookieConsent: false
eu: ui: enableCookieConsent: true
}
// Look up the configuration values associated with the environment name.
Component: Values: JurisdictionValues[Environments[EnvironmentName].jurisdiction]
```
```shell
EOF
```
### Inspecting the BuildPlans
With the above configuration, we can inspect the buildplans for this component.
The prod environment in Amsterdam has cookie consent enabled on line 26.
<Tabs groupId="6EC991F3-F78C-43F1-8A6D-E68D8BDAF58B">
<TabItem value="command" label="Command">
```bash
holos show buildplans --selector app.holos.run/city=ams
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: podinfo-ams
labels:
app.holos.run/city: ams
app.holos.run/component: podinfo
app.holos.run/name: podinfo-ams
app.holos.run/tier: prod
spec:
artifacts:
- artifact: components/podinfo-ams/podinfo-ams.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: podinfo
version: 6.6.2
release: podinfo
repository:
name: podinfo
url: https://stefanprodan.github.io/podinfo
values:
ui:
# highlight-next-line
enableCookieConsent: true
message: Hello World
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: components/podinfo-ams/podinfo-ams.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- helm.gen.yaml
- resources.gen.yaml
- artifact: gitops/podinfo-ams.application.gen.yaml
generators:
- kind: Resources
output: gitops/podinfo-ams.application.gen.yaml
resources:
Application:
podinfo-ams:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo-ams
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/components/podinfo-ams
repoURL: https://github.com/brenix/holos-demo.git
targetRevision: main
```
</TabItem>
</Tabs>
In Portland cookie consent is disabled.
<Tabs groupId="3438335B-1FFC-4838-B8DE-C54B8346CDB4">
<TabItem value="command" label="Command">
```bash
holos show buildplans --selector app.holos.run/city=pdx
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: podinfo-pdx
labels:
app.holos.run/city: pdx
app.holos.run/component: podinfo
app.holos.run/name: podinfo-pdx
app.holos.run/tier: prod
spec:
artifacts:
- artifact: components/podinfo-pdx/podinfo-pdx.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: podinfo
version: 6.6.2
release: podinfo
repository:
name: podinfo
url: https://stefanprodan.github.io/podinfo
values:
ui:
# highlight-next-line
enableCookieConsent: false
message: Hello World
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: components/podinfo-pdx/podinfo-pdx.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- helm.gen.yaml
- resources.gen.yaml
- artifact: gitops/podinfo-pdx.application.gen.yaml
generators:
- kind: Resources
output: gitops/podinfo-pdx.application.gen.yaml
resources:
Application:
podinfo-pdx:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo-pdx
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/components/podinfo-pdx
repoURL: https://github.com/brenix/holos-demo.git
targetRevision: main
```
</TabItem>
</Tabs>
## Concluding Remarks
In this topic we covered how to use a CUE structure to define attributes of prod
and nonprod environments.
1. We passed the environment name as a parameter to each component using a CUE `@tag`.
2. The component definition uses the environment name as a key to get a handle
on attributes. For example, the jurisdiction a service operates within.
3. The example podinfo component uses an additional structure to map
jurisdictions to concrete configuration values.

View File

@@ -0,0 +1,25 @@
---
slug: .
title: Structures
description: Commonly used CUE structures.
sidebar_position: 120
---
import DocCardList from '@theme/DocCardList';
# Structures
This section has self contained articles covering commonly used CUE structures.
These structures are organized and presented as recipes you may adopt and adjust
to your unique organization.
:::important
Structures are defined by Holos Users, unlike the standardized [Core] and
[Author] schemas defined by the Holos Authors.
:::
---
<DocCardList />
[Core]: ../../api/core.md
[Author]: ../../api/author.md

View File

@@ -1 +0,0 @@
# Overview

View File

@@ -1 +0,0 @@
# Setup

View File

@@ -1 +0,0 @@
# Hello Holos

View File

@@ -1 +0,0 @@
# Helm

View File

@@ -1 +0,0 @@
# Kustomize

View File

@@ -1 +0,0 @@
# CUE

View File

@@ -1 +0,0 @@
# Mix Ins

View File

@@ -1 +0,0 @@
# Core API

View File

@@ -1 +0,0 @@
# Author API

277
doc/md/tutorial/cue.mdx Normal file
View File

@@ -0,0 +1,277 @@
---
slug: cue
title: CUE
description: Render component manifests directly from CUE.
sidebar_position: 50
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# CUE
## Overview
This tutorial demonstrates how to add additional resources to a component using
CUE. Holos components often mix in resources, eliminating the need to modify
existing charts or manifests. In this tutorial, we'll add an [ExternalSecret]
resource to the podinfo Helm chart configured in the [Hello Holos] tutorial.
Key concepts:
1. Resources are validated against `#Resources` defined at the root.
2. Custom Resource Definitions need to be imported using Timoni.
3. Helm, Kustomize, and CUE can be mixed together within the same component.
## The Code
### Generating the Structure
Use `holos` to generate a minimal platform directory structure. First, create
and navigate into a blank directory. Then, use the `holos generate platform`
command to generate a minimal platform.
```shell
mkdir holos-cue-tutorial && cd holos-cue-tutorial
holos init platform v1alpha5
```
### Creating the Component
Create the directory for the `podinfo` component. Create an empty file, then add
the following CUE configuration to it.
```bash
mkdir -p components/podinfo
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
// export the component build plan to holos
holos: Component.BuildPlan
// Component is a Helm chart
Component: #Helm & {
Name: "podinfo"
Namespace: "default"
// Add metadata.namespace to all resources with kustomize.
KustomizeConfig: Kustomization: namespace: Namespace
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
```bash
EOF
```
Register the component with the platform.
```bash
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="tutorial-hello-render-manifests">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
</Tabs>
Add and commit the initial configuration.
```bash
git init . && git add . && git commit -m initial
```
### Mixing in Resources
We use the [ComponentConfig] `Resources` field to mix in resources to any
component kind. This field is a convenient wrapper around the core [BuildPlan]
[Resources] [Generator].
Create the mixins.cue file.
```bash
cat <<EOF > components/podinfo/mixins.cue
```
```cue showLineNumbers
package holos
// Component fields are unified with podinfo.cue
Component: {
// Concrete values are defined in podinfo.cue
Name: string
Namespace: string
// Resources represents mix-in resources organized as a struct.
Resources: ExternalSecret: (Name): {
// Name is consistent with the component name.
metadata: name: Name
// Namespace is consistent with the component namespace.
metadata: namespace: Namespace
spec: {
// Ensure the target secret name is consistent.
target: name: metadata.name
// Ensure the name in the SecretStore is consistent.
dataFrom: [{extract: {key: metadata.name}}]
refreshInterval: "30s"
secretStoreRef: kind: "SecretStore"
secretStoreRef: name: "default"
}
}
}
```
```bash
EOF
```
:::important
Holos uses CUE to validate mixed in resources against a schema. The `Resources`
field validates against the `#Resources` definition in [resources.cue].
:::
### Importing CRDs
Holos includes CUE schema definitions for the ExternalSecret Custom Resource
Definition (CRD). These schemas are located in the `cue.mod` directory,
generated by the `holos init platform` command executed at the start of this
tutorial.
To import your own custom resource definitions, use [Timoni]. We imported the
ExternalSecret CRDs embedded in `holos` using the following command.
<Tabs groupId="35B1A1A1-D7DF-4D27-A575-28556E182096">
<TabItem value="command" label="Command">
```bash
timoni mod vendor crds -f https://raw.githubusercontent.com/external-secrets/external-secrets/v0.10.5/deploy/crds/bundle.yaml
```
</TabItem>
<TabItem value="output" label="Output">
```txt
2:22PM INF schemas vendored: external-secrets.io/clusterexternalsecret/v1beta1
2:22PM INF schemas vendored: external-secrets.io/clustersecretstore/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/clustersecretstore/v1beta1
2:22PM INF schemas vendored: external-secrets.io/externalsecret/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/externalsecret/v1beta1
2:22PM INF schemas vendored: external-secrets.io/pushsecret/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/secretstore/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/secretstore/v1beta1
2:22PM INF schemas vendored: generators.external-secrets.io/acraccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/ecrauthorizationtoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/fake/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/gcraccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/githubaccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/password/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/uuid/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/vaultdynamicsecret/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/webhook/v1alpha1
```
</TabItem>
</Tabs>
:::tip
Take a look at
[cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue] to see
the imported definitions.
:::
Once imported, the final step is to add the resource kind to the `#Resources`
struct. Typically, this is done by creating a new file that CUE unifies with the
existing [resources.cue] file.
## Reviewing Changes
Render the platform with the `ExternalSecret` mixed into the podinfo component.
```shell
holos render platform
```
Take a look at the diff to see the mixed in `ExternalSecret`.
```shell
git diff deploy
```
```diff
diff --git a/deploy/components/podinfo/podinfo.gen.yaml b/deploy/components/podinfo/podinfo.gen.yaml
index 6e4aec0..f79e9d0 100644
--- a/deploy/components/podinfo/podinfo.gen.yaml
+++ b/deploy/components/podinfo/podinfo.gen.yaml
@@ -112,3 +112,19 @@ spec:
volumes:
- emptyDir: {}
name: data
+---
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+ name: podinfo
+ namespace: default
+spec:
+ dataFrom:
+ - extract:
+ key: podinfo
+ refreshInterval: 30s
+ secretStoreRef:
+ kind: SecretStore
+ name: default
+ target:
+ name: podinfo
```
We saw how to mix in resources using the `Resources` field of the
[ComponentConfig]. This approach works for every kind of component in Holos,
allowing seamless integration without needing to fork the upstream Helm chart.
This way, we can easily update to new podinfo versions as they're released.
## Trying Locally
Optionally, apply the manifests rendered by Holos to a [Local Cluster] for
testing.
## Next Steps
This tutorial uses the `#Resources` structure to map resource kinds to their
schema definitions in CUE. This structure is defined in `resources.cue` at the
root of the tree. Take a look at [resources.cue] to see this mapping structure.
[Local Cluster]: ../topics/local-cluster.mdx
[ExternalSecret]: https://external-secrets.io/latest/api/externalsecret/
[Artifact]: ../api/core.md#Artifact
[Resources]: ../api/core.md#Resources
[Generator]: ../api/core.md#Generator
[Hello Holos]: ./hello-holos.mdx
[cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue#L13
[ComponentConfig]: ../api/author.md#ComponentConfig
[timoni]: https://timoni.sh/install/
[resources.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/v1alpha5/resources.cue#L33

View File

@@ -0,0 +1,376 @@
---
slug: hello-holos
title: Hello Holos
description: Configure a simple Hello World service with Holos.
sidebar_position: 30
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
import PlatformSequence from '@site/src/diagrams/render-platform-sequence.mdx';
import ComponentSequence from '@site/src/diagrams/render-component-sequence.mdx';
# Hello Holos
## Overview
Like a traditional "Hello World" program, we'll start by configuring the
[podinfo Helm chart][podinfo] to output a greeting from a Kubernetes Service.
This introduces the core concept of wrapping Helm charts as Holos Components.
## Implementation
### Initialize Platform Structure
Create and initialize a minimal platform:
```shell
mkdir holos-tutorial && cd holos-tutorial
holos init platform v1alpha5
```
The resulting directory structure:
<Tabs groupId="80D04C6A-BC83-44D0-95CC-CE01B439B159">
<TabItem value="tree" label="Tree">
```text showLineNumbers
holos-tutorial/
├── components/
│   └── podinfo/
│   └── podinfo.cue
├── cue.mod/
├── platform/
│   ├── platform.gen.cue
│   └── podinfo.cue
├── resources.cue
├── schema.cue
└── tags.cue
```
</TabItem>
<TabItem value="details" label="Details">
<div style={{display: "flex"}}>
<div>
```text showLineNumbers
holos-tutorial/
├── components/
│   └── podinfo/
│   └── podinfo.cue
├── cue.mod/
├── platform/
│   ├── platform.gen.cue
│   └── podinfo.cue
├── resources.cue
├── schema.cue
└── tags.cue
```
</div>
<div>
- **Line 1** The platform root is the `holos-tutorial` directory we created.
- **Line 2** This tutorial places components in `components/`. They may reside
anywhere.
- **Line 3** A component is a collection of `*.cue` files at a path.
- **Line 4** We'll create this file and configure the podinfo helm chart in the
next section.
- **Line 5** The CUE module directory. Schema definitions for Kubernetes and
Holos resources reside within the `cue.mod` directory.
- **Line 6** The platform directory is the **main entrypoint** for the `holos
render platform` command.
- **Line 7** `platform.gen.cue` is initialized by `holos init platform` and
contains the Platform spec.
- **Line 8** `podinfo.cue` integrates podinfo with the platform by adding the
component to the platform spec. We'll add ths file after the next section.
- **Line 9** `resources.cue` Defines the Kubernetes resources available to
manage in CUE.
- **Line 10** `schema.cue` Defines the configuration common to all component
kinds.
- **Line 11** `tags.cue` Defines where component parameter values are injected
into the overall platform configuration. We don't need to be concerned with
this file until we cover component parameters.
- **Lines 9-11** Initialized by `holos init platform`, user editable after
initialization.
</div>
</div>
</TabItem>
</Tabs>
### Create the Component
Configure the `podinfo` component:
```bash
mkdir -p components/podinfo
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: HelmChart.BuildPlan
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
// Holos marshals Values into values.yaml for Helm.
Values: {
// message is a string with a default value. @tag indicates a value may
// be injected from the platform spec component parameters.
ui: {
message: string | *"Hello World" @tag(greeting, type=string)
}
}
}
```
```bash
EOF
```
:::important
Like Go packages, CUE loads all `*.cue` files in the component directory to
define the component.
:::
:::note
CUE recursively loads `*.cue` files from the component directory up to the
platform root. For example, `#Helm` referenced on line 6 is defined in
root-level `schema.cue`.
:::
### Add to Platform
Register the `podinfo` component in `platform/podinfo.cue`:
```bash
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
// Inject a value into the component.
parameters: greeting: "Hello Holos!"
}
```
```bash
EOF
```
:::tip
Parameter names are unrestricted, except for the reserved `holos_` prefix.
:::
## Generate Manifests
Render the `podinfo` configuration:
<Tabs groupId="E150C802-7162-4FBF-82A7-77D9ADAEE847">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
</Tabs>
Holos executes `helm template` with locally cached charts to generate:
```txt
deploy/components/podinfo/podinfo.gen.yaml
```
<Tabs groupId="0E9C231D-D0E8-410A-A4A0-601842A086A6">
<TabItem value="service" label="Service">
```yaml showLineNumbers
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
helm.sh/chart: podinfo-6.6.2
name: podinfo
spec:
ports:
- name: http
port: 9898
protocol: TCP
targetPort: http
- name: grpc
port: 9999
protocol: TCP
targetPort: grpc
selector:
app.kubernetes.io/name: podinfo
type: ClusterIP
```
</TabItem>
<TabItem value="deployment" label="Deployment">
```yaml showLineNumbers
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
helm.sh/chart: podinfo-6.6.2
name: podinfo
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: podinfo
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
prometheus.io/port: "9898"
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: podinfo
spec:
containers:
- command:
- ./podinfo
- --port=9898
- --cert-path=/data/cert
- --port-metrics=9797
- --grpc-port=9999
- --grpc-service-name=podinfo
- --level=info
- --random-delay=false
- --random-error=false
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!
- name: PODINFO_UI_COLOR
value: '#34577c'
image: ghcr.io/stefanprodan/podinfo:6.6.2
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/healthz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: podinfo
ports:
- containerPort: 9898
name: http
protocol: TCP
- containerPort: 9797
name: http-metrics
protocol: TCP
- containerPort: 9999
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/readyz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits: null
requests:
cpu: 1m
memory: 16Mi
volumeMounts:
- mountPath: /data
name: data
terminationGracePeriodSeconds: 30
volumes:
- emptyDir: {}
name: data
```
</TabItem>
</Tabs>
Holos renders the component with the greeting injected from the platform spec.
```shell
grep -B2 Hello deploy/components/podinfo/podinfo.gen.yaml
```
```yaml
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!
```
## Breaking it down
We run `holos render platform` because the CUE files in the platform directory
export a [Platform] resource to `holos`.
:::important
The `platform/` directory is the default entry point to the platform rendering
process. Override with `--platform <dir>`.
:::
Components are the building blocks of a Platform. The `platform/podinfo.cue`
file integrates the `podinfo` component with the Platform.
Holos requires two fields to integrate a component with the platform:
1. A unique name for the component.
2. The component path to the directory containing the CUE files that export a
`BuildPlan` defining the component.
Component parameters are optional and allow re-use of the same component.
<Tabs groupId="67C1EE71-3EA8-4568-9F6D-0072BA09FF12">
<TabItem value="overview" label="Rendering Overview">
Take a look at the other tabs for more detailed sequence diagrams.
<RenderingOverview />
</TabItem>
<TabItem value="platform" label="Platform Sequence">
<PlatformSequence />
</TabItem>
<TabItem value="component" label="Component Sequence">
<ComponentSequence />
</TabItem>
</Tabs>
## Next Steps
We've shown how to integrate one Helm chart into the Platform, but we haven't
yet covered multiple Helm charts. Continue with the next tutorial to learn how
Holos makes it easy to inject values into multiple components safely and
efficiently.
[podinfo]: https://github.com/stefanprodan/podinfo
[CUE Module]: https://cuelang.org/docs/reference/modules/
[CUE Tags]: https://cuelang.org/docs/howto/inject-value-into-evaluation-using-tag-attribute/
[Platform]: ../api/author.md#Platform

View File

@@ -0,0 +1,531 @@
---
slug: helm-values
title: Helm Values
description: Holos provides values to multiple charts easily and safely.
sidebar_position: 40
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import YouTube from '@site/src/components/YouTube';
<head>
<meta property="og:title" content="Helm Values | Holos" />
<meta property="og:image" content="https://holos.run/img/cards/guides-helm-2.png" />
</head>
# Helm Values
## Overview
Holos simplifies integrating multiple Helm charts by adding valuable
capabilities to Helm and Kustomize:
1. Inject the same value into multiple charts more safely than using Helm alone.
2. Add strong type checking and validation for Helm input values.
3. Implement the [rendered manifests pattern].
In this tutorial, we'll manage the [prometheus] and [blackbox] Helm charts. By
default, the upstream `values.yaml` files are misconfigured, causing Prometheus
to connect to Blackbox at the wrong host and port.
## The Video
The video below enhances this tutorial by offering greater detail on the issue
of poorly integrated Helm charts and the solution we've provided. If you're
looking for a deeper explanation of the code being presented, this video is a great
resource.
{/* cspell:disable-next-line */}
<YouTube id="PSdceGlhHGo"/>
## The Code
### Generating the structure
Use `holos` to generate a minimal platform directory structure. First, create
and navigate into a blank directory, then use the `holos init platform` command:
```shell
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5
```
Make an initial commit to track changes:
```bash
git init . && git add . && git commit -m "initial commit"
```
### Managing the Components
Create the `prometheus` and `blackbox` component directories, then add each of
the following file contents.
```bash
mkdir -p components/prometheus components/blackbox
```
<Tabs groupId="D15A3008-1EFC-4D34-BED1-15BC0C736CC3">
<TabItem value="prometheus.cue" label="prometheus.cue">
```bash
cat <<EOF > components/prometheus/prometheus.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
```
```bash
EOF
```
</TabItem>
<TabItem value="blackbox.cue" label="blackbox.cue">
```bash
cat <<EOF > components/blackbox/blackbox.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
```
```bash
EOF
```
</TabItem>
</Tabs>
### Register the Components
Register the components with the platform by adding the following file to the platform directory.
```bash
cat <<EOF > platform/prometheus.cue
```
```cue showLineNumbers
package holos
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="33D6BFED-62D8-4A42-A26A-F3121D57C4E5">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached prometheus-blackbox-exporter 9.0.1
rendered blackbox in 3.825430417s
cached prometheus 25.27.0
rendered prometheus in 4.840089667s
rendered platform in 4.840137792s
```
</TabItem>
</Tabs>
Commit the results.
<Tabs groupId="446CC550-A634-45C0-BEC7-992E5C56D4FA">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add blackbox and prometheus'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main b5df111] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue
```
</TabItem>
</Tabs>
### Importing Helm Values
Holos renders Helm charts with their default values. We can import these default
values into CUE to work with them as structured data instead of text markup.
```bash
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml
```
```bash
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
```
These commands convert the YAML data into CUE code and nest the values under the
`Values` field of the `Helm` struct.
:::important
CUE unifies `values.cue` with the other `\*.cue` files in the same directory.
:::
Render the platform using `holos render platform` and commit the results.
<Tabs groupId="BDDCD65A-2E9D-4BA6-AAE2-8099494D5E4B">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered blackbox in 365.936792ms
rendered prometheus in 371.855875ms
rendered platform in 372.109916ms
```
</TabItem>
</Tabs>
<Tabs groupId="1636C619-258E-4D49-8052-F64B588C9177">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'import values'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 52e90ea] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue
```
</TabItem>
</Tabs>
### Managing Common Configuration
To manage shared configuration for both Helm charts, define a structure that
holds the common configuration values. Place this configuration in the
`components` directory to ensure it is accessible to all components.
```bash
cat <<EOF > components/blackbox.cue
```
```cue showLineNumbers
package holos
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
Blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}
```
```bash
EOF
```
:::important
1. CUE loads and unifies all `*.cue` files from the root directory containing
`cue.mod` to the leaf component path directory.
2. CUE validates types _and_ constraints. Validation with CUE is better than
languages with only type checking.
:::
Add and commit the configuration.
<Tabs groupId="A738CCE4-F0C6-4CC7-BE1F-2B92F0E86FDC">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add blackbox configuration'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 1adcd08] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 components/blackbox.cue
```
</TabItem>
</Tabs>
### Using Common Configuration Across Components
Referencing common configuration across multiple components is straightforward
and reliable using Holos and CUE.
To apply the common configuration, patch the two `values.cue` files, or manually
edit them to reference `Blackbox.host` and `Blackbox.port`.
<Tabs groupId="5FFCE892-B8D4-4F5B-B2E2-39EC9E9F87A4">
<TabItem value="command" label="Command">
```bash
patch -p1 < values.patch
```
</TabItem>
<TabItem value="patch" label="values.patch">
```diff
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,8 @@
package holos
Helm: Values: {
+ fullnameOverride: Blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +194,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: Blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1083,7 +1083,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(Blackbox.host):\(Blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'
```
</TabItem>
</Tabs>
:::important
Both charts now use the same values in lock step. Holos and CUE integrate them
safely and easily.
:::
Remove the patch file, then commit the changes.
<Tabs groupId="6498B00E-FADA-4EB2-885C-808F1D22E04D">
<TabItem value="command" label="Command">
```bash
rm values.patch
```
```bash
git add . && git commit -m 'integrate blackbox and prometheus together'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 4221803] integrate blackbox and prometheus together
2 files changed, 4 insertions(+), 2 deletions(-)
```
</TabItem>
</Tabs>
## Reviewing Changes
Holos makes it easy to view and review platform-wide changes. Render the
platform to observe how both Prometheus and Blackbox update in sync.
<Tabs groupId="E7F6D8B1-22FA-4075-9B44-D9F2815FE0D3">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms
```
</TabItem>
</Tabs>
Changes are easily visible in version control.
<Tabs groupId="9789A0EF-24D4-4FB9-978A-3895C2778789">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target
```
</TabItem>
</Tabs>
From the diff, we can see this change will:
1. Reconfigure the Blackbox Exporter host from `prometheus-blackbox-exporter` to `blackbox`.
2. Have no effect on the Blackbox service port, as it was already using the default `9115`.
3. Reconfigure Prometheus to query the Blackbox Exporter at the correct host and port, `blackbox:9115`.
Without this change, Prometheus incorrectly assumed Blackbox was listening at
`blackbox` on port `80` when it was actually listening at
`prometheus-blackbox-exporter` on port `9115`. Going forward, changing the
Blackbox host or port will reconfigure both charts correctly.
Commit the changes and proceed to deploy them.
<Tabs groupId="F8C9A98D-DE1E-4EF6-92C1-017A9166F6C7">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'render integrated blackbox and prometheus manifests'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 67efe0d] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)
```
</TabItem>
</Tabs>
## Trying Locally
Optionally, apply the manifests rendered by Holos to a [Local Cluster].
## Next Steps
In this tutorial, we learned how Holos simplifies the holistic integration of
the [prometheus] and [blackbox] charts, ensuring they are configured
consistently. By using Holos, we overcome the limitations of relying solely on
Helm, which lacks an effective method to configure both charts to use the same
service endpoint.
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[prometheus]: https://github.com/prometheus-community/helm-charts/tree/prometheus-25.27.0/charts/prometheus
[blackbox]: https://github.com/prometheus-community/helm-charts/tree/prometheus-blackbox-exporter-9.0.1/charts/prometheus-blackbox-exporter
[httpbin]: https://github.com/mccutchen/go-httpbin/tree/v2.15.0
[Config Schema]: #config-schema
[Technical Overview]: ./overview.mdx
[Local Cluster]: ../topics/local-cluster.mdx

View File

@@ -0,0 +1,413 @@
---
slug: kustomize
title: Kustomize
description: Holos makes it easy to Kustomize configuration.
sidebar_position: 45
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Kustomize
## Overview
In the previous tutorial, we learned how Holos simplifies the holistic
integration of the [prometheus] and [blackbox] charts, ensuring they are
configured in sync.
In this tutorial, we'll go a step further by integrating the [httpbin] service
with Prometheus and Blackbox to automatically probe for availability.
We'll also explore how Holos manages [kustomize] bases, similar to the Helm kind
covered in the [Helm Values] tutorial.
## The Code
### Generating the structure
<Tabs>
<TabItem value="optional" label="Optional">
:::note Skip this step if you completed the [Helm Values] tutorial.
Otherwise click the **Generate** tab to generate a blank platform now.
:::
</TabItem>
<TabItem value="generate" label="Generate">
Use `holos` to generate a minimal platform directory structure. First, create
and navigate into a blank directory. Then, run the `holos init platform`
command.
```shell
mkdir holos-kustomize-tutorial
cd holos-kustomize-tutorial
holos init platform v1alpha5
```
Make a commit to track changes.
```bash
git init . && git add . && git commit -m initial
```
</TabItem>
</Tabs>
### Managing the Component
Create the `httpbin` component directory, and add the `httpbin.cue` and
`httpbin.yaml` files to it for configuration and setup.
<Tabs groupId="800C3AE7-E7F8-4AFC-ABF1-6AFECD945958">
<TabItem value="setup" label="Setup">
```bash
mkdir -p components/httpbin
```
</TabItem>
<TabItem value="components/httpbin/httpbin.cue" label="httpbin.cue">
```bash
cat <<EOF > components/httpbin/httpbin.cue
```
```cue showLineNumbers
package holos
// Produce a Kustomize BuildPlan for Holos
holos: Kustomize.BuildPlan
// https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/README.md
Kustomize: #Kustomize & {
KustomizeConfig: {
// Files tells Holos to copy the file from the component path to the
// temporary directory Holos uses for BuildPlan execution.
Files: {
"httpbin.yaml": _
}
CommonLabels: {
"app.kubernetes.io/name": "httpbin"
}
// Kustomization represents a kustomization.yaml file in CUE. Holos
// marshals this field into a `kustomization.yaml` while processing a
// BuildPlan. See
// https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
Kustomization: {
images: [{name: "mccutchen/go-httpbin"}]
// Use a hidden field to compose patches easily with a struct. Hidden
// fields are not included in exported structures.
_patches: {}
// Convert the hidden struct to a list.
patches: [for x in _patches {x}]
}
}
}
```
```bash
EOF
```
</TabItem>
<TabItem value="components/httpbin/httpbin.yaml" label="httpbin.yaml">
```bash
cat <<EOF > components/httpbin/httpbin.yaml
```
```yaml showLineNumbers
# https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/resources.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
template:
spec:
containers:
- name: httpbin
image: mccutchen/go-httpbin
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /status/200
port: http
readinessProbe:
httpGet:
path: /status/200
port: http
resources: {}
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
appProtocol: http
```
```bash
EOF
```
</TabItem>
</Tabs>
Holos knows the `httpbin.yaml` file is part of the BuildPlan because of the
`KustomizeConfig: Files: "httpbin.yaml": _` line in the `httpbin.cue`.
### Register the Components
Register `httpbin` with the platform by adding the following file to the
platform directory.
```bash
cat <<EOF > platform/httpbin.cue
```
```cue showLineNumbers
package holos
Platform: Components: {
httpbin: {
name: "httpbin"
path: "components/httpbin"
}
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="B120D5D1-0EAB-41E0-AD21-15526EBDD53D">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered httpbin in 707.554666ms
rendered platform in 707.9845ms
```
</TabItem>
</Tabs>
Commit the results.
<Tabs groupId="446CC550-A634-45C0-BEC7-992E5C56D4FA">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add httpbin'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main c05f9ef] add httpbin
4 files changed, 118 insertions(+)
create mode 100644 components/httpbin/httpbin.cue
create mode 100644 components/httpbin/httpbin.yaml
create mode 100644 deploy/components/httpbin/httpbin.gen.yaml
create mode 100644 platform/httpbin.cue
```
</TabItem>
</Tabs>
### Inspecting the Build Plan
We can see the [BuildPlan] exported to `holos` by the `holos:
Kustomize.BuildPlan` line in `httpbin.cue`. Holos processes this build plan to
produce the fully rendered manifests.
<Tabs groupId="DD697D65-5BEC-4B92-BB33-59BE4FEC112F">
<TabItem value="command" label="Command">
```bash
holos cue export --expression holos --out=yaml ./components/httpbin
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
- kind: File
output: httpbin.yaml
file:
source: httpbin.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
- httpbin.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/name: httpbin
patches: []
images:
- name: mccutchen/go-httpbin
resources:
- resources.gen.yaml
- httpbin.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
source:
component:
name: no-name
path: no-path
parameters: {}
```
</TabItem>
</Tabs>
### Transforming Manifests
Review the BuildPlan exported in the previous command:
1. The [File Generator] copies the plain `httpbin.yaml` file into the build.
2. The [Kustomize Transformer] uses `httpbin.yaml` as an input resource.
3. The final artifact is the output from Kustomize.
This BuildPlan transforms the raw YAML by labeling all of the resources with
`"app.kubernetes.io/name": "httpbin"` using the [KustomizeConfig] `CommonLabels`
field.
To complete the integration with Prometheus, annotate the Service with
`prometheus.io/probe: "true"`. Holos makes this easier with CUE, so there's no
need to edit any YAML files manually.
Add a new `patches.cue` file to the `httpbin` component with the following
content.
```bash
cat <<EOF > components/httpbin/patches.cue
```
```cue showLineNumbers
package holos
import "encoding/yaml"
// Mix in a Kustomize patch to the configuration.
Kustomize: KustomizeConfig: Kustomization: _patches: {
probe: {
target: kind: "Service"
target: name: "httpbin"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/prometheus.io~1probe"
value: "true"
}])
}
}
```
```bash
EOF
```
:::note
We use a hidden `_patches` field to easily unify data into a struct, then
convert the struct into a list for export.
:::
## Reviewing Changes
Render the platform to see the result of the kustomization patch.
<Tabs groupId="5D1812DD-8E7B-4F97-B349-275214F38B6E">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered httpbin in 197.030208ms
rendered platform in 197.416416ms
```
</TabItem>
</Tabs>
Holos is configuring Kustomize to patch the plain `httpbin.yaml` file with the
annotation.
<Tabs groupId="3D80279E-8EDE-4B3E-9269-50F5D1C1CA42">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/components/httpbin/httpbin.gen.yaml b/deploy/components/httpbin/httpbin.gen.yaml
index 298b9a8..a16bd1a 100644
--- a/deploy/components/httpbin/httpbin.gen.yaml
+++ b/deploy/components/httpbin/httpbin.gen.yaml
@@ -1,6 +1,8 @@
apiVersion: v1
kind: Service
metadata:
+ annotations:
+ prometheus.io/probe: "true"
labels:
app.kubernetes.io/name: httpbin
name: httpbin
```
</TabItem>
</Tabs>
Add and commit the final changes.
<Tabs groupId="54C335C8-B382-4277-AE87-0D6556921955">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'annotate httpbin for prometheus probes'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 6eeeadb] annotate httpbin for prometheus probes
2 files changed, 3 insertions(+), 1 deletion(-)
```
</TabItem>
</Tabs>
## Trying Locally
Optionally, apply the manifests rendered by Holos to a [Local Cluster] for
testing and validation.
## Next Steps
In this tutorial, we learned how Holos simplifies managing [httpbin], which is
distributed as a Kustomize base. We used a Kustomize component similar to the
Helm component covered previously. Holos provides a straightforward way to
customize any component, demonstrated by patching an annotation onto the
`httpbin` Service.
Continue with the tutorial to learn how Holos facilitates certificate management
and makes services accessible outside of a cluster.
[httpbin]: https://github.com/mccutchen/go-httpbin/tree/v2.15.0
[prometheus]: https://github.com/prometheus-community/helm-charts/tree/prometheus-25.27.0/charts/prometheus
[blackbox]: https://github.com/prometheus-community/helm-charts/tree/prometheus-blackbox-exporter-9.0.1/charts/prometheus-blackbox-exporter
[kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
[Helm Values]: ./helm-values.mdx
[File Generator]: ../api/core.md#File
[Kustomize Transformer]: ../api/core.md#Kustomize
[BuildPlan]: ../api/core.md#BuildPlan
[KustomizeConfig]: ../api/author.md#KustomizeConfig
[Local Cluster]: ../topics/local-cluster.mdx

View File

@@ -0,0 +1,80 @@
---
slug: overview
title: Overview
description: Learn how Holos integrates software into a holistic platform.
sidebar_position: 10
---
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
import YouTube from '@site/src/components/YouTube';
# Tutorial
## Overview
Holos is a configuration management tool for Kubernetes implementing the
[rendered manifests pattern]. It handles configurations ranging from single
resources to multi-cluster platforms across regions.
{/* truncate */}
Key components:
- Platform schemas defining component integration
- Building blocks unifying Helm, Kustomize and Kubernetes configs with CUE
- BuildPlan pipeline for generating, transforming and validating manifests
<RenderingOverview />
{/* TODO: Replace this with the Advantages diagram we talked about. */}
## Video
The video below offers a basic overview of Holos by walking you through the
tooling gap at the Kubernetes integration layer and demonstrating how to
integrate multiple Helm charts using data from CUE.
{/* cspell:disable-next-line */}
<YouTube id="PSdceGlhHGo"/>
## Holos' role in your organization
Platform engineers use Holos to generate Kubernetes manifests, both locally and
in CI pipelines. The manifests are committed to version control and deployed via
GitOps tools like ArgoCD or Flux.
Holos integrates seamlessly with existing Helm charts, Kustomize bases, and
other version-controlled configurations.
## Advantages of Holos
### Safe
Holos leverages [CUE] for strong typing and validation of configuration data,
ensuring consistent output from Helm and other tools.
### Consistent
A unified pipeline processes all configurations - whether from CUE, Helm, or
Kustomize - through the same well-defined stages.
### Flexible
Composable building blocks for generation, transformation, validation and
integration let teams assemble workflows that match their needs.
The core is intentionally unopinionated about platform configuration patterns.
Common needs like environments and clusters are provided as customizable
[topics] recipes rather than enforced structures.
## Getting Help
Get support through our [Discord] channel or [GitHub discussions]. Configuration
challenges arise at all experience levels - we welcome your questions and are
here to help.
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[CUE]: https://cuelang.org/
[Discord]: https://discord.gg/JgDVbNpye7
[GitHub discussions]: https://github.com/holos-run/holos/discussions
[Why CUE for Configuration]: https://holos.run/blog/why-cue-for-configuration/
[topics]: ../topics.mdx

104
doc/md/tutorial/setup.mdx Normal file
View File

@@ -0,0 +1,104 @@
---
slug: setup
title: Setup
description: Install Holos and build the initial structure.
sidebar_position: 20
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
# Setup
## Installing
Holos is a single executable that can be installed via:
<Tabs groupId="FE2C74C8-B3A3-4AEA-BBD3-F57FAA654B6F">
<TabItem value="brew" label="macOS">
```bash
brew install holos-run/tap/holos
```
</TabItem>
<TabItem value="linux" label="Linux">
Download holos from the [releases] page and place the executable into your shell
path.
</TabItem>
<TabItem value="win" label="Windows">
Download holos from the [releases] page and place the executable into your shell
path.
</TabItem>
<TabItem value="go" label="Go">
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
</TabItem>
</Tabs>
### Completion
:::tip
Completion is automatically enabled if [brew shell
completion](https://docs.brew.sh/Shell-Completion) is also enabled.
:::
<Tabs groupId="65F79D28-2E57-4A90-8EBA-3D8758C80233">
<TabItem value="zsh" label="zsh">
Add the following to `~/.zshrc` if not already present to initialize zsh completion.
```bash
autoload -Uz compinit
compinit
```
Then load holos completion after zsh completion has been initialized.
```bash
source <(holos completion zsh)
```
</TabItem>
<TabItem value="bash" label="bash">
```bash
source <(holos completion bash)
```
</TabItem>
<TabItem value="fish" label="fish">
```bash
source <(holos completion fish)
```
</TabItem>
<TabItem value="powershell" label="powershell">
```bash
holos completion powershell | Invoke-Expression
```
</TabItem>
</Tabs>
### Dependencies
Install these tools to use Holos's full capabilities:
- [Helm] for chart management and rendering
- [Kubectl] for [kustomize] operations
:::note
Holos is tested with Helm `v3.16.2`. If you see `Error: chart requires
kubeVersion` errors, try upgrading Helm.
:::
## Next Steps
With your platform structure initialized, proceed to [Hello Holos] to learn Helm
chart management.
[Helm]: https://github.com/helm/helm/releases
[Kubectl]: https://kubernetes.io/docs/tasks/tools/
[kustomize]: https://kustomize.io/
[Local Cluster]: ../topics/local-cluster.mdx
[Hello Holos]: ./hello-holos.mdx
[releases]: https://github.com/holos-run/holos/releases
[k3d]: https://k3d.io/
[OrbStack]: https://docs.orbstack.dev/install
[Docker]: https://docs.docker.com/get-started/get-docker/

View File

@@ -0,0 +1,428 @@
---
slug: validators
title: Validators
description: Validate rendered manifests against policy definitions.
sidebar_position: 60
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
# Validators
## Overview
Sometimes Helm charts render Secrets we do not wanted committed to version
control for security. Helm charts often render incorrect manifests, even if
they're accepted by the api server. For example, passing `null` to collection
fields. We'll solve both of these issues using a [Validator] to block artifacts
with a Secret resource, and verifying the artifact against Kubernetes type
definitions.
1. If a Helm chart renders a Secret, Holos errors before writing the artifact
and suggests an ExternalSecret instead.
2. Each resource is validated against a field named by the value of the kind
field. For example, a `kind: Secret` resource validates against `secret: {}` in
CUE. `kind: Deployment` validates against `deployment: {}` in CUE.
3. The final artifact is validated, covering the output of all generators and
transformers.
<RenderingOverview />
## The Code
### Generating the Structure
Use `holos` to generate a minimal platform directory structure. First, create
and navigate into a blank directory. Then, use the `holos generate platform`
command to generate a minimal platform.
```shell
mkdir holos-validators-tutorial && cd holos-validators-tutorial
holos init platform v1alpha5
```
### Creating the Component
Create the directory for the `podinfo` component. Create an empty file, then add
the following CUE configuration to it.
```bash
mkdir -p components/podinfo
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
// export the component build plan to holos
holos: Component.BuildPlan
// Component is a Helm chart
Component: #Helm & {
Name: "podinfo"
Namespace: "default"
// Add metadata.namespace to all resources with kustomize.
KustomizeConfig: Kustomization: namespace: Namespace
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
```bash
EOF
```
Register the component with the platform.
```bash
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="tutorial-hello-render-manifests">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
</Tabs>
Add and commit the initial configuration.
```bash
git init . && git add . && git commit -m initial
```
### Define the Valid Schema
We'll use a CUE package named `policy` so the entire platform configuration in
package `holos` isn't loaded every time we validate an artifact.
Create `policy/validation-schema.cue` with the following content.
```shell
mkdir -p policy
cat <<EOF > policy/validation-schema.cue
```
```cue showLineNumbers
package policy
import apps "k8s.io/api/apps/v1"
// Organize by kind then name to avoid conflicts.
kind: [KIND=string]: [NAME=string]: {...}
// Useful when one component manages the same resource kind and name across
// multiple namespaces.
let KIND = kind
namespace: [NS=string]: KIND
// Block Secret resources. kind will not unify with "Secret"
kind: secret: [NAME=string]: kind: "Use an ExternalSecret instead. Forbidden by security policy. secret/\(NAME)"
// Validate Deployment against Kubernetes type definitions.
kind: deployment: [_]: apps.#Deployment
```
```shell
EOF
```
### Configuring Validators
Configure the Validators [ComponentConfig] field to configure each [BuildPlan]
to validate the rendered [Artifact] files.
```shell
cat <<EOF > validators.cue
```
```cue showLineNumbers
package holos
// Configure all component kinds to validate against the policy directory.
#ComponentConfig: Validators: cue: {
kind: "Command"
// Note --path maps each resource to a top level field named by the kind.
command: args: [
"holos",
"cue",
"vet",
"./policy",
"--path=\"namespace\"",
"--path=metadata.namespace",
"--path=strings.ToLower(kind)",
"--path=metadata.name",
]
}
```
```shell
EOF
```
### Patching Errors
Render the platform to see validation fail. The podinfo chart has no Secret,
but it produces an invalid Deployment because it sets the container resource
limits field to `null`.
```shell
holos render platform
```
```txt
deployment.spec.template.spec.containers.0.resources.limits: conflicting values null and {[string]:"k8s.io/apimachinery/pkg/api/resource".#Quantity} (mismatched types null and struct):
./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:355:9
./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:376:12
./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2840:11
./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2968:14
./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:15
./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:18
./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:5027:9
./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:6407:16
./policy/validation-schema.cue:9:13
../../../../../var/folders/22/T/holos.validate1636392304/components/podinfo/podinfo.gen.yaml:104:19
could not run: terminating because of errors
could not run: could not validate podinfo path ./components/podinfo: could not run command: holos cue vet ./policy --path strings.ToLower(kind) /var/folders/22/T/holos.validate1636392304/components/podinfo/podinfo.gen.yaml: exit status 1 at builder/v1alpha5/builder.go:411
could not run: could not render component: could not run command: holos --log-level info --log-format console render component --inject holos_component_name=podinfo --inject holos_component_path=components/podinfo ./components/podinfo: exit status 1 at cli/render/render.go:155
```
We'll use a [Kustomize] patch [Transformer] to replace the `null` limits field
with a valid equivalent value.
:::important
This configuration is defined in CUE, not YAML, even though we're configuring a
Kustomize patch transformer. CUE gives us access to the unified platform
configuration.
:::
```shell
cat <<EOF > components/podinfo/patch.cue
```
```cue showLineNumbers
package holos
import "encoding/yaml"
Component: KustomizeConfig: Kustomization: {
_patches: limits: {
target: kind: "Deployment"
patch: yaml.Marshal([{
op: "test"
path: "/spec/template/spec/containers/0/resources/limits"
value: null
}, {
op: "replace"
path: "/spec/template/spec/containers/0/resources/limits"
value: {}
}])
}
patches: [for x in _patches {x}]
}
```
```shell
EOF
```
Now the platform renders.
<Tabs groupId="3A050092-8E56-49D4-84A9-71E544A21276">
<TabItem value="command" label="Command">
```bash
holos render platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered podinfo in 181.875083ms
rendered platform in 181.975833ms
```
</TabItem>
</Tabs>
## Inspecting the BuildPlan
The BuildPlan patches the output of the upstream helm chart without modifying
it, then validates the artifact against the Kubernetes type definitions.
<Tabs groupId="1DAB4C46-0793-4CCA-8930-7B2E60BDA1BE">
<TabItem value="command" label="Command">
```bash
holos show buildplans
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: podinfo
spec:
artifacts:
- artifact: components/podinfo/podinfo.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: podinfo
version: 6.6.2
release: podinfo
repository:
name: podinfo
url: https://stefanprodan.github.io/podinfo
values: {}
namespace: default
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: components/podinfo/podinfo.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: default
patches:
- patch: |
- op: test
path: /spec/template/spec/containers/0/resources/limits
value: null
- op: replace
path: /spec/template/spec/containers/0/resources/limits
value: {}
target:
kind: Deployment
name: ""
resources:
- helm.gen.yaml
- resources.gen.yaml
validators:
- kind: Command
inputs:
- components/podinfo/podinfo.gen.yaml
command:
args:
- holos
- cue
- vet
- ./policy
- --path
- strings.ToLower(kind)
```
</TabItem>
</Tabs>
## Catching Mistakes
Suppose a teammate downloads a helm chart that includes a Secret unbeknown to
them. Holos catches the problem and suggests an ExternalSecret instead.
Mix in a Secret to see what happens
```shell
cat <<EOF > components/podinfo/secret.cue
```
```cue showLineNumbers
package holos
Component: Resources: Secret: example: metadata: name: "example"
```
```shell
EOF
```
Render the platform to see the error.
```shell
holos render platform
```
```txt
secret.kind: conflicting values "Use an ExternalSecret instead. Forbidden by security policy." and "Secret":
./policy/validation-schema.cue:6:15
../../../../../var/folders/22/T/holos.validate2549739170/components/podinfo/podinfo.gen.yaml:1:7
could not run: terminating because of errors
could not run: could not validate podinfo path ./components/podinfo: could not run command: holos cue vet ./policy --path strings.ToLower(kind) /var/folders/22/T/holos.validate2549739170/components/podinfo/podinfo.gen.yaml: exit status 1 at builder/v1alpha5/builder.go:411
could not run: could not render component: could not run command: holos --log-level info --log-format console render component --inject holos_component_name=podinfo --inject holos_component_path=components/podinfo ./components/podinfo: exit status 1 at cli/render/render.go:155
```
:::important
Holos quickly returns an error if validated artifacts have a Secret.
:::
Remove the secret to resolve the issue.
```shell
rm components/podinfo/secret.cue
```
## Inspecting the diff
The validation and patch results in a correct Deployment, verified against the
Kubernetes type definitions.
```shell
git diff
```
```diff
diff --git a/deploy/components/podinfo/podinfo.gen.yaml b/deploy/components/podinfo/podinfo.gen.yaml
index 6e4aec0..a145e3f 100644
--- a/deploy/components/podinfo/podinfo.gen.yaml
+++ b/deploy/components/podinfo/podinfo.gen.yaml
@@ -101,7 +101,7 @@ spec:
successThreshold: 1
timeoutSeconds: 5
resources:
- limits: null
+ limits: {}
requests:
cpu: 1m
memory: 16Mi
```
## Trying Locally
Optionally, apply the manifests rendered by Holos to a [Local Cluster] for
testing.
[Local Cluster]: ../topics/local-cluster.mdx
[ExternalSecret]: https://external-secrets.io/latest/api/externalsecret/
[Artifact]: ../api/core.md#Artifact
[BuildPlan]: ../api/core.md#BuildPlan
[Resources]: ../api/core.md#Resources
[Validator]: ../api/core.md#Validator
[Transformer]: ../api/core.md#Transformer
[Kustomize]: ../api/core.md#Kustomize
[Generator]: ../api/core.md#Generator
[Hello Holos]: ./hello-holos.mdx
[cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue#L13
[ComponentConfig]: ../api/author.md#ComponentConfig
[timoni]: https://timoni.sh/install/
[resources.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/v1alpha5/resources.cue#L33

View File

@@ -7,6 +7,9 @@ image: /img/cards/announcing-holos.png
description: Holistically manage Helm and Kustomize with CUE
---
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
<head>
<title>Announcing Holos</title>
<meta property="og:title" content="Announcing Holos" />
@@ -21,39 +24,10 @@ manifests pattern as a data pipeline to fully render manifests generated from
[Kustomize]: https://kustomize.io/
[CUE]: https://cuelang.org/
```mermaid
---
title: Rendered Manifest Pipeline
---
graph LR
Platform[<a href="/docs/api/author/v1alpha4/#Platform">Platform</a>]
Component[<a href="/docs/api/author/v1alpha4/#ComponentConfig">Components</a>]
<RenderingOverview />
Helm[<a href="/docs/api/author/v1alpha4/#Helm">Helm</a>]
Kustomize[<a href="/docs/api/author/v1alpha4/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/api/author/v1alpha4/#Kubernetes">Kubernetes</a>]
{/* truncate */}
BuildPlan[<a href="/docs/api/core/v1alpha4/#buildplan">BuildPlan</a>]
ResourcesArtifact[<a href="/docs/api/core/v1alpha4/#artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/api/core/v1alpha4/#artifact">GitOps<br/>Artifact</a>]
Generators[<a href="/docs/api/core/v1alpha4/#generators">Generators</a>]
Transformers[<a href="/docs/api/core/v1alpha4/#transformer">Transformers</a>]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Files
```
<!-- truncate -->
At the start of the pandemic I was migrating our platform from VMs managed by
Puppet to Kubernetes. My primary goal was to build an observability system
@@ -85,6 +59,8 @@ a Go command line tool to implement the pattern as a data pipeline. Id been
thinking about the comments from the [Why are we templating YAML] posts and
wondering what an answer to this question would look like.
<RenderPlatformDiagram />
The Go command line tool was an incremental improvement over the CI scripts, but
we still didnt have a good way to handle the data values. We were still
templating YAML which didnt catch errors early enough. It was too easy to
@@ -101,22 +77,16 @@ Take a look at Holos if youre looking to implement the rendered manifests
pattern or cant shake that feeling it should be easier to integrate third party
software into Kubernetes like we felt.
1. [Helm Guide] Walks through how we solved the challenges we faced with the prometheus Helm charts.
2. [Quickstart] Works through how a platform team can define golden paths for other teams using CUE.
3. [Author API] provides an ergonomic way to work with Helm, Kustomize, and CUE resources.
1. [Tutorial] takes a tour of Holos features starting from scratch.
2. [Topics] cover how to customize and tailor Holos to your unique needs.
[Helm Guide]: /docs/guides/helm/
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story
[Holos]: https://holos.run/
[Quickstart]: /docs/quickstart/
[Tutorial]: /docs/v1alpha5/tutorial/overview/
[Topics]: /docs/v1alpha5/topics/
[Holos]: https://holos.run/
[Helm]: https://helm.sh/
[Kustomize]: https://kustomize.io/
[CUE]: https://cuelang.org/

View File

@@ -0,0 +1,35 @@
---
slug: validators-feature
title: Validators added in Holos v0.101.0
authors: [jeff]
tags: [holos, feature]
image: /img/cards/validators.png
description: Validators are useful to enforce policy and catch Helm errors.
---
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
We've added support for [Validators] in [v0.101.0]. Validators are useful to
enforce policies and ensure consistency early in the process. This feature
addresses two primary use cases:
1. Prevent insecure configuration early in the process. For example, prevent
Helm from rendering a `Secret` which would otherwise be committed to version control.
2. Prevent unsafe configuration by validating manifests against Kubernetes core
and custom resource type definitions.
Check out the [Validators] tutorial for examples of both use cases.
[Validators]: https://holos.run/docs/v1alpha5/tutorial/validators/
[v0.101.0]: https://github.com/holos-run/holos/releases/tag/v0.101.0
{/* truncate */}
<RenderingOverview />
Validators complete the core functionality of the Holos manifest rendering
pipeline. We are seeking design partners to help enhance Generators and
Transformers. Validators are implemented using a generic `Command` kind, and
we're considering a similar kind for Generators and Transformers. Please
connect with us if you'd like to help design these enhancements.

View File

@@ -59,6 +59,14 @@ const config: Config = {
showLastUpdateAuthor: true,
showLastUpdateTime: true,
sidebarPath: './sidebars.ts',
// https://docusaurus.io/docs/versioning#configuring-versioning-behavior
lastVersion: 'current',
versions: {
current: {
label: 'v1alpha5',
path: 'v1alpha5',
}
}
},
blog: {
path: "blog",
@@ -90,7 +98,7 @@ const config: Config = {
}
},
navbar: {
title: '',
title: 'Holos',
logo: {
src: 'img/logo.svg',
srcDark: 'img/logo-dark.svg',
@@ -98,17 +106,9 @@ const config: Config = {
items: [
{
type: 'doc',
docId: 'guides/quickstart',
position: 'left',
label: 'Quickstart',
},
{ to: '/docs/technical-overview', label: 'Docs', position: 'left' },
{ to: '/docs/guides', label: 'Guides', position: 'left' },
{
type: 'doc',
docId: 'api',
position: 'left',
label: 'API',
docId: 'tutorial/overview',
label: 'Docs',
position: 'left'
},
{ to: '/blog', label: 'Blog', position: 'left' },
{
@@ -121,71 +121,17 @@ const config: Config = {
label: 'Discord',
position: 'right',
},
{
type: 'docsVersionDropdown',
position: 'right',
// dropdownItemsAfter: [{ to: '/versions', label: 'All versions' }],
dropdownActiveClassDisabled: true,
},
],
},
footer: {
style: 'dark',
links: [
{
title: 'Docs',
items: [
{
label: 'Quickstart',
to: '/docs/quickstart',
},
{
label: 'Concepts',
to: '/docs/concepts',
},
{
label: 'Documentation',
to: '/docs',
},
{
label: 'API Reference',
to: '/docs/api',
},
],
},
{
title: 'Community',
items: [
{
label: 'Support',
href: '/docs/support',
},
{
label: 'Discord',
href: 'https://discord.gg/JgDVbNpye7',
},
{
label: 'Discussion List',
href: 'https://groups.google.com/g/holos-discuss',
},
{
label: 'Discussion Forum',
href: 'https://github.com/holos-run/holos/discussions',
},
],
},
{
title: 'More',
items: [
{
label: 'Blog',
to: '/blog',
},
{
label: 'GitHub',
href: 'https://github.com/holos-run/holos',
},
{
label: 'GoDoc',
href: 'https://pkg.go.dev/github.com/holos-run/holos?tab=doc',
}
],
},
],
links: [],
copyright: `Copyright © ${new Date().getFullYear()} The Holos Authors.`,
},
prism: {

File diff suppressed because it is too large Load Diff

View File

@@ -15,10 +15,10 @@
"typecheck": "tsc"
},
"dependencies": {
"@docusaurus/core": "^3.6.0",
"@docusaurus/plugin-client-redirects": "^3.6.0",
"@docusaurus/preset-classic": "^3.6.0",
"@docusaurus/theme-mermaid": "^3.6.0",
"@docusaurus/core": "^3.6.1",
"@docusaurus/plugin-client-redirects": "^3.6.1",
"@docusaurus/preset-classic": "^3.6.1",
"@docusaurus/theme-mermaid": "^3.6.1",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
@@ -26,9 +26,9 @@
"react-dom": "^18.0.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.6.0",
"@docusaurus/tsconfig": "^3.6.0",
"@docusaurus/types": "^3.6.0",
"@docusaurus/module-type-aliases": "^3.6.1",
"@docusaurus/tsconfig": "^3.6.1",
"@docusaurus/types": "^3.6.1",
"@wcj/html-to-markdown-cli": "^2.1.1",
"cspell": "^8.10.4",
"html-to-markdown": "^1.0.0",

View File

@@ -15,14 +15,14 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/base00/undraw_software_engineer_re_tnjc.svg').default,
description: (
<>
<p align="left">
<p style={{ textAlign: 'left' }}>
<ul>
<li>Provide simple definitions for other teams to use as golden paths.</li>
<li>Define integrations in <a href="https://cuelang.org/">CUE</a> with strong type checking. No more text templates or bash scripts.</li>
<li>Reuse your existing Helm charts and Kustomize bases.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
<a href="/docs/v1alpha5/tutorial/overview/">Learn More</a>
</>
),
},
@@ -31,14 +31,14 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/base00/undraw_through_the_park_lxnl.svg').default,
description: (
<>
<p align="left">
<p style={{ textAlign: 'left' }}>
<ul>
<li>Move faster using paved paths from your platform and security teams.</li>
<li>Develop locally or in the cloud.</li>
<li>Spend more time developing software and fewer cycles fighting infrastructure challenges.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
<a href="/docs/v1alpha5/tutorial/overview/">Learn More</a>
</>
),
},
@@ -47,14 +47,14 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/base00/undraw_security_on_re_e491.svg').default,
description: (
<>
<p align="left">
<p style={{ textAlign: 'left' }}>
<ul>
<li>Define security policy as reusable, typed configurations.</li>
<li>Automatically enforce security policy on new projects.</li>
<li>Ensure a consistent security posture cross-platform with fewer code changes.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
<a href="/docs/v1alpha5/tutorial/overview/">Learn More</a>
</>
),
}

View File

@@ -0,0 +1,16 @@
import styles from './styles.module.css';
//Pulled from: https://gaudion.dev/blog/mdx-youtube-embed
//components/mdx/YouTube.tsx
export default function YouTube({ id }: { id: string }) {
return (
<div className={styles.videoWrapper}>
<iframe
className="aspect-video w-full"
src={"https://www.youtube.com/embed/" + id + "?rel=0"}
title="YouTube Video Player"
allow="picture-in-picture; fullscreen; accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope;"
></iframe>
</div>
);
};

View File

@@ -0,0 +1,14 @@
.videoWrapper {
position: relative;
padding-bottom: 56.25%; /* 16:9 */
height: 0;
overflow: hidden;
}
.videoWrapper iframe {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
}

View File

@@ -0,0 +1,31 @@
```mermaid
---
title: holos render component sequence diagram
---
sequenceDiagram
participant HRC as holos<br />render component
participant CUE as CUE<br />(embedded)
participant G as Generator<br />(e.g. Helm)
participant T as Transformer<br />(e.g. Kustomize)
participant V as Validator<br />(e.g. CUE)
participant M as Manifests
HRC ->>+ CUE: Get apiVersion
HRC ->>+ CUE: Get BuildPlan
loop For each Artifact in BuildPlan concurrently
loop For each Generator in Artifact concurrently
HRC ->>+ G: Generate Config
G ->>+ HRC: Config
end
loop For each Transformer in Artifact sequentially
HRC ->>+ T: Transform Config
T ->>+ HRC: Config
end
loop For each Validator in Artifact concurrently
HRC ->>+ V: Validate Config
V ->>+ HRC: Valid / Invalid
end
HRC ->>+ M: Write Artifact File
end
Note over M: Ready for deployment
```

View File

@@ -0,0 +1,22 @@
```mermaid
---
title: holos render platform sequence diagram
---
sequenceDiagram
participant HRP as holos<br />render platform
participant HRC as holos<br />render component
participant CUE as CUE<br />(embedded)
participant A as Artifacts
participant M as Manifests
HRP ->>+ CUE: Get apiVersion
HRP ->>+ CUE: Get Platform Definition
loop For each Component in Platform concurrently
HRP ->>+ HRC: Execute
HRC ->>+ CUE: Get apiVersion
HRC ->>+ CUE: Get BuildPlan
HRC ->>+ A: Build Artifacts
A ->>+ HRC: Manifests
HRC ->>+ M: Write Manifest Files
end
Note over M: Ready for deployment
```

View File

@@ -0,0 +1,32 @@
```mermaid
---
title: Rendering Overview
---
graph LR
Platform[<a href="https://holos.run/docs/v1alpha5/api/author/#Platform">Platform</a>]
Component[<a href="https://holos.run/docs/v1alpha5/api/author/#ComponentConfig">Components</a>]
Helm[<a href="https://holos.run/docs/v1alpha5/api/author/#Helm">Helm</a>]
Kustomize[<a href="https://holos.run/docs/v1alpha5/api/author/#Kustomize">Kustomize</a>]
Kubernetes[<a href="https://holos.run/docs/v1alpha5/api/author/#Kubernetes">Kubernetes</a>]
BuildPlan[<a href="https://holos.run/docs/v1alpha5/api/core/#BuildPlan">BuildPlan</a>]
ResourcesArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">GitOps<br/>Artifact</a>]
Generators[<a href="https://holos.run/docs/v1alpha5/api/core/#Generator">Generators</a>]
Transformers[<a href="https://holos.run/docs/v1alpha5/api/core/#Transformer">Transformers</a>]
Validators[<a href="https://holos.run/docs/v1alpha5/api/core/#Validator">Validators</a>]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Validators --> Files
```

View File

@@ -23,17 +23,11 @@ function HomepageHeader() {
<div className={styles.buttons}>
<Link
className="button button--secondary button--lg"
to="docs/quickstart">
Get Started
</Link>
<span className={styles.divider}></span>
<Link
className="button button--primary button--lg"
to="docs/technical-overview/">
to="docs/v1alpha5/tutorial/overview/">
Learn More
</Link>
<span className={styles.divider}></span>
</div>
</div>
</div>
</div >
</header >

View File

@@ -0,0 +1,22 @@
/docs /docs/v1alpha5/ 301
/docs/ /docs/v1alpha5/ 301
/docs/tutorial /docs/v1alpha5/tutorial/overview/ 301
/docs/tutorial/ /docs/v1alpha5/tutorial/overview/ 301
/docs/quickstart /docs/v1alpha5/tutorial/overview/ 301
/docs/quickstart/ /docs/v1alpha5/tutorial/overview/ 301
/docs/overview /docs/v1alpha5/tutorial/overview/ 301
/docs/overview/ /docs/v1alpha5/tutorial/overview/ 301
/docs/topics /docs/v1alpha5/topics/ 301
/docs/topics/ /docs/v1alpha5/topics/ 301
/docs/setup /docs/v1alpha5/tutorial/setup/ 301
/docs/setup/ /docs/v1alpha5/tutorial/setup/ 301
/docs/local-cluster /docs/v1alpha5/topics/local-cluster/ 301
/docs/local-cluster/ /docs/v1alpha5/topics/local-cluster/ 301
/docs/guides/helm /docs/v1alpha5/tutorial/helm-values/ 301
/docs/guides/helm/ /docs/v1alpha5/tutorial/helm-values/ 301
/docs/kargo /docs/v1alpha5/topics/kargo/ 301
/docs/kargo/ /docs/v1alpha5/topics/kargo/ 301
/docs/comparison /docs/v1alpha5/topics/comparison/ 301
/docs/comparison/ /docs/v1alpha5/topics/comparison/ 301
/docs/support /docs/v1alpha5/tutorial/overview/#getting-help 301
/docs/support/ /docs/v1alpha5/tutorial/overview/#getting-help 301

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

View File

Before

Width:  |  Height:  |  Size: 248 KiB

After

Width:  |  Height:  |  Size: 248 KiB

View File

Before

Width:  |  Height:  |  Size: 206 KiB

After

Width:  |  Height:  |  Size: 206 KiB

View File

@@ -1,724 +0,0 @@
---
description: Try Holos with this quick start guide.
slug: /archive/2024-09-15-quickstart
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Quickstart
In this guide, you'll experience how Holos makes the process of operating a
Platform safer, easier, and more consistent. We'll use Holos to manage a
vendor-provided Helm chart as a Component. Next, we'll mix in our own custom
resources to manage the Component with GitOps. Finally, you'll see how Holos
makes it safer and easier to maintain software over time by surfacing the exact
changes that will be applied when upgrading the vendor's chart to a new version,
before they are actually made.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap upstream Helm charts.
Optionally, if you'd like to apply the rendered manifests to a real Cluster,
first complete the [Local Cluster Guide](/docs/guides/local-cluster).
## Install Holos
Install Holos with the following command or other methods listed on the
[Installation](/docs/install/) page.
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
## Create a Git Repository
Start by initializing an empty Git repository. Holos operates on local files
stored in a Git repository.
<Tabs groupId="init">
<TabItem value="command" label="Command">
```bash
mkdir holos-quickstart
cd holos-quickstart
git init
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Initialized empty Git repository in /holos-quickstart/.git/
```
</TabItem>
</Tabs>
This guide assumes you will run commands from the root directory of the Git
repository unless stated otherwise.
## Generate the Platform {#Generate-Platform}
Generate the Platform code in the repository root. A Platform refers to the
entire set of software holistically integrated to provide a software development
platform for your organization. In this guide, the Platform will include a
single Component to demonstrate how the concepts fit together.
```bash
holos generate platform quickstart
```
Commit the generated platform config to the repository.
<Tabs groupId="commit-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate platform quickstart - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main (root-commit) 0b17b7f] holos generate platform quickstart
213 files changed, 72349 insertions(+)
...
```
</TabItem>
</Tabs>
## Generate a Component {#generate-component}
The platform you generated is currently empty. Run the following command to
generate the CUE code that defines a Helm Component.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
holos generate component podinfo --component-version 6.6.1
```
</TabItem>
<TabItem value="output" label="Output">
```txt
generated component
```
</TabItem>
</Tabs>
The --component-version 6.6.1 flag intentionally installs an older release.
You'll see how Holos assists with software upgrades later in this guide.
The generate component command creates two files: a leaf file,
`components/podinfo/podinfo.gen.cue`, and a root file, `podinfo.gen.cue`. Holos
leverages the fact that [order is
irrelevant](https://cuelang.org/docs/tour/basics/order-irrelevance/) in CUE to
register the component with the Platform by adding a file to the root of the Git
repository. The second file defines the component in the leaf component
directory.
<Tabs groupId="podinfo-files">
<TabItem value="components/podinfo/podinfo.gen.cue" label="Leaf">
`components/podinfo/podinfo.gen.cue`
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
(#Helm & Chart).Output
let Chart = {
Name: "podinfo"
Version: "6.6.1"
Namespace: "default"
Repo: name: "podinfo"
Repo: url: "https://stefanprodan.github.io/podinfo"
Values: {}
}
```
</TabItem>
<TabItem value="podinfo.gen.cue" label="Root">
`podinfo.gen.cue`
```cue showLineNumbers
package holos
// Manage podinfo on workload clusters only
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name)/podinfo": {
path: "components/podinfo"
cluster: Cluster.name
}
}
```
</TabItem>
</Tabs>
In this example, we provide the minimal information needed to manage the Helm
chart: the name, version, Kubernetes namespace for deployment, and the chart
repository location.
This chart deploys cleanly without any values provided, but we include an empty
Values struct to show how Holos improves consistency and safety in Helm by
leveraging the strong type-checking in CUE. You can safely pass shared values,
such as the organizations domain name, to all Components across all clusters in
the Platform by defining them at the root of the configuration.
Commit the generated component config to the repository.
<Tabs groupId="commit-component">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate component podinfo - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main cc0e90c] holos generate component podinfo
2 files changed, 24 insertions(+)
create mode 100644 components/podinfo/podinfo.gen.cue
create mode 100644 podinfo.gen.cue
```
</TabItem>
</Tabs>
## Render the Component
You can render individual components without adding them to a Platform, which is
helpful when developing a new component.
<Tabs groupId="render-podinfo">
<TabItem value="command" label="Command">
```bash
holos render component ./components/podinfo --cluster-name=default
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached
rendered podinfo
```
</TabItem>
</Tabs>
First, the command caches the Helm chart locally to speed up subsequent
renderings. Then, the command runs Helm to produce the output and writes it into
the deploy directory.
<Tabs groupId="tree-podinfo">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
└── default
└── components
└── podinfo
└── podinfo.gen.yaml
5 directories, 1 file
```
</TabItem>
</Tabs>
The component deploys to one cluster named `default`. In practice, the same
component is often deployed to multiple clusters, such as `east` and `west` to
provide redundancy and increase availability.
:::tip
This example is equivalent to running `helm template` on the chart and saving
the output to a file. Holos simplifies this task, making it safer and more
consistent when managing many charts.
:::
## Mix in an ArgoCD Application
We've seen how Holos works with Helm, but we haven't yet explored how Holos
makes it easier to consistently and safely manage all of the software in a
Platform.
Holos allows you to easily mix in resources that differentiate your Platform.
We'll use this feature to mix in an ArgoCD [Application][application] to manage
the podinfo Component with GitOps. We'll define this configuration in a way that
can be automatically and consistently reused across all future Components added
to the Platform.
Create a new file named `argocd.cue` in the root of your Git repository with the
following contents:
<Tabs groupId="argocd-config">
<TabItem value="command" label="argocd.cue">
```cue showLineNumbers
package holos
#ArgoConfig: {
Enabled: true
RepoURL: "https://github.com/holos-run/holos-quickstart-guide"
}
```
</TabItem>
</Tabs>
:::tip
If you plan to apply the rendered output to a real cluster, change the
`example.com` RepoURL to the URL of the Git repository you created in this
guide. You don't need to change the example if you're just exploring Holos by
inspecting the rendered output without applying it to a live cluster.
:::
With this file in place, render the component again.
<Tabs groupId="render-podinfo-argocd">
<TabItem value="command" label="Command">
```bash
holos render component ./components/podinfo --cluster-name=default
```
</TabItem>
<TabItem value="output" label="Output">
```txt
wrote deploy file
rendered gitops/podinfo
rendered podinfo
```
</TabItem>
</Tabs>
Holos uses the locally cached chart to improve performance and reliability. It
then renders the Helm template output along with an ArgoCD Application resource
for GitOps.
:::tip
By defining the ArgoCD configuration at the root, we again take advantage of the
fact that [order is
irrelevant](https://cuelang.org/docs/tour/basics/order-irrelevance/) in CUE.
:::
Defining the configuration at the root ensures all future leaf Components take
the ArgoCD configuration and render an Application manifest for GitOps
management.
<Tabs groupId="tree-podinfo-argocd">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
└── default
├── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── gitops
└── podinfo.application.gen.yaml
6 directories, 2 files
```
</TabItem>
</Tabs>
Notice the new `podinfo.application.gen.yaml` file created by enabling ArgoCD in
the Helm component. The Application resource in the file looks like this:
<Tabs groupId="podinfo-application">
<TabItem value="file" label="podinfo.application.gen.yaml">
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: ./deploy/clusters/default/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
</Tabs>
:::tip
Holos generates a similar Application resource for every additional Component
added to your Platform.
:::
Finally, add and commit the results to your Platform's Git repository.
<Tabs groupId="commit-argo">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos render component ./components/podinfo --cluster-name=default"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main f95cef1] holos render component ./components/podinfo --cluster-name=default
3 files changed, 134 insertions(+)
create mode 100644 argocd.cue
create mode 100644 deploy/clusters/default/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/default/gitops/podinfo.application.gen.yaml
```
</TabItem>
</Tabs>
In this section, we learned how Holos simplifies mixing resources into
Components, like an ArgoCD Application. Holos ensures consistency by managing an
Application resource for every Component added to the Platform through the
configuration you define in `argocd.cue` at the root of the repository.
## Define Workload Clusters {#workload-clusters}
We've generated a Component to manage podinfo and integrated it with our
Platform, but rendering the Platform doesn't render podinfo. Podinfo isn't
rendered because we haven't assigned any Clusters to the workload Fleet.
Define two new clusters, `east` and `west`, and assign them to the workload
Fleet. Create a new file named `clusters.cue` in the root of your Git repository
with the following contents:
<Tabs groupId="clusters">
<TabItem value="clusters.cue" label="clusters.cue">
```cue showLineNumbers
package holos
// Define two workload clusters for disaster recovery.
#Fleets: workload: clusters: {
// In CUE _ indicates values are defined elsewhere.
east: _
west: _
}
```
</TabItem>
</Tabs>
This example shows how Holos simplifies configuring multiple clusters with
similar configuration by grouping them into a Fleet.
:::tip
Fleets help segment a group of Clusters into one leader and multiple followers
by designating one cluster as the primary. Holos makes it safer, easier, and
more consistent to reconfigure which cluster is the primary. The primary can be
set to automatically restore persistent data from backups, while non-primary
clusters can be configured to automatically replicate from the primary.
Automatic database backup, restore, and streaming replication is an advanced
topic enabled by Cloud Native PG and CUE. Check back for a guide on this and
other Day 2 operations topics.
:::
## Render the Platform {#render-platform}
Render the Platform to render the podinfo Component for each of the workload
clusters.
<Tabs groupId="render-platform">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered components/podinfo for cluster west in 99.480792ms
rendered components/podinfo for cluster east in 99.882667ms
```
</TabItem>
</Tabs>
The render platform command iterates over every Cluster in the Fleet and renders
each Component assigned to the Fleet. Notice the two additional subdirectories
created under the deploy directory, one for each cluster: `east` and `west`.
<Tabs groupId="tree-platform">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
├── default
│   ├── components
│   │   └── podinfo
│   │   └── podinfo.gen.yaml
│   └── gitops
│   └── podinfo.application.gen.yaml
# highlight-next-line
├── east
│   ├── components
│   │   └── podinfo
│   │   └── podinfo.gen.yaml
│   └── gitops
│   └── podinfo.application.gen.yaml
# highlight-next-line
└── west
├── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── gitops
└── podinfo.application.gen.yaml
14 directories, 6 files
```
</TabItem>
</Tabs>
Holos ensures consistency and safety by defining the ArgoCD Application once,
with strong type checking, at the configuration root.
New Application resources are automatically generated for the `east` and `west`
workload Clusters.
<Tabs groupId="applications">
<TabItem value="east" label="east">
`deploy/clusters/east/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/east/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
<TabItem value="west" label="west">
`deploy/clusters/west/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/west/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
<TabItem value="default" label="default">
`deploy/clusters/default/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/default/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
</Tabs>
Add and commit the rendered Platform and workload Clusters.
<Tabs groupId="commit-render-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos render platform ./platform - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 5aebcf5] holos render platform ./platform - 0.93.2
5 files changed, 263 insertions(+)
create mode 100644 clusters.cue
create mode 100644 deploy/clusters/east/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/east/gitops/podinfo.application.gen.yaml
create mode 100644 deploy/clusters/west/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/west/gitops/podinfo.application.gen.yaml
```
</TabItem>
</Tabs>
## Upgrade a Helm Chart
Holos is designed to ease the burden of Day 2 operations. With Holos, upgrading
software, integrating new software, and making safe platform-wide configuration
changes become easier.
Let's upgrade the podinfo Component to see how this works in practice. First,
update the Component version field to the latest upstream Helm chart version.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
holos generate component podinfo --component-version 6.6.2
```
</TabItem>
<TabItem value="output" label="Output">
```txt
generated component
```
</TabItem>
</Tabs>
Remove the cached chart version.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
rm -rf components/podinfo/vendor
```
</TabItem>
</Tabs>
Now re-render the Platform.
<Tabs groupId="render-platform2">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered components/podinfo for cluster east in 327.10475ms
rendered components/podinfo for cluster west in 327.796541ms
```
</TabItem>
</Tabs>
Notice we're still using the upstream chart without modifying it. The Holos
component wraps around the chart to mix in additional resources and integrate
the component with the broader Platform.
## Visualize the Changes
Holos makes it easier to see exactly what changes are made and which resources
will be applied to the API server. By design, Holos operates on local files,
leaving the task of applying them to ecosystem tools like `kubectl` and ArgoCD.
This allows platform operators to inspect changes during code review, or before
committing the change at all.
For example, using `git diff`, we see that the only functional change when
upgrading this Helm chart is the deployment of a new container image tag to each
cluster. Additionally, we can roll out this change gradually by applying it to
the east cluster first, then to the west cluster, limiting the potential blast
radius of a problematic change.
<Tabs groupId="git-diff">
<TabItem value="command" label="Command">
```bash
git diff deploy/clusters/east
```
</TabItem>
<TabItem value="output" label="Output">
```diff showLineNumbers
diff --git a/deploy/clusters/east/components/podinfo/podinfo.gen.yaml b/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
index 7cc3332..8c1647d 100644
--- a/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
+++ b/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
@@ -5,9 +5,9 @@ kind: Service
metadata:
name: podinfo
labels:
- helm.sh/chart: podinfo-6.6.1
+ helm.sh/chart: podinfo-6.6.2
app.kubernetes.io/name: podinfo
- app.kubernetes.io/version: "6.6.1"
+ app.kubernetes.io/version: "6.6.2"
app.kubernetes.io/managed-by: Helm
spec:
type: ClusterIP
@@ -29,9 +29,9 @@ kind: Deployment
metadata:
name: podinfo
labels:
- helm.sh/chart: podinfo-6.6.1
+ helm.sh/chart: podinfo-6.6.2
app.kubernetes.io/name: podinfo
- app.kubernetes.io/version: "6.6.1"
+ app.kubernetes.io/version: "6.6.2"
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
@@ -53,7 +53,7 @@ spec:
terminationGracePeriodSeconds: 30
containers:
- name: podinfo
# highlight-next-line
- image: "ghcr.io/stefanprodan/podinfo:6.6.1"
# highlight-next-line
+ image: "ghcr.io/stefanprodan/podinfo:6.6.2"
imagePullPolicy: IfNotPresent
command:
- ./podinfo
```
</TabItem>
</Tabs>
:::tip
Holos is designed to surface the _fully rendered_ manifests intended for the
Kubernetes API server, making it easier to see and reason about platform-wide
configuration changes.
:::
## Recap {#recap}
In this quickstart guide, we learned how Holos makes it easier, safer, and more
consistent to manage a Platform composed of multiple Clusters and upstream Helm
charts.
We covered how to:
1. Generate a Git repository for the Platform config.
2. Wrap the unmodified upstream podinfo Helm chart into a Component.
3. Render an individual Component.
4. Mix-in your Platform's unique resources to all Components. For example, ArgoCD Application resources.
5. Define multiple similar, but not identical, workload clusters.
6. Render the manifests for the entire Platform with the `holos render platform` command.
7. Upgrade a Helm chart to the latest version as an important Day 2 task.
8. Visualize and surface the details of planned changes Platform wide.
## Dive Deeper
If you'd like to dive deeper, check out the [Schema API][schema] and [Core
API][core] reference docs. The main difference between the schema and core
packages is that the schema is used by users to write refined CUE, while the
core package is what the schema produces for `holos` to execute. Users rarely
need to interact with the Core API when on the happy path, but can use the core
package as an escape hatch when the happy path doesn't go where you want.
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[schema]: /docs/api/author/
[core]: /docs/api/core/

View File

@@ -1,106 +0,0 @@
---
description: Self service platform resource management for project teams.
slug: /archive/guides/2024-09-17-manage-a-project
sidebar_position: 250
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Manage a Project
In this guide we'll explore how Holos easily, safely, and consistently manages
platform resources for teams to develop the projects they're working on.
Intended Audience: Platform Engineers and Software Engineers.
Goal is to demonstrate how the platform team can consistently, easily, and
safely provide platform resources to software engineers.
Assumption is software engineers have a container they want to deploy onto the
platform and make accessible. We'll use httpbin as a stand-in for the dev
team's container.
Project is roughly equivalent to Dev Team for the purpose of this guide, but in
practice multiple teams work on a given project over the lifetime of the
project, so we structure the files into projects instead of teams.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Helm Components.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Kustomize Components.
If you'd like to apply the manifests we render in this guide complete the
following optional, but recommended, steps.
a. Complete the [Local Cluster] guide to set up a local cluster to work with.
b. You'll need a GitHub account to fork the repository associated with this
guide.
## Fork the Guide Repository
<Tabs groupId="fork">
<TabItem value="command" label="Command">
```bash
```
</TabItem>
<TabItem value="output" label="Output">
```txt showLineNumbers
```
</TabItem>
</Tabs>
This guide assumes you will run commands from the root directory of this
repository unless stated otherwise.
[Quickstart]: /docs/quickstart
[Local Cluster]: /docs/guides/local-cluster
## Render the Platform
So we can build the basic platform. Don't dwell on the platform bits.
## Apply the Manifests
Deploy ArgoCD, but not any of the Application resources.
## Browse to ArgoCD
Note there is nothing here yet.
## Switch to your Fork
Note all of the Applications change consistently.
## Apply the Applications
Note how ArgoCD takes over management, no longer need to k apply.
## Create a Project
Project is a conceptual, not technical, thing in Holos. Mainly about how components are laid out in the filesystem tree.
We use a schematic built into holos as an example, the platform team could use the same or provide a similar template and instructions for development teams to self-serve.
## Render the Platform
Notice:
1. Project is registered with the platform at the root.
2. HTTPRoute and Namespace resources are added close to the root in `projects`
3. Deployment and Service resources are added at the leaf in `projects/httpbin/backend`
## Update the image tag
Add a basic schematic to demonstrate this. May need to add two new flags for image url and image tag to the generate subcommand, but should just be two new fields on the struct.
## Dive Deeper
Set the stage for constraints. Ideas: Limit what resources can be added,
namespaces can be operated in, enforce labels, etc...
Simple, consistent, easy constraints.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 934 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 703 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1014 KiB

Some files were not shown because too many files have changed in this diff Show More