Compare commits

...

149 Commits

Author SHA1 Message Date
Jeff McCune
6a60b613ff render: fix selectors (#394)
Without this patch selectors don't work as expected.  This patch
fixes selectors such that each --selector flag value configures one
selector containing multiple positive or negative label matchers.

Result:

Render build plans for cluster dev or cluster test.  Note the use of two
flags indicating logical OR.

    holos render platform --selector cluster=test --selector cluster=dev
    rendered external-secrets for cluster test in 299.897542ms
    rendered external-secrets for cluster dev in 299.9225ms
    rendered external-secrets-crds for cluster test in 667.6075ms
    rendered external-secrets-crds for cluster dev in 708.126541ms
    rendered platform in 708.795625ms

Render build plans for prod clusters that are not customer facing.  Note
the use of one selector with comma separated labels.

    holos render platform --selector "tier=prod,scope!=customer"
2025-01-08 21:09:00 -08:00
Jeff McCune
5862725bab builder: deprecate ExtractYAML, use cue embed instead
Easier to place the data, better supported in the ecosystem.
2025-01-02 18:53:10 -08:00
Jeff McCune
8660826b05 builder: protect LoadInstance with a mutex
CUE is not safe for concurrent access so we protect the main
LoadInstance function with a mutex lock.
2025-01-02 17:32:53 -08:00
Jeff McCune
449df91e33 docs: app.holos.run/description not cli
The core component documentation on the annotation used to configure the
display line for each rendered component was incorrect.
2025-01-02 08:36:37 -08:00
Jeff McCune
ac59173b30 ci: update holos-run/holos-action version (try 3)
Fix the use of digests when pulling and pushing images.  Pull the image
from ghcr.io before pushing it to quay.io
2024-12-23 10:33:45 -08:00
Jeff McCune
fb75e560fc ci: update holos-run/holos-action version (try 2)
When new container image versions are built, automatically update the
holos-run/holos-action to use the new version.

Users of the action automatically update by default as a result.
2024-12-23 09:52:09 -08:00
Jeff McCune
69a064e3ea ci: update holos-run/holos-action version
When new container image versions are built, automatically update the
holos-run/holos-action to use the new version.

Users of the action automatically update by default as a result.
2024-12-23 07:23:36 -08:00
Jeff McCune
71b72807bb ci: tag v0.102.1 for container images
We need a released tag to reference in workflows that use the container
image to render the platform configuration.

This is the first image, subsequent git tags will also build and publish
container images.
2024-12-21 08:08:51 -08:00
Jeff McCune
0e4ecf9d13 ci: fix error in containers.yaml 2024-12-21 07:33:31 -08:00
Jeff McCune
ec2fdadd44 ci: build container from any ref
Too hard to try and build back in time, so let's just get it working
then build containers going forward for tags.
2024-12-21 07:31:09 -08:00
Jeff McCune
38b082095f ci: drop linux/arm/v7 support
There aren't kubectl images to build against.
2024-12-21 07:14:21 -08:00
Jeff McCune
f9346ea7c0 ci: use Dockerfile from main when building tags
Problem: We can't build old tags because the wrong Dockerfile is used
from the old tag.

Solution: Save the Dockerfile from main and use it to build the tag.
This create a dirty working directory but that's OK.
2024-12-21 07:11:29 -08:00
Jeff McCune
0f7010288a ci: build distroless container image for holos
Push it to ghcr and quay.

 * sign images with cosign and oidc id token
 * add kustomize v5.5.0 to tools for distroless image

Usage:

    docker run -v $(pwd):/app -w /app --rm -it ghcr.io/holos-run/holos:v0.101.8 holos render platform
2024-12-21 06:58:57 -08:00
Jeff McCune
386fb89cc6 ci: replace lint workflow with cspell
The lint workflow was slow and we don't often change buf or angular
these days so they're not necessary.

The remaining valuable task is cspell, which we can speed up with a
dedicated step.
2024-12-20 13:52:54 -08:00
Jeff McCune
c5401d6b02 ci: speed up tests by killing steps 2024-12-20 11:57:05 -08:00
Jeff McCune
f215405643 docs: fix links in readme 2024-12-20 07:28:04 -08:00
Jeff McCune
2c79982bd3 cue: enable @embed for loading yaml (#385)
mpvl suggests @embed is a more ideal solution than our implementation of
core.Component.Instances for the use case of unifying YAML data updated
by Kargo Stage resources.

See the issue for a link to the discussion.
2024-12-20 07:14:01 -08:00
Jeff McCune
e5e4de3073 cue: update to 0.11.1
go get cuelang.org/go/cmd/cue@latest

    go: downloading cuelang.org/go v0.11.1
    go: upgraded cuelang.org/go v0.11.0 => v0.11.1
2024-12-20 07:09:39 -08:00
Jeff McCune
ec462f5f0b docs: redirect /docs/support 2024-12-19 22:13:04 -08:00
Jeff McCune
0e95a2812e cmd: expose MakeMain() for testing
I'd like to add the kargo-demo repository to Unity to test evalv3, but
can't get a handle on the main function to wire up to testscript.

This patch fixes the problem by moving the MakeMain function to a public
package so the kargo-demo go module can import and call it using the go
mod tools technique.
2024-12-19 15:19:46 -08:00
Jeff McCune
54efe3e24a core: pass --extract-yaml flag from platform to component (#376)
Previously holos render platform was not setting the --extract-yaml file
when calling holos render component, causing data file instances defined
in the Platform spec to be discarded.

This patch passes the value along using the flag.
2024-12-19 08:39:55 -08:00
Jeff McCune
f693f049f4 core: refactor --instance to --extract-yaml (#376)
Extract YAML is more clear and aligns with the schema docs for the
Component Instance field which has an extractYAML kind.  This also
leaves the door open for additional kinds of data extractors which are
almost certainly going to be needed.
2024-12-19 08:34:05 -08:00
Jeff McCune
85238710ac core: unify data files into config (#376)
Previously there isn't a good way to unify json and yaml files with the
cue configuration.  This is a problem for use cases where data can be
generated idempotentialy prior to rendering the platform configuration.

The first use case is to explore unifying configuration with decrypted
sops values, which isn't typical since Holos is designed to handle
secrets with ExternalSecret resources, but does fit into the use case of
executing a command to produce data idempotently, then make the data
available to the platform configuration.

Other use cases this feature is intended to support are the prior
experiment where we fetch top level platform configuration from an rpc
service, and the future goal of integrating with data provided by
Terraform.
2024-12-19 08:34:05 -08:00
Jeff McCune
3ec62d272e v1alpha5: update kargo crds to 1.1.1 2024-12-19 08:34:04 -08:00
Jeff McCune
49afb44fd4 docs: redirect /docs/comparison/ 2024-12-18 14:37:36 -08:00
Gary Larizza
a023f135ab Add a Comparisons page
PROBLEM:

We've noticed that Holos almost immediately gets compared to Timoni, and
we frequently get asked for specifics in how they're similar/different.

SOLUTION:

* Add a `Comparison` page.
* Include a section that compares Holos to Timoni

OUTCOME:

Fewer questions about how Holos compares to Timoni because people are
able to find that answer themselves on our docs page.
2024-12-18 14:33:52 -08:00
Jeff McCune
c6a3a5d689 docs: redirect /docs/kargo/ 2024-12-17 06:30:20 -08:00
Jeff McCune
3f1eed3f06 platform: add kargo.akuity.io custom resource definitions
Needed for Kargo integration.  Imported with timoni from v1.0.3 Kargo
CRD's.
2024-12-16 13:19:39 -08:00
Jeff McCune
7fb7df1441 docs: make the linter happy 2024-12-16 11:04:35 -05:00
Jeff McCune
a798111d4d docs: add oci helm charts example
Question came up in chat, there isn't a good example and it's a pain to
piece together from the reference docs.
2024-12-16 10:56:50 -05:00
Jeff McCune
3ddb823341 docs: add note about compinit
Andy ran into issues enabling completion without first figuring out how
to initialize the completion system.
2024-12-16 08:15:45 -05:00
Jeff McCune
70d48592c4 docs: fix environments topic
It didn't work, failed with:

  ❯ holos show buildplans --selector app.holos.run/city=ams
  could not run: Component.Name: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/instance.go:66
  Component.Name: 2 errors in empty disjunction:
  Component.Name: conflicting values "no-name" and "podinfo-ams":
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:6:12
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:6:13
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:35:2
      /Users/jeff/Holos/foo/holos-environments-tutorial/tags.cue:13:19
  Component.Name: conflicting values "podinfo" and "podinfo-ams":
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:6:12
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:7:8
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:6:13
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:35:2

This was likely because the podinfo component was used in different ways
in different topics.  Don't use the shared component to fix the problem.
2024-12-13 09:20:52 -05:00
Jeff McCune
006f08df93 docs: add kargo place holder (#378) 2024-12-11 09:58:54 -08:00
Jeff McCune
39e2db5d37 docs: remove related content from youtube embed
Except stuff in our own channel.
2024-12-08 19:43:12 -08:00
Jeff McCune
ceb293fd8a docs: fix typescript className not class check error 2024-12-08 19:36:36 -08:00
Jeff McCune
188ff95015 docs: enable youtube fullscreen
Without this patch the fullscreen button is disabled.
2024-12-08 19:33:06 -08:00
Jeff McCune
5f658e0ba0 docs: add flux kustomization example (#374)
Almost identical to the ArgoCD Application example.
2024-12-08 19:20:12 -08:00
Jeff McCune
18b2850d3c platform: import flux custom resources
kustomize build https://github.com/fluxcd/flux2/manifests/crds\?ref=v2.4.0 \
      timoni mod vendor crds -f-
2024-12-08 19:03:18 -08:00
Jeff McCune
366a7fe93d docs: private helm repos need updated schemas (#370)
Document the need to run holos init platform v1alpha5 --force to use the
private helm repository feature.
2024-12-08 17:13:56 -08:00
Jeff McCune
f71d6d5bd9 helm: support private helm repositories (#370)
Previously holos unconditionally executed helm repo add which failed for
private repositories requiring basic authentication.

This patch addresses the problem by using the Helm SDK to pull and cache
charts without adding them as repositories.  New fields for the
core.Helm type allow basic auth credentials to be read from environment
variables.

Multiple repositories are supported by using different env vars for
different repositories.
2024-12-06 15:38:46 -08:00
Gary Larizza
4529673e93 Embed YouTube Video (#369)
PROBLEM:

We've created a YouTube video walking people through Holos and the Helm
Values tutorial, but now we need to embed it on the site for visitors to
watch.

SOLUTION:

* Create a `YouTube` MDX plugin
* Use that plugin on Overview and Helm Values
* Tune the video size/attributes using CSS

OUTCOME:

The Helm Values YouTube video is embedded on our site for visitors to
watch.
2024-12-06 15:33:03 -08:00
Jeff McCune
16a6447926 helm: support oci images in chart name
Without this patch we do not support installing Kargo from an OCI helm
chart.  We want to support:

```
Component: #Helm & {
	Name:      "kargo"
	Namespace: Kargo.Namespace

	Chart: {
		name:    "oci://ghcr.io/akuity/kargo-charts/kargo"
		version: "1.0.3"
		release: Name
	}
	EnableHooks: true

	Values: Kargo.Values
}
```

This patch fixes the problem by using the base name for filesystem cache
operations.
2024-12-03 12:15:06 -08:00
Jeff McCune
111a5944ff cue: bump to 0.11.0
go get cuelang.org/go/cmd/cue@latest
2024-12-02 12:37:19 -08:00
Jeff McCune
ff1446dc93 docs: redirect /docs/guides/helm/
This shows up in the Unity tests I'm working on with mvdan and goes to a
blank page without the redirect in place.

	--- FAIL: TestGuides_v1alpha5 (0.00s)
	   --- FAIL: TestGuides_v1alpha5/helm (0.60s)
	       testscript.go:584: # Helm Guide https://holos.run/docs/guides/helm/
2024-12-02 09:05:13 -08:00
Jeff McCune
67ef990c37 v0.101.2 build tags 2024-12-02 08:09:23 -08:00
Jeff McCune
6bd54ab856 render: pass build tags from platform to component (#366)
Previously, build tags were not propagated from `holos render platform
-t validate` through to the underlying `holos render component` command.
This is a problem because validators need to be selectively enabled as a
work around until we have an audit mode field.

This patch fixes the problem by propagating command line tags from the
render platform command to the underlying commands.  This patch also
propagates tags for the show command.
2024-11-30 20:56:11 -08:00
Jeff McCune
89a23a10fd docs: remove DIRECTORY from holos render platform --help
The directory argument is deprecated now, use the --platform flag
instead.
2024-11-30 13:08:01 -08:00
Jeff McCune
5a939bb6fe render: support cue build tags e.g. -t foo for @if(foo) (#366)
Previously Holos only supported tags in the form of key=value.  CUE
supports boolean style tags in the form of `key [ "=" value ]` which we
want to use to conditionally use to register components with the
platform.

This patch modifies the flag parsing to support -t foo like cue does,
for use with the @if(foo) build tag.
2024-11-30 12:50:31 -08:00
Jeff McCune
ee16f14e03 refactor build plan pipeline
Previously the BuildPlan pipeline didn't execute generators and
transformers concurrently.  All steps were sequentially executed.  Holos
was primarily concurrent by executing multiple BuildPlans at once.

This patch changes the Build implementation for each BuildPlan to
execute a GoRoutine pipeline.  One producer fans out to a group of
routines each executing the pipeline for one artifact in the build plan.
The pipeline has 3 stages:

1: Fan-out to build each Generator concurrently.
2: Fan-in to build each Transformer sequentially.
3: Fan-out again to run each validator concurrently.

When the artifact pipelines return, the producer closes the tasks
channel causing the worker tasks to return.

Note the overall runtime for 8 BuildPlans is roughly equivalent to
previously at 160ms with --concurrency=8 on my M3 Max.  I expect this to
perform better than previously when multiple artifacts are rendered for
each BuildPlan.
2024-11-29 14:52:06 -08:00
Jeff McCune
7530345620 main: add tracing and profiling
Writes files based on parent pid and process pid to avoid collisions.

Analyze with:

export HOLOS_TRACE=trace.%d.%d.out
go tool trace trace.999.1000.out

export HOLOS_CPU_PROFILE=cpu.%d.%d.prof
go tool pprof cpu.999.1000.prof

export HOLOS_MEM_PROFILE=mem.%d.%d.prof
go tool pprof mem.999.1000.prof
2024-11-29 14:52:06 -08:00
Jeff McCune
47d60ef86d docs: fix cue vet path in validators post (#357)
Without this patch the validator fails if a component manages two of the
same kind of resource, which is common.

This patch updates the example to use the metadata namespace and name as
lookup keys.  This works for most components, but may not for
ClusterResources.  Use the kind top level field in that case and pass
the field name of the validator as a tag value to vary by component.
2024-11-25 19:35:02 -08:00
Jeff McCune
9e9f6efd04 docs: fix typos in validators blog post (#357) 2024-11-25 15:46:54 -08:00
Jeff McCune
fb4a043823 docs: add card for validators blog post (#357) 2024-11-25 15:11:11 -08:00
Jeff McCune
d718ab1910 docs: redirect /docs/local-cluster to the v1alpha5 topic 2024-11-25 10:53:31 -08:00
Jeff McCune
c649db18a9 docs: redirect /docs/quickstart to the overview 2024-11-25 10:43:16 -08:00
Jeff McCune
b3bddf3ee3 docs: add validators blog post (#357) 2024-11-25 08:49:27 -08:00
Jeff McCune
77836be250 docs: update readme 2024-11-25 08:10:11 -08:00
Jeff McCune
4db670b854 docs: add validators tutorial (#357)
Add a tutorial page on validators.
2024-11-24 21:34:09 -08:00
Jeff McCune
d87c919519 docs: redirect /docs/topics to structures
/docs/v1alpha5/api/author/ links to it in the opening paragraphs.
2024-11-24 19:38:30 -08:00
Jeff McCune
2184bda2a1 v1alpha5: add validators (#357) 2024-11-24 17:40:41 -08:00
Jeff McCune
a8ab4dabaa cue: fix holos cue vet exit code (#358)
Without this patch `holos cue vet` always returns exit code 0, even when
there are errors.

This patch fixes the problem by catching the error and returning it to
our own top level error handler.  Note the final error, "could not run:
terminating because of errors" which wraps the generic error reported by
cue in the presence of multiple errors.

Result:

```
❯ holos cue vet ./policy --path 'strings.ToLower(kind)' /tmp/podinfo.gen.yaml
deployment.kind: conflicting values "Forbidden" and "Deployment":
    ./policy/validations.cue:18:8
    ../../../../../tmp/podinfo.gen.yaml:25:7
deployment.spec.template.spec.containers.0.resources.limits: conflicting values null and {[string]:"k8s.io/apimachinery/pkg/api/resource".#Quantity} (mismatched types null and struct):
    ./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:355:9
    ./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:376:12
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2840:11
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2968:14
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:15
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:18
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:5027:9
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:6407:16
    ./policy/validations.cue:17:13
    ../../../../../tmp/podinfo.gen.yaml:104:19
could not run: terminating because of errors
```
2024-11-24 16:39:56 -08:00
Jeff McCune
7175950ce0 docs: redirect /docs/overview permalink 2024-11-24 08:58:08 -08:00
Jeff McCune
e186c1be37 docs: edits for grammar 2024-11-21 15:55:14 -08:00
Jeff McCune
a998513e34 docs: edit for grammar and make concise 2024-11-21 15:26:20 -08:00
Jeff McCune
2f821ec33c docs: add environments topic page (#354)
Similar to the Clusters topic, add a topic about configuring multiple
environments.  This likely needs some work, the example is a bit
contrivied but at least shows how we can look up attributes, then use
those attributes to look up additional configuration from platform-wide
configuration data.
2024-11-21 15:07:46 -08:00
Jeff McCune
4d54190f2e docs: remove empty topic pages
We'll add them back in as we write them.
2024-11-21 14:40:39 -08:00
Gary Larizza
b466ec3457 Merge pull request #353 from holos-run/gl/fix-platform-args
Remove deprecated argument to 'holos render platform'
2024-11-21 13:04:17 -08:00
Gary Larizza
45120797b9 Remove deprecated argument to 'holos render platform'
This commit removes the extra `./platform` argument from any of the
current tutorial/topic docs to reflect the change that was made to
`holos render platform` in version `0.100.0`.
2024-11-21 11:49:44 -08:00
Jeff McCune
e6d25bf5eb holos: improve OrderedEncoder error handling
If someone accidentally provides the same index multiple times, or
indexes less than the next expected, the program would silently discard
the data.  This would be difficult to troubleshoot since an
OrderedEncoder is usually used with concurrent go routines, which would
likely mislead the investigator.

Better to just fail hard with an error indicating the caller in these
situations.
2024-11-20 17:02:16 -08:00
Jeff McCune
3ad6e69336 docs: update report issue template
Show how we can copy and paste directly from the issue into testscript.
2024-11-20 15:58:02 -08:00
Gary Larizza
9b10e23e43 Redirect docs to current version (#352)
* Redirect docs to current version

PROBLEM:

See #350 for context. There's a GitHub issue open for this on the
facebook docusaurus repo:
https://github.com/facebook/docusaurus/issues/9049

SOLUTION:

Use the redirect workaround for the time being.

OUTCOME:

https://holos.run/docs will link to the most recent version of our docs
site.

* Update doc/website/static/_redirects

Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>

---------

Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>
Co-authored-by: Jeff McCune <jeff@openinfrastructure.co>
2024-11-20 15:33:14 -08:00
Jeff McCune
b19022d3ff tests: fill in the buildplan.yaml for better coverage (#348)
Assert against the complete build plan so we know if we change the
output format in the future.

It's easy to update if so:

  HOLOS_UPDATE_SCRIPTS=1 go test github.com/holos-run/holos/cmd/holos
2024-11-20 15:21:41 -08:00
Jeff McCune
3fd06f594b schemas: fix kustomize patch name field (#348)
Without this patch trying to use a Kustomize patch with the optional
name field omitted results in the following error:

  could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
  holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
      $WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2

This patch fixes the problem by providing a default value for the name
field matching the Go zero value for a string.
2024-11-20 15:08:44 -08:00
Jeff McCune
a75338f21c docs: add bug report issue template, disable dev deploy 2024-11-20 14:50:25 -08:00
Jeff McCune
6002040360 version 0.100.0 2024-11-20 11:34:04 -08:00
Jeff McCune
864d7d442b build: pass platform component labels and annotations to BuildPlan
Without this patch the BuildPlan resulting from a Platform that has
components with labels and annotations does not have the labels or
annotations of the source component.

Holos should copy the labels and annotations defined on each of the
Platform.spec.components to the resulting BuildPlan so end users can see
clearly where a BuildPlan originated from, and filter with selectors the
intermediate output BuildPlan the same way we filter with selectors the
original Platform spec components list.

Result:

```
holos init platform v1alpha5 --force
holos show buildplans  | head
```

```yaml
kind: BuildPlan
apiVersion: v1alpha5
metadata:
  name: podinfo
  labels:
    app.holos.run/cluster: local
    app.holos.run/name: podinfo
  annotations:
    app.holos.run/description: podinfo for cluster local
```
2024-11-20 11:34:03 -08:00
Jeff McCune
791c0a9ffd util: fix misleading RunCmd error message source location
The error message misleads the reader to the utility function.  It
should lead to the caller.
2024-11-20 11:23:54 -08:00
Jeff McCune
bfe4a8d7c4 remove unmaintained embedded platforms
v1alpha5 is current and maintained.
2024-11-20 11:23:54 -08:00
Jeff McCune
eaa508a6f8 remove v1alpha1 schemas and support 2024-11-20 11:23:53 -08:00
Jeff McCune
63256a2845 remove embedded k3d platform
No longer used, v1alpha5 is used in the docs and maintained.
2024-11-20 11:23:53 -08:00
Jeff McCune
937c1dc953 show: fix inconsistent ordering of output
Without this patch the holos show buildplans command returns results in
an inconsistent order.  This is a problem because the output should be
idempotent.

This patch fixes the problem by adding an EncodeSeq(idx int, v any) method to
the encoder interface.  idx represents the index position of the
Platform.spec.components list after selector filtering has been applied.

This patch modifies the json and yaml encoders to buffer out of order
results from the concurrent go routines.

Result:

Concurrent execution is preserved. The buffer is kept to a reasonable
size, entries are deleted once they're encoded in the correct order.

Most importantly the output is consistent and idempotent so we can write
effective integration tests.
2024-11-20 11:23:52 -08:00
Jeff McCune
11bd50e2eb show: fix buildplans selector inconsistency
Sometimes, but not always, the holos show buildplans command produces no
output.

```
❯ holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug
finalized config from flags
rendered platform in 13.458µs
```

It only happens when there's a selector.  It doesn't happen without the
selector flag.  It only happens with ==, not with =.

This test fails quickly.

```
while [[ $(holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug | wc -l) -eq 39 ]]; do true; done
```

This test runs until killed.

```
while [[ $(holos show buildplans --log-level=debug | wc -l) -eq 279 ]]; do true; done
```

Solution:

The problem is the use of the map.  Iterating over the keys happens in a
random order.  With the fix we check in an explicit order.
2024-11-20 11:23:52 -08:00
Jeff McCune
7cfcf55565 add yaml struct tags to core v1alpha5 schemas
Without this patch the `holos show buildplans` BuildPlan output has
incorrect yaml with the v3 encoder.  For example apiversion: v1alpha5
instead of apiVersion.
2024-11-20 11:23:52 -08:00
Jeff McCune
8b22ba04e1 cli: add show command and refactor interfaces (#331)
Show subcommand:

This is large change that accomplishes a number of goals.  First, there
was no convenient way to show a build plan without using the debug logs
to indentify the tags to inject, then calling the cue command with the
right incantation to inspect the BuildPlan.

This patch addresses the problem by adding a `holos show buildplans`
command.  The command loads the Platform spec from the platform
directory, then iterates over all Components to produce the BuildPlan.

This patch adds labels and annotations to the platform Components
collection in order to select and filter the output.

Result:

```
❯ holos show components --selector app.holos.run/cluster=local --format=yaml | head
kind: BuildPlan
apiversion: v1alpha5
metadata:
  name: podinfo
spec:
  artifacts:
    - artifact: clusters/local/components/podinfo/podinfo.gen.yaml
      generators:
        - kind: Helm
          output: helm.gen.yaml
```

---

Interface refactor:

This refactors the interface between the `holos` Go CLI layer and the
various core schema data structures.  We now use a proper Go interface.
Concurrent execution over platform components has been improved to
accept a closure function so we can use the same interface method to
process the components.  We use this to show each component and render
each component from different subcommands using the same interface
embedded in the builder.Platform struct.

The embedded interface allows us to easily swap in different versions,
e.g. v1beta1 and eventually v1.  The number of interface methods are
quite small.  14 methods across 4 interfaces in holos/interface.go.

---

Remove old versions:

This patch removes support for versions prior to v1alpha5 in an effort
to clean up cruft.
2024-11-20 11:23:51 -08:00
Nate McCurdy
7d5187873b docs: Remove non-existent CLI completions, add pwsh
Cobra provides completions for bash, zsh, fish, and powershell, not
ksh.
2024-11-18 10:27:33 -08:00
Jeff McCune
03b796312a cli: gate grpc client and auth flags behind feature flag
Previously the holos render platform and component subcommands had flags
for oidc authentication and client access to the gRPC service.  These
flags aren't currently used, they're remnants from the json powered form
prototype.

This patch gates the flags behind a feature flag which is disabled by
default.

Result:

  holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int   number of components to render concurrently (default 8)
  -v, --version           version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")

---

  HOLOS_FEATURE_CLIENT=1 holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int             number of components to render concurrently (default 8)
      --oidc-client-id string       oidc client id. (default "270319630705329162@holos_platform")
      --oidc-extra-scopes strings   optional oidc scopes
      --oidc-force-refresh          force refresh
      --oidc-issuer string          oidc token issuer url. (default "https://login.holos.run")
      --oidc-scopes strings         required oidc scopes (default openid,email,profile,groups,offline_access)
      --server string               server to connect to (default "https://app.holos.run:443")
  -v, --version                     version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")
2024-11-17 16:06:57 -08:00
Jeff McCune
20fb39e49b docs: add clusters topic (#343)
Previously we didn't have good documentation on how to manage multiple
sets of clusters.

This patch adds a clusters topic in the structures category.  Each one
of the environments, projects, owners, etc... structures follow the same
pattern as #Clusters and #ClusterSets, so it makes sense to put them
into a dedicated sidebar category for specific CUE structures.
2024-11-17 14:45:32 -08:00
Jeff McCune
c9c8c13810 docs: replace touch with cat 2024-11-15 09:54:09 -07:00
Jeff McCune
374cd872e9 docs: CUE not cue and typo fix in hello holos 2024-11-15 09:40:02 -07:00
Jeff McCune
8db06dd0e1 releaser: fix brew test command (#327)
holos version isn't a valid command but holos --version is.
2024-11-14 16:14:26 -07:00
Jeff McCune
66acadf86d docs: support brew install (#327) 2024-11-14 16:07:13 -07:00
Jeff McCune
032f72b435 render: log helm pull errors (#332)
Previously errors were not logged, giving no indication what went wrong.
This patch changes the error handler to log errors from helm.
2024-11-14 09:44:27 -07:00
Jeff McCune
2380223794 docs: add argocd application example (#340)
When we moved from v1alpha4 to v1alpha5 we removed ArgoConfig from the
author schema.  There was no longer a clear example of how to configure
an ArgoCD Application for every component in v1alpha5.

This patch adds a topic document with an example of how to add an
Application along side the resources by mixing an additional Artifact
into the BuildPlan.
2024-11-13 16:30:59 -07:00
Jeff McCune
e6892c3b16 v0.99.0 2024-11-13 12:49:28 -07:00
Jeff McCune
847fd2958e helm: add support for helm template --kube-version capabilities (#330)
Previously the Helm generator had no support for the --kube-version
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:43:01 -07:00
Jeff McCune
cf622835db helm: add support for helm template --api-versions capabilities (#330)
Previously the Helm generator had no support for the --api-versions
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:42:50 -07:00
Jeff McCune
1f5dc3a082 docs: add note about tested helm version (#335)
To help users understand what should definitely work.
2024-11-13 09:45:56 -07:00
Jeff McCune
9f4da68dc9 v0.98.2 2024-11-13 09:19:30 -07:00
Jeff McCune
2ee056be9f cue: fix panic with no args (#334)
Fixes:

```
❯ holos
panic: runtime error: slice bounds out of range [2:1]

goroutine 1 [running]:
github.com/holos-run/holos/internal/cli.newCueCmd(...)
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:121
github.com/holos-run/holos/internal/cli.New(0xc0002837c0, {0x3826e00, 0x4f60860})
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:102 +0x772
main.main.MakeMain.func1()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/main.go:22 +0x5b
main.main()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/cmd/holos/main.go:10 +0x3e
```
2024-11-13 09:04:37 -07:00
Jeff McCune
394e2cb0b2 docs: add cue tutorial (#318)
Show how to use the ComponentConfig Resources field to mix in resources.
2024-11-13 08:00:37 -07:00
Jeff McCune
cf95c9664d docs: change hello holos parameters to greeting (#328)
Version doesn't make as much sense since we're doing a hello world
tutorial.

Also consolidate the values information into one step, and consolidate
the breaking it down section to make it shorter and clearer.
2024-11-12 09:46:19 -07:00
Jeff McCune
0192eeeb7e docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest

This time in the correct directory.
2024-11-11 17:25:17 -07:00
Jeff McCune
ed54bcc58f docs: rename cue-generator to cue
The main use case is to manage resources from CUE, but CUE has many uses
in Holos such as validation and driving Kustomize.
2024-11-11 17:16:53 -07:00
Jeff McCune
9ac7f185f9 docs: fix broken validators link in diagram 2024-11-11 17:11:35 -07:00
Jeff McCune
7de72d3dab docs: add component parameters example to hello holos (#328)
The important note was weird because we didn't show an example of how to
use component parameters.  This patch replaces the note with an example.
2024-11-11 16:56:16 -07:00
Jeff McCune
2e3c998454 docs: add directory tree to hello holos doc (#324)
Feedback from Zack, give a tree so people skimming know where to figure
out the lay of the land.
2024-11-11 16:19:48 -07:00
Jeff McCune
580afffa7f 0.98.1 - holos init platform 2024-11-11 14:44:00 -07:00
Jeff McCune
67535e1e1d doc: remove init platform from setup guide
We have it in the hello guide, setup should just be install only and not
how to use the tool yet.
2024-11-11 14:41:58 -07:00
Nate McCurdy
767ea69d2e docs: Add a tree view to Hello Holos
A tree view of the `holos-tutorial/` directory should give readers a
quick, high-level understanding of the folder structure of a typical
Holos platform project.
2024-11-11 14:04:40 -07:00
Jeff McCune
21e1a116e4 cli: hide help flags and command (#325)
They're unnecessary.
2024-11-11 14:02:25 -07:00
Jeff McCune
65fe7779be cli: rename generate to init (#325)
This patch changes the `holos generate` command to `holos init` to match
other tools like `go mod init`.
2024-11-11 14:02:25 -07:00
Jeff McCune
0e7abf0173 docs: consolidate diagrams to @site/src/diagrams/
So we don't have two different copies in two different places.
2024-11-11 13:40:46 -07:00
Jeff McCune
cca022ac99 docs: move architecture diagrams (#323) 2024-11-11 12:03:21 -07:00
Jeff McCune
43e939d06a doc: refactor breaking it down table in hello holos
So it displays nicely on mobile.
2024-11-09 14:52:11 -08:00
Jeff McCune
8096268826 docs: fix diagram urls again 2024-11-09 14:40:08 -08:00
Jeff McCune
631b23091d docs: fix rendering overview diagram on blog 2024-11-09 14:33:54 -08:00
Jeff McCune
09c6476282 docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-09 14:30:44 -08:00
Jeff McCune
a768d16c5f docs: set current version to v1alpha5
Previously the current version would always be unreleased at /docs/next
and we'd have to copy the doc/md/ folder into the
doc/website/versioned_docs/version-v1alpha5/ every time we made a
change.

We're going to be working on v1alpha5 for awhile so it makes sense to
just have the current version published at /docs/v1alpha5/ and we can
start /docs/v1alpha6/ whenever we're ready.

This also has the nice effect of giving us permalinks if we change the
structure again.  /docs/v1alpha5/ will remain over time.
2024-11-09 14:29:27 -08:00
Jeff McCune
3834a7ef85 docs: add missing link to kustomize tutorial 2024-11-08 22:25:34 -08:00
Jeff McCune
606a1aae73 docs: add nav bar title back 2024-11-08 19:29:05 -08:00
Jeff McCune
340d07ee7a docs: fix announcing holos blog (#321) 2024-11-08 19:24:46 -08:00
Jeff McCune
12d2cec4d5 docs: fix rendering overview diagram links (#321) 2024-11-08 19:12:23 -08:00
Jeff McCune
e93feb49b7 docs: add version drop down to nav bar (#321) 2024-11-08 17:05:37 -08:00
Jeff McCune
dcf8602a0b docs: release v1alpha5 (#321) 2024-11-08 17:00:34 -08:00
Jeff McCune
e07c4d11c8 docs: revise helm values and kustomize tutorials (#316)
These are now where I'd like them to be.
2024-11-08 15:12:22 -08:00
Jeff McCune
b7e1c14192 docs: kustomize tutorial (#316)
Add httpbin using kustomize and patch the result, all from CUE.  The is
the second half of the v1alpha4 helm guide split into a dedicated
tutorial.
2024-11-08 14:08:48 -08:00
Jeff McCune
29f44cdac9 docs: helm values (#316)
Add a helm values tutorial which is a cut down version of the v1alpha4
helm guide.  The httpbin kustomize will immediately follow building on
the prometheus and blackbox charts.
2024-11-08 12:03:14 -08:00
Jeff McCune
96be7a4ae3 docs: add generate platform to hello holos (#311) 2024-11-07 17:59:07 -08:00
Jeff McCune
d6bd030a72 docs: fixup overview 2024-11-07 17:59:07 -08:00
Jeff McCune
75047b590f docs: hello holos edits (#311) 2024-11-07 17:19:53 -08:00
Gary Larizza
a05881df0f Add the Hello Holos tutorial 2024-11-07 16:47:36 -08:00
Jeff McCune
5f406fce5c docs: organize docs for new structure (#301) 2024-11-07 15:10:15 -08:00
Jeff McCune
49c945a037 docs: setup tutorial with diagrams (#301) 2024-11-07 14:13:49 -08:00
Gary Larizza
54de20f0b8 docs: setup tutorial (#301) 2024-11-07 10:24:49 -08:00
Gary Larizza
80b4ab9852 Merge pull request #310 from holos-run/jeff/308-holos-field
docs: rewrite technical overview for v1alpha5
2024-11-07 10:15:01 -08:00
Jeff McCune
acd98aa63c docs: rewrite technical overview for v1alpha5
Attribution: following the structure and length of the tokio docs, with
some more diagrams.
2024-11-07 10:07:45 -08:00
Jeff McCune
0afaab8f2b render: nest output under the holos top level field (#308)
Previously the holos command line expected a Platform and BuildPlan
resource at the top level of the exported data from CUE.  This forced us
to use hidden fields for everything else.

This patch modifies the BuildData struct to first look for a holos top
level field and use it if present.  This opens up other top level fields
for use by end users.

Our intent is to reserve any top level field prefixed with holos.

Note this follows how Timoni works as well.
2024-11-07 07:00:26 -08:00
Jeff McCune
7ded38bc3f v1alpha5: strip down the core and author schemas (#306)
This patch strips down the v1alpha4 core and author schemas to only with
is absolutely necessary for all holos users.  Aspects of platform
configuration applicable to some, even most, but not all users will be
moved into documentation topics organized as a recipe book.

The functionality removed from the v1alpha4 author schemas in v1alpha5
will move into self contained examples documented as topics on the docs
site.

The overall purpose is to have a focused, composeable, maintainable
author schema to help people get started and ideally we can support for
years with making breaking changes.

With this patch the v1alpha5 helm guide test passes.  We're not going to
have this guide anymore but it demonstrates we're back to where we were
with v1alpha4.
2024-11-06 15:22:17 -08:00
Jeff McCune
840676709a docs: partially write the overview doc for v1alpha5
Partial, switching gears to v1alpha5 to unblock others working on
discrete topics.
2024-11-05 10:19:59 -08:00
Jeff McCune
ee30c52673 docs: generate version specific api docs (#303)
Without this patch each version of the core and author schemas are
duplicated into each docs version.  This is unnecessary and difficult to
maintain now that we have docusaurus versioned docs enabled.

This patch updates the schema generation script to check if the docs
version has been released, and if so write into a markdown file in the
versioned docs folder.  If not, the version is written into the next
version folder.

This patch also updates some, but not all, document links to the md or
mdx relative file paths.  This is necessary to generate the correct
versioned links.

A nice outcome of this change is that technical docs no longer need to
link to version specific pages.  For example, `[Core Schema]:
./api/core.md` will always refer to the correct auto generated docs
associated with the docs version.
2024-11-05 07:20:53 -08:00
Jeff McCune
117a00334f docs: restructure docs into tutorial and topics (#301)
The docs for v1alpha4 have the right information, but in the wrong
places.  The most important bits are tucked away in the Core API docs.
One of our first users entirely missed the `holos generate platform`
command mentioned in the Helm guide.

We'll fix this by organizing the docs into two distinct categories.
First, a tutorial written as a series progressively building up the
minimum knowledge to use holos effectively and gain the benefits.  Think
of it as a tour of the essential bits.

The second category are focused topics which stand alone.  They're the
things most people using holos will need to know eventually, but aren't
essential for everyone to know.  For example, Clusters and Fleets will
move from the Author API to stand alone examples of how to implement
these features if necessary.

Then there's a Glossary which serves as the place to describe our
concepts and domain specific language.

Finally there's the API documentation which should be cut down to the
specific version.  The next release version will be v1alpha5.

Attribution: We're copying the Tokio docs structure, it's concise and a
similar size and complexity to our own project.

The Go docs are also an inspiration, but the project is much larger so
not directly comparable.  The organization of https://go.dev/doc/ feels
complete at first glance, despite the size and age of the project.  The
site also makes clear who each section is for without needing to come
right out and say it. Getting started, Using and understanding Go,
Writing modules, using databases, etc...
2024-11-04 20:25:04 -08:00
Jeff McCune
1e03debfac tests: add make unity target
For https://cuelabs.dev/unity/
2024-11-04 19:08:44 -08:00
Jeff McCune
72137b2fa9 docs: upgrade docusaurus to 3.6.0
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-04 06:47:48 -08:00
Jeff McCune
5abf967116 docs: npm run docusaurus docs:version v1alpha4 (#299)
Tag version v1alpha4 so we can start working on v1alpha5 as the next
version in main.
2024-11-04 06:43:15 -08:00
Jeff McCune
5d882f465d website: fix resources.yaml tab in helm guide (#293)
We switched from using a kustomize remote base to a local file so the
tests don't need to make a network round trip to github.  It's also
better practice to use local files for this sort of thing.

In doing so I botched the location of the file, putting it in the
platform registration section.  This patch claifies how `resources.yaml`
is linked to `httpbin.cue` through the `KustomizeConfig: Files:
"resources.yaml": _` field.
2024-11-03 10:57:52 -08:00
Jeff McCune
45bdaac833 main: cue v0.10.1 and add e2e test for helm guide (#293)
Previously there was no test coverage of the
https://holos.run/docs/guides/helm/ guide.  This patch uses Roger's
testscript package, which the CUE folks also use to add comprehensive
test coverage of each step in the guide.  Ideally we would execute these
commands directly from the guide itself, but for now we'll duplicate the
commands into the test script.  This could be enhanced by generating the
test script from the document itself in some way.

When updating the script, use the `holos txtar` command to embed entire
helm charts into the test script.  It's not super fast, but it's better
than network access and it's not terribly slow either.  A few seconds to
unpack.

---

txtar: quote files for testscript unquote

For the helm guide test script we want to include the entire helm chart
which may have files that need to be quoted.  This patch changes the
default behavior of the holos txtar command to quote files if necessary
and list them in an unquote script command in the comment of the
archive.

The purpose is for testscript authors to copy and paste the entire thing
into a test script and include the unquote command at the top.

---

This change also updates CUE to v0.10.1
2024-11-03 10:27:46 -08:00
Jeff McCune
7ae1f990ef website: update quickstart diagram to match helm
Avoid confusion, got a question about this in discord.
2024-11-03 08:53:43 -08:00
Jeff McCune
b526fd1669 testdata: clean up old v1alpha1 tests (#292)
No longer necessary now that we're on v1alpha4.  Test coverage for
v1alpha4 and the user facing guides will be added back soon for use both
in the holos repo and in Unity.
2024-11-01 15:22:13 -07:00
Jeff McCune
5e07655f35 website: fix port in helm guide
Should be 9115 not 6115.
2024-11-01 06:39:48 -07:00
837 changed files with 66357 additions and 189713 deletions

View File

@@ -6,10 +6,12 @@
],
"words": [
"acmesolver",
"acraccesstoken",
"acraccesstokens",
"admissionregistration",
"alertmanager",
"alertmanagers",
"anchore",
"anthos",
"apiextensions",
"apimachinery",
@@ -27,19 +29,25 @@
"authpolicy",
"authproxy",
"authroutes",
"autoload",
"automount",
"automounting",
"autoscaler",
"balancereader",
"blackbox",
"buildplan",
"buildplans",
"Buildx",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
"cainjector",
"CAROOT",
"certificaterequest",
"certificaterequests",
"certificatesigningrequests",
"chartmuseum",
"clientset",
"clsx",
"clusterexternalsecret",
"clusterexternalsecrets",
@@ -50,8 +58,12 @@
"clustersecretstore",
"clustersecretstores",
"clusterwide",
"Cmds",
"CNCF",
"CODEOWNERS",
"compinit",
"componentconfig",
"configdir",
"configmap",
"configmapargs",
"connectrpc",
@@ -63,6 +75,7 @@
"creds",
"crossplane",
"crunchydata",
"ctxt",
"cuecontext",
"cuelang",
"customresourcedefinition",
@@ -70,9 +83,12 @@
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devel",
"devicecode",
"distroless",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtoken",
"ecrauthorizationtokens",
"edns",
"endpointslices",
@@ -88,9 +104,11 @@
"fieldmaskpb",
"fieldspec",
"flushcache",
"fluxcd",
"fullname",
"gatewayclass",
"gatewayclasses",
"gcraccesstoken",
"gcraccesstokens",
"gendoc",
"generationbehavior",
@@ -99,8 +117,10 @@
"genproto",
"ggnpl",
"ghaction",
"githubaccesstoken",
"githubaccesstokens",
"gitops",
"GOBIN",
"godoc",
"golangci",
"gomarkdoc",
@@ -128,6 +148,7 @@
"httproute",
"httproutes",
"iampolicygenerator",
"incpatch",
"Infima",
"intstr",
"isatty",
@@ -137,6 +158,7 @@
"jetstack",
"jiralert",
"Jsonnet",
"Kargo",
"kfbh",
"killall",
"kubeadm",
@@ -144,6 +166,7 @@
"kubelet",
"kubelogin",
"kubernetesobjects",
"kubeversion",
"Kustomization",
"Kustomizations",
"kustomize",
@@ -160,6 +183,7 @@
"loadbalancer",
"loadrestrictions",
"logfmt",
"lxnl",
"mattn",
"mccutchen",
"metav",
@@ -170,7 +194,9 @@
"Multicluster",
"mutatingwebhookconfiguration",
"mutatingwebhookconfigurations",
"mvdan",
"mxcl",
"mychart",
"myhostname",
"myRegistrKeySecretName",
"mysecret",
@@ -184,9 +210,11 @@
"oauthproxy",
"objectmap",
"objectmeta",
"omitempty",
"organizationconnect",
"orgid",
"otelconnect",
"outfile",
"overriden",
"Parentspanid",
"patchstrategicmerge",
@@ -238,9 +266,11 @@
"requestauthentications",
"resourcequotas",
"retryable",
"rogpeppe",
"rolebinding",
"rootfs",
"ropc",
"sboms",
"seccomp",
"secretargs",
"SECRETKEY",
@@ -253,6 +283,7 @@
"serviceentries",
"serviceentry",
"servicemonitor",
"sigstore",
"somevalue",
"SOMEVAR",
"sortoptions",
@@ -274,6 +305,7 @@
"systemconnect",
"tablewriter",
"templatable",
"testscript",
"thanos",
"Tiltfile",
"timestamppb",
@@ -282,13 +314,17 @@
"tokencache",
"Tokener",
"tolerations",
"TOPLEVEL",
"Traceid",
"traefik",
"transactionhistory",
"tsdb",
"txtar",
"typemeta",
"udev",
"uibutton",
"Unmarshal",
"unshallow",
"unstage",
"untar",
"upbound",
@@ -300,12 +336,14 @@
"userservice",
"validatingwebhookconfiguration",
"validatingwebhookconfigurations",
"vaultdynamicsecret",
"vaultdynamicsecrets",
"virtualservice",
"virtualservices",
"volumeattachments",
"wasmplugin",
"wasmplugins",
"workdir",
"workloadentries",
"workloadentry",
"workloadgroup",

131
.github/ISSUE_TEMPLATE/bug-report.md vendored Normal file
View File

@@ -0,0 +1,131 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: NeedsInvestigation, Triage
assignees: ''
---
<!--
Please answer these questions before submitting your issue. Thanks!
To ask questions, see https://github.com/holos-run/holos/discussions
-->
### What version of holos are you using (`holos --version`)?
```
0.0.0
```
### Does this issue reproduce with the latest release?
<!--
Get the latest release with:
brew install holos-run/tap/holos
Or see https://holos.run/docs/v1alpha5/tutorial/setup/
-->
### What did you do?
<!--
Please provide a testscript that should pass, but does not because of the bug.
See the below example.
You can create a txtar from a directory with:
holos txtar ./path/to/dir
Refer to: https://github.com/rogpeppe/go-internal/tree/master/cmd/testscript
-->
Steps to reproduce:
```shell
testscript -v -continue <<EOF
```
```txtar
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work.
exec holos --version
exec holos init platform v1alpha5 --force
# remove the fix to trigger the bug
rm cue.mod/pkg/sigs.k8s.io/kustomize/api/types/var.cue
# want a BuildPlan shown
exec holos show buildplans
cmp stdout buildplan.yaml
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- buildplan.yaml --
kind: BuildPlan
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
```
```shell
EOF
```
### What did you expect to see?
The testscript should pass.
### What did you see instead?
The testscript fails because of the bug.
```txt
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work. (0.168s)
> exec holos --version
[stdout]
0.100.1-2-g9b10e23-dirty
> exec holos init platform v1alpha5 --force
# remove the fix to trigger the bug (0.000s)
> rm cue.mod/pkg/sigs.k8s.io/kustomize/api/types/var.cue
# want a BuildPlan shown (0.091s)
> exec holos show buildplans
[stderr]
could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
$WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2
[exit status 1]
FAIL: <stdin>:8: unexpected command failure
> cmp stdout buildplan.yaml
diff stdout buildplan.yaml
--- stdout
+++ buildplan.yaml
@@ -0,0 +1,1 @@
+kind: BuildPlan
FAIL: <stdin>:9: stdout and buildplan.yaml differ
# want this error to go away (0.000s)
> ! stderr 'cannot convert non-concrete value string'
FAIL: <stdin>:11: unexpected match for `cannot convert non-concrete value string` found in stderr: cannot convert non-concrete value string
failed run
```

143
.github/workflows/container.yaml vendored Normal file
View File

@@ -0,0 +1,143 @@
name: Container
# Only allow actors with write permission to the repository to trigger this
# workflow.
permissions:
contents: write
on:
push:
tags:
- 'v*'
workflow_dispatch:
inputs:
git_ref:
description: 'Git ref to build (e.g., refs/tags/v1.2.3, refs/heads/main)'
required: true
type: string
jobs:
buildx:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
attestations: write
id-token: write
steps:
- name: Set tag from trigger event
id: opts
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "ref=${{ inputs.git_ref }}" >> $GITHUB_OUTPUT
else
echo "ref=${GITHUB_REF}" >> $GITHUB_OUTPUT
fi
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ steps.opts.outputs.ref }}
- name: SHA
id: sha
run: echo "sha=$(/usr/bin/git log -1 --format='%H')" >> $GITHUB_OUTPUT
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Fetch tags
run: git fetch --prune --unshallow --tags
- name: Set Tags
id: tags
run: |
echo "detail=$(/usr/bin/git describe --tags HEAD)" >> $GITHUB_OUTPUT
echo "suffix=$(test -n "$(git status --porcelain)" && echo '-dirty' || echo '')" >> $GITHUB_OUTPUT
echo "tag=$(/usr/bin/git describe --tags HEAD)$(test -n "$(git status --porcelain)" && echo '-dirty' || echo '')" >> $GITHUB_OUTPUT
- name: Login to ghcr.io
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push container images
id: build-and-push
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: |
ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}
ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}
- name: Setup Cosign to sign container images
uses: sigstore/cosign-installer@v3.7.0
- name: Sign with GitHub OIDC Token
env:
DIGEST: ${{ steps.build-and-push.outputs.digest }}
run: |
cosign sign --yes ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST}
cosign sign --yes ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST}
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Get GitHub App User ID
id: get-user-id
run: echo "user-id=$(gh api "/users/${{ steps.app-token.outputs.app-slug }}[bot]" --jq .id)" >> "$GITHUB_OUTPUT"
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
- run: |
git config --global user.name '${{ steps.app-token.outputs.app-slug }}[bot]'
git config --global user.email '${{ steps.get-user-id.outputs.user-id }}+${{ steps.app-token.outputs.app-slug }}[bot]@users.noreply.github.com'
- name: Update holos-run/holos-action
env:
IMAGE: ghcr.io/holos-run/holos:v0.102.1
VERSION: ${{ steps.tags.outputs.tag }}
USER_ID: ${{ steps.get-user-id.outputs.user-id }}
TOKEN: ${{ steps.app-token.outputs.token }}
run: |
set -euo pipefail
git clone "https://github.com/holos-run/holos-action"
cd holos-action
git remote set-url origin https://${USER_ID}:${TOKEN}@github.com/holos-run/holos-action
docker pull --quiet "${IMAGE}"
docker run -v $(pwd):/app --workdir /app --rm "${IMAGE}" \
holos cue export --out yaml action.cue -t "version=${VERSION}" > action.yml
git add action.yml
git commit -m "ci: update holos to ${VERSION} - https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" || (echo "No changes to commit"; exit 0)
git push origin HEAD:main HEAD:v0 HEAD:v1
- name: Login to quay.io
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USER }}
password: ${{ secrets.QUAY_TOKEN }}
- name: Push to quay.io
env:
DIGEST: ${{ steps.build-and-push.outputs.digest }}
run: |
# docker push quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}
docker pull --quiet ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST}
docker tag ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST} \
quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}
docker push quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}
docker pull --quiet ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST}
docker tag ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST} \
quay.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}
docker push quay.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}
- name: Sign quay.io image
env:
DIGEST: ${{ steps.build-and-push.outputs.digest }}
run: |
cosign sign --yes quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST}
cosign sign --yes quay.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST}
outputs:
tag: ${{ steps.tags.outputs.tag }}
detail: ${{ steps.tags.outputs.detail }}

View File

@@ -2,7 +2,7 @@ name: Dev Deploy
on:
push:
branches: ['main', 'dev-deploy']
branches: ['dev-deploy']
jobs:
deploy:

View File

@@ -1,6 +1,5 @@
---
# https://github.com/golangci/golangci-lint-action?tab=readme-ov-file#how-to-use
name: Lint
name: Spelling
"on":
push:
branches:
@@ -8,35 +7,11 @@ name: Lint
- test
pull_request:
types: [opened, synchronize]
permissions:
contents: read
jobs:
lint:
name: lint
cspell:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: stable
## Not needed on ubuntu-latest
# - name: Install Packages
# run: sudo apt update && sudo apt -qq -y install git curl zip unzip tar bzip2 make
- name: Install Tools
run: make tools
- name: Lint
# golangci-lint runs in a separate workflow.
run: make lint -o golangci-lint
- uses: actions/checkout@v4
- run: ./hack/cspell

View File

@@ -35,6 +35,9 @@ jobs:
with:
go-version: stable
- name: Setup Syft
uses: anchore/sbom-action/download-syft@1ca97d9028b51809cf6d3c934c3e160716e1b605 # v0.17.5
# Necessary to run these outside of goreleaser, otherwise
# /home/runner/_work/holos/holos/internal/frontend/node_modules/.bin/protoc-gen-connect-query is not in PATH
- name: Install Tools
@@ -54,11 +57,19 @@ jobs:
- name: Git diff
run: git diff
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
version: '~> v2'
args: release --clean
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -13,7 +13,7 @@ permissions:
jobs:
test:
runs-on: gha-rs
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -28,19 +28,11 @@ jobs:
with:
go-version: stable
- name: Install Packages
run: sudo apt update && sudo apt -qq -y install git curl zip unzip tar bzip2 make
- name: Set up Helm
uses: azure/setup-helm@v4
- name: Set up Kubectl
uses: azure/setup-kubectl@v3
- name: Install Tools
run: |
set -x
make tools
uses: azure/setup-kubectl@v4
- name: Test
run: ./scripts/test

View File

@@ -6,7 +6,7 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
version: 2
before:
hooks:
@@ -50,3 +50,39 @@ changelog:
exclude:
- "^docs:"
- "^test:"
source:
enabled: true
name_template: '{{ .ProjectName }}_{{ .Version }}_source_code'
sboms:
- id: source
artifacts: source
documents:
- "{{ .ProjectName }}_{{ .Version }}_sbom.spdx.json"
brews:
- name: holos
repository:
owner: holos-run
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
homepage: "https://holos.run"
description: "Holos CLI"
dependencies:
- name: helm
type: optional
- name: kubectl
type: optional
install: |
bin.install "holos"
bash_output = Utils.safe_popen_read(bin/"holos", "completion", "bash")
(bash_completion/"holos").write bash_output
zsh_output = Utils.safe_popen_read(bin/"holos", "completion", "zsh")
(zsh_completion/"_holos").write zsh_output
fish_output = Utils.safe_popen_read(bin/"holos", "completion", "fish")
(fish_completion/"holos.fish").write fish_output
test: |
system "#{bin}/holos --version"

View File

@@ -1,8 +1,31 @@
FROM quay.io/holos-run/debian:bullseye AS final
USER root
WORKDIR /app
ADD bin bin
RUN chown -R app: /app
# Kubernetes requires the user to be numeric
USER 8192
ENTRYPOINT bin/holos server
FROM registry.k8s.io/kubectl:v1.31.0 AS kubectl
# https://github.com/GoogleContainerTools/distroless
FROM golang:1.23 AS build
WORKDIR /go/src/app
COPY . .
RUN CGO_ENABLED=0 make install
RUN CGO_ENABLED=0 go install sigs.k8s.io/kustomize/kustomize/v5
# Install helm to /usr/local/bin/helm
# https://helm.sh/docs/intro/install/#from-script
# https://holos.run/docs/v1alpha5/tutorial/setup/#dependencies
RUN curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 \
&& chmod 700 get_helm.sh \
&& DESIRED_VERSION=v3.16.2 ./get_helm.sh \
&& rm -f get_helm.sh
COPY --from=kubectl /bin/kubectl /usr/local/bin/
# distroless
FROM gcr.io/distroless/static-debian12 AS final
COPY --from=build \
/go/bin/holos \
/go/bin/kustomize \
/usr/local/bin/kubectl \
/usr/local/bin/helm \
/bin/
# Usage: docker run -v $(pwd):/app --workdir /app --rm -it quay.io/holos-run/holos holos render platform
CMD ["/bin/holos"]

View File

@@ -150,6 +150,10 @@ dev-deploy: install image ## deploy to dev
website: ## Build website
./hack/build-website
.PHONY: unity
unity: ## https://cuelabs.dev/unity/
./scripts/unity
.PHONY: help
help: ## Display this help menu.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)

129
README.md
View File

@@ -1,35 +1,130 @@
## Holos - A Holistic Development Platform
# Holos
<img width="50%"
align="right"
style="display: block; margin: 40px auto;"
src="https://openinfrastructure.co/blog/2016/02/27/logo/logorectangle.png">
Building and maintaining a software development platform is a complex and time
consuming endeavour. Organizations often dedicate a team of 3-4 who need 6-12
months to build the platform.
[Holos] is a configuration management tool for Kubernetes implementing the
[rendered manifests pattern]. It handles configurations ranging from single
resources to multi-cluster platforms across regions.
Holos is a tool and a reference platform to reduce the complexity and speed up
the process of building a modern, cloud native software development platform.
Key components:
- Platform schemas defining component integration
- Building blocks unifying Helm, Kustomize and Kubernetes configs with CUE
- BuildPlan pipeline for generating, transforming and validating manifests
- **Accelerate new projects** - Reduce time to market and operational complexity by starting your new project on top of the Holos reference platform.
- **Modernize existing projects** - Incrementally incorporate your existing platform services into Holos for simpler integration.
- **Unified configuration model** - Increase safety and reduce the risk of config changes with CUE.
- **First class Helm and Kustomize support** - Leverage and reuse your existing investment in existing configuration tools such as Helm and Kustomize.
- **Modern Authentication and Authorization** - Holos seamlessly integrates platform identity and access management with zero-trust beyond corp style authorization policy.
```mermaid
---
title: Rendering Overview
---
graph LR
Platform[<a href="https://holos.run/docs/v1alpha5/api/author/#Platform">Platform</a>]
Component[<a href="https://holos.run/docs/v1alpha5/api/author/#ComponentConfig">Components</a>]
## Quick Installation
Helm[<a href="https://holos.run/docs/v1alpha5/api/author/#Helm">Helm</a>]
Kustomize[<a href="https://holos.run/docs/v1alpha5/api/author/#Kustomize">Kustomize</a>]
Kubernetes[<a href="https://holos.run/docs/v1alpha5/api/author/#Kubernetes">Kubernetes</a>]
```console
go install github.com/holos-run/holos/cmd/holos@latest
BuildPlan[<a href="https://holos.run/docs/v1alpha5/api/core/#BuildPlan">BuildPlan</a>]
ResourcesArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">GitOps<br/>Artifact</a>]
Generators[<a href="https://holos.run/docs/v1alpha5/api/core/#Generator">Generators</a>]
Transformers[<a href="https://holos.run/docs/v1alpha5/api/core/#Transformer">Transformers</a>]
Validators[<a href="https://holos.run/docs/v1alpha5/api/core/#Validator">Validators</a>]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Validators --> Files
```
## Docs and Support
## Setup
The documentation for developing and using Holos is available at: https://holos.run
```shell
brew install holos-run/tap/holos
```
For discussion and support, [open a discussion](https://github.com/orgs/holos-run/discussions/new/choose).
Refer to [setup] for other installation methods and dependencies.
## Example
See our [tutorial] for a complete hello world example.
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
## Organizational Role
Platform engineers use Holos to generate Kubernetes manifests, both locally and
in CI pipelines. The manifests are committed to version control and deployed via
GitOps tools like ArgoCD or Flux.
Holos integrates seamlessly with existing Helm charts, Kustomize bases, and
other version-controlled configurations.
## Advantages of Holos
### Safe
Holos leverages [CUE] for strong typing and validation of configuration data,
ensuring consistent output from Helm and other tools.
### Consistent
A unified pipeline processes all configurations - whether from CUE, Helm, or
Kustomize - through the same well-defined stages.
### Flexible
Composable building blocks for generation, transformation, validation and
integration let teams assemble workflows that match their needs.
The core is intentionally unopinionated about platform configuration patterns.
Common needs like environments and clusters are provided as customizable
[topics] recipes rather than enforced structures.
## Getting Help
Get support through our [Discord] channel or [GitHub discussions]. Configuration
challenges arise at all experience levels - we welcome your questions and are
here to help.
## License
Holos is licensed under Apache 2.0 as found in the [LICENSE file](LICENSE).
[Holos]: https://holos.run/docs/overview/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[CUE]: https://cuelang.org/
[Discord]: https://discord.gg/JgDVbNpye7
[GitHub discussions]: https://github.com/holos-run/holos/discussions
[Why CUE for Configuration]: https://holos.run/blog/why-cue-for-configuration/
[tutorial]: https://holos.run/docs/overview/
[setup]: https://holos.run/docs/setup/
[topics]: https://holos.run/docs/topics/

View File

@@ -0,0 +1,152 @@
// Package author contains a standard set of schemas for component authors to
// generate common [core] BuildPlans.
//
// Holos values stability, flexibility, and composition. This package
// intentionally defines only the minimal necessary set of structures.
// Component authors are encouraged to define their own structures building on
// our example [topics].
//
// The Holos Maintainers may add definitions to this package if the community
// identifies nearly all users must define the exact same structure. Otherwise,
// definitions should be added as a customizable example in [topics].
//
// For example, structures representing a cluster and environment almost always
// need to be defined. Their definition varies from one organization to the
// next. Therefore, customizable definitions for a cluster and environment are
// best maintained in [topics], not standardized in this package.
//
// [core]: https://holos.run/docs/api/core/
// [topics]: https://holos.run/docs/topics/
package author
import core "github.com/holos-run/holos/api/core/v1alpha5"
//go:generate ../../../hack/gendoc
// Platform assembles a core Platform in the Resource field for the holos render
// platform command. Use the Components field to register components with the
// platform.
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// ComponentConfig represents the configuration common to all kinds of
// components for use with the holos render component command. All component
// kinds may be transformed with [kustomize] configured with the
// [KustomizeConfig] field.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the kustomize configuration.
KustomizeConfig KustomizeConfig
// Validators represent checks that must pass for output to be written.
Validators map[NameLabel]core.Validator
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
// Helm assembles a BuildPlan rendering a helm chart. Useful to mix in
// additional resources from CUE and transform the helm output with kustomize.
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kubernetes assembles a BuildPlan containing inline resources exported from
// CUE.
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize assembles a BuildPlan rendering manifests from a [kustomize]
// kustomization.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// KustomizeConfig represents the configuration for [kustomize] post processing.
// Use the Files field to mix in plain manifest files located in the component
// directory. Use the Resources field to mix in manifests from network urls.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
// NameLabel represents the common use case of converting a struct to a list
// where the name field of each value unifies with the field name of the outer
// struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,5 @@
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---

View File

@@ -1,4 +1,4 @@
---
description: Core v1alpha2 schema for advanced use cases.
sidebar_position: 996
description: Core schema for holos to render a component BuildPlan.
sidebar_position: 100
---

View File

@@ -0,0 +1,5 @@
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---

349
api/core/v1alpha5/types.go Normal file
View File

@@ -0,0 +1,349 @@
// Package core contains schemas for a [Platform] and [BuildPlan]. Holos takes
// a [Platform] as input, then iterates over each [Component] to produce a
// [BuildPlan]. Holos processes the [BuildPlan] to produce fully rendered
// manifests, each an [Artifact].
package core
//go:generate ../../../hack/gendoc
// BuildPlan represents an implementation of the [rendered manifest pattern].
// Holos processes a BuildPlan to produce one or more [Artifact] output files.
// BuildPlan artifact files usually contain Kubernetes manifests, but they may
// have any content.
//
// A BuildPlan usually produces two artifacts. One artifact contains a manifest
// of resources. A second artifact contains a GitOps resource to manage the
// first, usually an ArgoCD Application resource.
//
// Holos uses CUE to construct a BuildPlan. A future enhancement will support
// user defined executables providing a BuildPlan to Holos in the style of an
// [external credential provider].
//
// [rendered manifest pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Validators []Validator `json:"validators,omitempty" yaml:"validators,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// Each Generator in an [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents Kubernetes resources. Most commonly used to mix
// resources into the [BuildPlan] generated from CUE, but may be generated from
// elsewhere.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source" yaml:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
//
// The Auth field is useful to configure http basic authentication to the Helm
// repository. Holos gets the username and password from the environment
// variables represented by the Auth field.
type Repository struct {
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
Auth Auth `json:"auth,omitempty" yaml:"auth,omitempty"`
}
// Auth represents environment variable names containing auth credentials.
type Auth struct {
Username AuthSource `json:"username" yaml:"username"`
Password AuthSource `json:"password" yaml:"password"`
}
// AuthSource represents a source for the value of an [Auth] field.
type AuthSource struct {
Value string `json:"value,omitempty" yaml:"value,omitempty"`
FromEnv string `json:"fromEnv,omitempty" yaml:"fromEnv,omitempty"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
// outputs into one output. [Kustomize] is the most commonly used transformer.
// A simple [Join] is also supported for use with plain manifest files.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// FileContent represents file contents.
type FileContent string
// Validator validates files. Useful to validate an [Artifact] prior to writing
// it out to the final destination. Holos may execute validators concurrently.
// See the [validators] tutorial for an end to end example.
//
// [validators]: https://holos.run/docs/v1alpha5/tutorial/validators/
type Validator struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Command\""`
// Inputs represents the files to validate. Usually the final Artifact.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Command represents a validation command. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Command represents a command vetting one or more artifacts. Holos appends
// fully qualified input file paths to the end of the args list, then executes
// the command. Inputs are written into a temporary directory prior to
// executing the command and removed afterwards.
type Command struct {
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
}
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `app.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
// collection and integrates the components together into a holistic platform.
// Holos iterates over the [Component] collection producing a [BuildPlan] for
// each, which holos then executes to render manifests.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components" yaml:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
// from a path containing parameterized CUE configuration.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path" yaml:"path"`
// Instances represents additional cue instance paths to unify with Path.
// Useful to unify data files into a component BuildPlan. Added in holos
// 0.101.7.
Instances []Instance `json:"instances,omitempty" yaml:"instances,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `app.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Instance represents a data instance to unify with the configuration.
//
// Useful to unify json and yaml files with cue configuration files for
// integration with other tools. For example, executing holos render platform
// from a pull request workflow after [Kargo] executes the [yaml update] and
// [git wait for pr] promotion steps.
//
// [Kargo]: https://docs.kargo.io/
// [yaml update]: https://docs.kargo.io/references/promotion-steps#yaml-update
// [git wait for pr]: https://docs.kargo.io/references/promotion-steps#git-wait-for-pr
type Instance struct {
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"ExtractYAML\""`
// Ignored unless kind is ExtractYAML.
ExtractYAML ExtractYAML `json:"extractYAML,omitempty" yaml:"extractYAML,omitempty"`
}
// ExtractYAML represents a cue data instance encoded as yaml or json. If Path
// refers to a directory all files in the directory are extracted
// non-recursively. Otherwise, path must refer to a file.
type ExtractYAML struct {
Path string `json:"path" yaml:"path"`
}

View File

@@ -1,56 +0,0 @@
package v1alpha1
import (
"errors"
"fmt"
"strings"
)
// BuildPlan is the primary interface between CUE and the Holos cli.
type BuildPlan struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
Spec BuildPlanSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
}
type BuildPlanSpec struct {
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
Components BuildPlanComponents `json:"components,omitempty" yaml:"components,omitempty"`
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
type BuildPlanComponents struct {
HelmChartList []HelmChart `json:"helmChartList,omitempty" yaml:"helmChartList,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty" yaml:"kubernetesObjectsList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty" yaml:"kustomizeBuildList,omitempty"`
Resources map[string]KubernetesObjects `json:"resources,omitempty" yaml:"resources,omitempty"`
}
func (bp *BuildPlan) Validate() error {
errs := make([]string, 0, 2)
if bp.Kind != BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", BuildPlanKind, bp.Kind))
}
if bp.APIVersion != APIVersion {
errs = append(errs, fmt.Sprintf("apiVersion invalid: want: %s have: %s", APIVersion, bp.APIVersion))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (bp *BuildPlan) ResultCapacity() (count int) {
if bp == nil {
return 0
}
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}

View File

@@ -1,30 +0,0 @@
package v1alpha1
// HolosComponent defines the fields common to all holos component kinds including the Render Result.
type HolosComponent struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
// APIObjectMap holds the marshalled representation of api objects. Think of
// these as resources overlaid at the back of the render pipeline.
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty" yaml:"apiObjectMap,omitempty"`
// Kustomization holds the marshalled representation of the flux kustomization
// which reconciles resources in git with the api server.
Kustomization `json:",inline" yaml:",inline"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:",inline" yaml:",inline"`
// Skip causes holos to take no action regarding the component.
Skip bool
}
func (hc *HolosComponent) NewResult() *Result {
return &Result{HolosComponent: *hc}
}
func (hc *HolosComponent) GetAPIVersion() string {
return hc.APIVersion
}
func (hc *HolosComponent) GetKind() string {
return hc.Kind
}

View File

@@ -1,11 +0,0 @@
package v1alpha1
const (
APIVersion = "holos.run/v1alpha1"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,2 +0,0 @@
// Package v1alpha1 defines the api boundary between CUE and Holos.
package v1alpha1

View File

@@ -1,13 +0,0 @@
package v1alpha1
import object "github.com/holos-run/holos/service/gen/holos/object/v1alpha1"
// Form represents a collection of Formly json powered form.
type Form struct {
TypeMeta `json:",inline" yaml:",inline"`
Spec FormSpec `json:"spec" yaml:"spec"`
}
type FormSpec struct {
Form object.Form `json:"form" yaml:"form"`
}

View File

@@ -1,184 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// A HelmChart represents a helm command to provide chart values in order to render kubernetes api objects.
type HelmChart struct {
HolosComponent `json:",inline" yaml:",inline"`
// Namespace is the namespace to install into. TODO: Use metadata.namespace instead.
Namespace string `json:"namespace"`
Chart Chart `json:"chart"`
ValuesContent string `json:"valuesContent"`
EnableHooks bool `json:"enableHooks"`
}
type Chart struct {
Name string `json:"name"`
Version string `json:"version"`
Release string `json:"release"`
Repository Repository `json:"repository,omitempty"`
}
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
func (hc *HelmChart) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: hc.HolosComponent}
if err := hc.helm(ctx, &result, path); err != nil {
return nil, err
}
result.addObjectMap(ctx, hc.APIObjectMap)
if err := result.kustomize(ctx); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not kustomize: %w", err))
}
return &result, nil
}
// runHelm provides the values produced by CUE to helm template and returns
// the rendered kubernetes api objects in the result.
func (hc *HelmChart) helm(ctx context.Context, r *Result, path holos.InstancePath) error {
log := logger.FromContext(ctx).With("chart", hc.Chart.Name)
if hc.Chart.Name == "" {
log.WarnContext(ctx, "skipping helm: no chart name specified, use a different component type")
return nil
}
cachedChartPath := filepath.Join(string(path), ChartDir, filepath.Base(hc.Chart.Name))
if isNotExist(cachedChartPath) {
// Add repositories
repo := hc.Chart.Repository
if repo.URL != "" {
out, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo add: %w", err))
}
// Update repository
out, err = util.RunCmd(ctx, "helm", "repo", "update", repo.Name)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo update: %w", err))
}
} else {
log.DebugContext(ctx, "no chart repository url proceeding assuming oci chart")
}
// Cache the chart
if err := cacheChart(ctx, path, ChartDir, hc.Chart); err != nil {
return fmt.Errorf("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos")
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, tempDir)
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, []byte(hc.ValuesContent), 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "helm: wrote values", "path", valuesPath, "bytes", len(hc.ValuesContent))
// Run charts
chart := hc.Chart
args := []string{"template"}
if !hc.EnableHooks {
args = append(args, "--no-hooks")
}
namespace := hc.Namespace
args = append(args, "--include-crds", "--values", valuesPath, "--namespace", namespace, "--kubeconfig", "/dev/null", "--version", chart.Version, chart.Release, cachedChartPath)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Wrap(fmt.Errorf("could not run helm template: %w", err))
}
r.accumulatedOutput = helmOut.Stdout.String()
return nil
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// It is assumed that the only method responsible for writing to chartDir is
// cacheChart itself.
//
// This relies on the atomicity of moving temporary directories into place on
// the same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is an
// expected scenario when this function is called concurrently.
func cacheChart(ctx context.Context, path holos.InstancePath, chartDir string, chart Chart) error {
log := logger.FromContext(ctx)
cacheTemp, err := os.MkdirTemp(string(path), chartDir)
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, cacheTemp)
chartName := chart.Name
if chart.Repository.Name != "" {
chartName = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, chartName)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
cachePath := filepath.Join(string(path), chartDir)
if err := os.MkdirAll(cachePath, 0777); err != nil {
return errors.Wrap(fmt.Errorf("could not mkdir: %w", err))
}
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
for _, item := range items {
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cachePath, item.Name())
log.DebugContext(ctx, "rename", "src", src, "dst", dst)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
}
}
log.InfoContext(ctx, "cached", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
return nil
}
func isNotExist(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}

View File

@@ -1,21 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents CUE output which directly provides Kubernetes api objects to holos.
type KubernetesObjects struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces kubernetes api objects from the APIObjectMap
func (o *KubernetesObjects) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: o.HolosComponent}
result.addObjectMap(ctx, o.APIObjectMap)
return &result, nil
}

View File

@@ -1,7 +0,0 @@
package v1alpha1
// Kustomization holds the rendered flux kustomization api object content for git ops.
type Kustomization struct {
// KsContent is the yaml representation of the flux kustomization for gitops.
KsContent string `json:"ksContent,omitempty" yaml:"ksContent,omitempty"`
}

View File

@@ -1,47 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
const KustomizeBuildKind = "KustomizeBuild"
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process raw yaml file resources in a holos component directory.
// 2. Post process a HelmChart to inject istio, add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty" yaml:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty" yaml:"resourcesFile,omitempty"`
}
// KustomizeBuild renders plain yaml files in the holos component directory using kubectl kustomize build.
type KustomizeBuild struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces a Result by executing kubectl kustomize on the holos
// component path. Useful for processing raw yaml files.
func (kb *KustomizeBuild) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
log := logger.FromContext(ctx)
result := Result{HolosComponent: kb.HolosComponent}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", string(path))
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return nil, errors.Wrap(err)
}
// Replace the accumulated output
result.accumulatedOutput = kOut.Stdout.String()
// Add CUE based api objects.
result.addObjectMap(ctx, kb.APIObjectMap)
return &result, nil
}

View File

@@ -1,14 +0,0 @@
package v1alpha1
// Label is an arbitrary unique identifier. Defined as a type for clarity and type checking.
type Label string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
type Kind string
// APIObjectMap is the shape of marshalled api objects returned from cue to the
// holos cli. A map is used to improve the clarity of error messages from cue.
type APIObjectMap map[Kind]map[Label]string
// FileContentMap is a map of file names to file contents.
type FileContentMap map[string]string

View File

@@ -1,15 +0,0 @@
package v1alpha1
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are by holos objects distinct
// from kubernetes api objects.
type ObjectMeta struct {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
Name string `json:"name,omitempty" yaml:"name,omitempty"`
// Namespace confines a holos component to a single namespace via kustomize if set.
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Labels are not used but are copied from api machinery ObjectMeta for completeness.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations are not used but are copied from api machinery ObjectMeta for completeness.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,32 +0,0 @@
package v1alpha1
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
TypeMeta `json:",inline" yaml:",inline"`
Metadata ObjectMeta `json:"metadata" yaml:"metadata"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform build plan specification.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// holos.platform.v1alpha1.PlatformService.GetPlatform method and provides to
// CUE using a tag.
Model structpb.Struct `json:"model" yaml:"model"`
Components []PlatformSpecComponent `json:"components" yaml:"components"`
}
// PlatformSpecComponent represents a component to build or render with flags to
// pass, for example the cluster name.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path" yaml:"path"`
// Cluster is the cluster name to use when building the component.
Cluster string `json:"cluster" yaml:"cluster"`
}

View File

@@ -1,22 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
type Renderer interface {
GetKind() string
Render(ctx context.Context, path holos.InstancePath) (*Result, error)
}
// Render produces a Result representing the kubernetes api objects to
// configure. Each of the various holos component types, e.g. Helm, Kustomize,
// et al, should implement the Renderer interface. This process is best
// conceptualized as a data pipeline, for example a component may render a
// result by first calling helm template, then passing the result through
// kustomize, then mixing in overlay api objects.
func Render(ctx context.Context, r Renderer, path holos.InstancePath) (*Result, error) {
return r.Render(ctx, path)
}

View File

@@ -1,165 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"slices"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// Result is the build result for display or writing. Holos components Render the Result as a data pipeline.
type Result struct {
HolosComponent
// accumulatedOutput accumulates rendered api objects.
accumulatedOutput string
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
// Continue returns true if Skip is true indicating the result is to be skipped over.
func (r *Result) Continue() bool {
if r == nil {
return false
}
return r.Skip
}
func (r *Result) Name() string {
return r.Metadata.Name
}
func (r *Result) Filename(writeTo string, cluster string) string {
name := r.Metadata.Name
return filepath.Join(writeTo, "clusters", cluster, "components", name, name+".gen.yaml")
}
func (r *Result) KustomizationFilename(writeTo string, cluster string) string {
return filepath.Join(writeTo, "clusters", cluster, "holos", "components", r.Metadata.Name+"-kustomization.gen.yaml")
}
// AccumulatedOutput returns the accumulated rendered output.
func (r *Result) AccumulatedOutput() string {
return r.accumulatedOutput
}
// addObjectMap renders the provided APIObjectMap into the accumulated output.
func (r *Result) addObjectMap(ctx context.Context, objectMap APIObjectMap) {
log := logger.FromContext(ctx)
b := []byte(r.AccumulatedOutput())
kinds := make([]Kind, 0, len(objectMap))
// Sort the keys
for kind := range objectMap {
kinds = append(kinds, kind)
}
slices.Sort(kinds)
for _, kind := range kinds {
v := objectMap[kind]
// Sort the keys
names := make([]Label, 0, len(v))
for name := range v {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
yamlString := v[name]
log.Debug(fmt.Sprintf("%s/%s", kind, name), "kind", kind, "name", name)
b = util.EnsureNewline(b)
header := fmt.Sprintf("---\n# Source: CUE apiObjects.%s.%s\n", kind, name)
b = append(b, []byte(header+yamlString)...)
b = util.EnsureNewline(b)
}
}
r.accumulatedOutput = string(b)
}
// kustomize replaces the accumulated output with the output of kustomize build
func (r *Result) kustomize(ctx context.Context) error {
log := logger.FromContext(ctx)
if r.ResourcesFile == "" {
log.DebugContext(ctx, "skipping kustomize: no resourcesFile")
return nil
}
if len(r.KustomizeFiles) < 1 {
log.DebugContext(ctx, "skipping kustomize: no kustomizeFiles")
return nil
}
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
// Write the main api object resources file for kustomize.
target := filepath.Join(tempDir, r.ResourcesFile)
b := []byte(r.AccumulatedOutput())
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write resources: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
// Write the kustomization tree, kustomization.yaml must be in this map for kustomize to work.
for file, content := range r.KustomizeFiles {
target := filepath.Join(tempDir, file)
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return errors.Wrap(err)
}
b := []byte(content)
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return errors.Wrap(err)
}
// Replace the accumulated output
r.accumulatedOutput = kOut.Stdout.String()
return nil
}
func (r *Result) WriteDeployFiles(ctx context.Context, path string) error {
log := logger.FromContext(ctx)
if len(r.DeployFiles) == 0 {
return nil
}
for k, content := range r.DeployFiles {
path := filepath.Join(path, k)
if err := r.Save(ctx, path, content); err != nil {
return errors.Wrap(err)
}
log.InfoContext(ctx, "wrote deploy file", "path", path, "bytes", len(content))
}
return nil
}
// Save writes the content to the filesystem for git ops.
func (r *Result) Save(ctx context.Context, path string, content string) error {
log := logger.FromContext(ctx)
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, os.FileMode(0775)); err != nil {
log.WarnContext(ctx, "could not mkdir", "path", dir, "err", err)
return errors.Wrap(err)
}
// Write the file content
if err := os.WriteFile(path, []byte(content), os.FileMode(0644)); err != nil {
log.WarnContext(ctx, "could not write", "path", path, "err", err)
return errors.Wrap(err)
}
log.DebugContext(ctx, "out: wrote "+path, "action", "write", "path", path, "status", "ok")
return nil
}

View File

@@ -1,20 +0,0 @@
package v1alpha1
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
func (tm *TypeMeta) GetKind() string {
return tm.Kind
}
func (tm *TypeMeta) GetAPIVersion() string {
return tm.APIVersion
}
// Discriminator is an interface to discriminate the kind api object.
type Discriminator interface {
GetKind() string
GetAPIVersion() string
}

63
cmd/cmd.go Normal file
View File

@@ -0,0 +1,63 @@
package cmd
import (
"context"
"fmt"
"log/slog"
"os"
"runtime/pprof"
"runtime/trace"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/internal/holos"
)
// MakeMain makes a main function for the cli or tests.
func MakeMain(options ...holos.Option) func() int {
return func() (exitCode int) {
cfg := holos.New(options...)
slog.SetDefault(cfg.Logger())
ctx := context.Background()
if format := os.Getenv("HOLOS_CPU_PROFILE"); format != "" {
f, _ := os.Create(fmt.Sprintf(format, os.Getppid(), os.Getpid()))
err := pprof.StartCPUProfile(f)
defer func() {
pprof.StopCPUProfile()
f.Close()
}()
if err != nil {
return cli.HandleError(ctx, err, cfg)
}
}
defer memProfile(ctx, cfg)
if format := os.Getenv("HOLOS_TRACE"); format != "" {
f, _ := os.Create(fmt.Sprintf(format, os.Getppid(), os.Getpid()))
err := trace.Start(f)
defer func() {
trace.Stop()
f.Close()
}()
if err != nil {
return cli.HandleError(ctx, err, cfg)
}
}
feature := &holos.EnvFlagger{}
if err := cli.New(cfg, feature).ExecuteContext(ctx); err != nil {
return cli.HandleError(ctx, err, cfg)
}
return 0
}
}
func memProfile(ctx context.Context, cfg *holos.Config) {
if format := os.Getenv("HOLOS_MEM_PROFILE"); format != "" {
f, _ := os.Create(fmt.Sprintf(format, os.Getppid(), os.Getpid()))
defer f.Close()
if err := pprof.WriteHeapProfile(f); err != nil {
_ = cli.HandleError(ctx, err, cfg)
}
}
}

View File

@@ -3,9 +3,9 @@ package main
import (
"os"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/cmd"
)
func main() {
os.Exit(cli.MakeMain()())
os.Exit(cmd.MakeMain()())
}

View File

@@ -2,20 +2,50 @@ package main
import (
"os"
"path/filepath"
"testing"
"github.com/holos-run/holos/internal/cli"
cue "cuelang.org/go/cmd/cue/cmd"
"github.com/holos-run/holos/cmd"
"github.com/rogpeppe/go-internal/testscript"
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cli.MakeMain(),
"holos": cmd.MakeMain(),
"cue": cue.Main,
}))
}
func TestGetSecrets(t *testing.T) {
testscript.Run(t, testscript.Params{
Dir: "testdata",
})
func TestGuides_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "guides")))
}
func TestSchemas_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "schemas")))
}
func TestIssues_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "issues")))
}
func TestCLI(t *testing.T) {
testscript.Run(t, params("cli"))
}
func params(dir string) testscript.Params {
return testscript.Params{
Dir: filepath.Join("tests", dir),
RequireExplicitExec: true,
RequireUniqueNames: os.Getenv("HOLOS_WORKDIR_ROOT") == "",
WorkdirRoot: os.Getenv("HOLOS_WORKDIR_ROOT"),
UpdateScripts: os.Getenv("HOLOS_UPDATE_SCRIPTS") != "",
Setup: func(env *testscript.Env) error {
// Just like cmd/cue/cmd.TestScript, set up separate cache and config dirs per test.
env.Setenv("CUE_CACHE_DIR", filepath.Join(env.WorkDir, "tmp/cachedir"))
configDir := filepath.Join(env.WorkDir, "tmp/configdir")
env.Setenv("CUE_CONFIG_DIR", configDir)
return nil
},
}
}

View File

@@ -1,34 +0,0 @@
# Want support for intermediary constraints
exec holos build ./foo/... --log-level debug
stdout '^bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- foo/constraints.cue --
package holos
metadata: name: "jeff"
-- foo/bar/bar.cue --
package holos
spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
apiObjectMap: foo: bar: "bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b"
}
]
-- schema.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#KubernetesObjects: {
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
apiObjectMap: {...}
}
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"

View File

@@ -1,20 +0,0 @@
# Want cue errors to show files and lines
! exec holos build .
stderr 'apiObjectMap.foo.bar: cannot convert incomplete value'
stderr '/component.cue:\d+:\d+$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: foo: bar: _baz}]
_baz: string

View File

@@ -1,61 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,62 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
stderr 'skipping helm: no chart name specified'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,25 +0,0 @@
# Want api object kind and name in errors
! exec holos build .
stderr 'apiObjects.secretstore.default.foo: field not allowed'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
metadata: name: string
}
apiObjects: {
secretstore: {
default: #SecretStore & { foo: "not allowed" }
}
}

View File

@@ -1,289 +0,0 @@
# Want helm errors to show up
! exec holos build .
stderr 'Error: execution error at \(zitadel/templates/secret_zitadel-masterkey.yaml:2:4\): Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- zitadel.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [_HelmChart]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
_HelmChart: {
apiVersion: "holos.run/v1alpha1"
kind: "HelmChart"
metadata: name: "zitadel"
namespace: "zitadel"
chart: {
name: "zitadel"
version: "7.9.0"
release: name
repository: {
name: "zitadel"
url: "https://charts.zitadel.com"
}
}
}
-- vendor/zitadel/templates/secret_zitadel-masterkey.yaml --
{{- if (or (and .Values.zitadel.masterkey .Values.zitadel.masterkeySecretName) (and (not .Values.zitadel.masterkey) (not .Values.zitadel.masterkeySecretName)) ) }}
{{- fail "Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName" }}
{{- end }}
{{- if .Values.zitadel.masterkey -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: zitadel-masterkey
{{- with .Values.zitadel.masterkeyAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "zitadel.labels" . | nindent 4 }}
stringData:
masterkey: {{ .Values.zitadel.masterkey }}
{{- end -}}
-- vendor/zitadel/Chart.yaml --
apiVersion: v2
appVersion: v2.46.0
description: A Helm chart for ZITADEL
icon: https://zitadel.com/zitadel-logo-dark.svg
kubeVersion: '>= 1.21.0-0'
maintainers:
- email: support@zitadel.com
name: zitadel
url: https://zitadel.com
name: zitadel
type: application
version: 7.9.0
-- vendor/zitadel/values.yaml --
# Default values for zitadel.
zitadel:
# The ZITADEL config under configmapConfig is written to a Kubernetes ConfigMap
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
configmapConfig:
ExternalSecure: true
Machine:
Identification:
Hostname:
Enabled: true
Webhook:
Enabled: false
# The ZITADEL config under secretConfig is written to a Kubernetes Secret
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
secretConfig:
# Annotations set on secretConfig secret
secretConfigAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# Reference the name of a secret that contains ZITADEL configuration.
configSecretName:
# The key under which the ZITADEL configuration is located in the secret.
configSecretKey: config-yaml
# ZITADEL uses the masterkey for symmetric encryption.
# You can generate it for example with tr -dc A-Za-z0-9 </dev/urandom | head -c 32
masterkey: ""
# Reference the name of the secret that contains the masterkey. The key should be named "masterkey".
# Note: Either zitadel.masterkey or zitadel.masterkeySecretName must be set
masterkeySecretName: ""
# Annotations set on masterkey secret
masterkeyAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The CA Certificate needed for establishing secure database connections
dbSslCaCrt: ""
# The Secret containing the CA certificate at key ca.crt needed for establishing secure database connections
dbSslCaCrtSecret: ""
# The db admins secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslAdminCrtSecret: ""
# The db users secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslUserCrtSecret: ""
# Generate a self-signed certificate using an init container
# This will also mount the generated files to /etc/tls/ so that you can reference them in the pod.
# E.G. KeyPath: /etc/tls/tls.key CertPath: /etc/tls/tls.crt
# By default, the SAN DNS names include, localhost, the POD IP address and the POD name. You may include one more by using additionalDnsName like "my.zitadel.fqdn".
selfSignedCert:
enabled: false
additionalDnsName:
replicaCount: 3
image:
repository: ghcr.io/zitadel/zitadel
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
chownImage:
repository: alpine
pullPolicy: IfNotPresent
tag: "3.19"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
# Annotations to add to the deployment
annotations: {}
# Annotations to add to the configMap
configMap:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podAdditionalLabels: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
securityContext: {}
# Additional environment variables
env:
[]
# - name: ZITADEL_DATABASE_POSTGRES_HOST
# valueFrom:
# secretKeyRef:
# name: postgres-pguser-postgres
# key: host
service:
type: ClusterIP
# If service type is "ClusterIP", this can optionally be set to a fixed IP address.
clusterIP: ""
port: 8080
protocol: http2
annotations: {}
scheme: HTTP
ingress:
enabled: false
className: ""
annotations: {}
hosts:
- host: localhost
paths:
- path: /
pathType: Prefix
tls: []
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
topologySpreadConstraints: []
initJob:
# Once ZITADEL is installed, the initJob can be disabled.
enabled: true
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "1"
resources: {}
backoffLimit: 5
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
# Available init commands :
# "": initialize ZITADEL instance (without skip anything)
# database: initialize only the database
# grant: set ALL grant to user
# user: initialize only the database user
# zitadel: initialize ZITADEL internals (skip "create user" and "create database")
command: ""
setupJob:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "2"
resources: {}
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
additionalArgs:
- "--init-projections=true"
machinekeyWriter:
image:
repository: bitnami/kubectl
tag: ""
resources: {}
readinessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
livenessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
startupProbe:
enabled: true
periodSeconds: 1
failureThreshold: 30
metrics:
enabled: false
serviceMonitor:
# If true, the chart creates a ServiceMonitor that is compatible with Prometheus Operator
# https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#monitoring.coreos.com/v1.ServiceMonitor.
# The Prometheus community Helm chart installs this operator
# https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#kube-prometheus-stack
enabled: false
honorLabels: false
honorTimestamps: true
pdb:
enabled: false
# these values are used for the PDB and are mutally exclusive
minAvailable: 1
# maxUnavailable: 1
annotations: {}

View File

@@ -1,39 +0,0 @@
# Kustomize is a supported holos component kind
exec holos render component --cluster-name=mycluster . --log-level=debug
# Want generated output
cmp want.yaml deploy/clusters/mycluster/components/kstest/kstest.gen.yaml
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KustomizeBuildList: [{metadata: name: "kstest"}]
-- kustomization.yaml --
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mynamespace
resources:
- serviceaccount.yaml
-- serviceaccount.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
-- want.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
namespace: mynamespace

View File

@@ -1,17 +0,0 @@
# https://github.com/holos-run/holos/issues/72
# Want holos to fail on unknown fields to catch typos and aid refactors
! exec holos build .
stderr 'unknown field \\"TypoKubernetesObjectsList\\"'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: TypoKubernetesObjectsList: []

View File

@@ -0,0 +1,12 @@
# https://github.com/holos-run/holos/issues/358
# holos cue vet should fail verifications with exit code 1
! exec holos cue vet ./policy --path strings.ToLower(kind) ./data/secret.yaml
# holos cue vet should report validation errors to stderr
stderr 'Forbidden. Use an ExternalSecret instead.'
-- data/secret.yaml --
kind: Secret
-- policy/validators.cue --
package policy
secret: kind: "Forbidden. Use an ExternalSecret instead."

View File

@@ -0,0 +1,2 @@
# https://github.com/holos-run/holos/issues/334
exec holos

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,422 @@
# https://github.com/holos-run/holos/issues/331
# ensure holos show components --labels selects correctly.
# ensure BuildPlan includes labels and annotations from the platform component.
# ensure holos render platform injects the holos_component_labels and
# holos_component_annotations tags.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cmp stdout want/platform.yaml
# all buildplans are selected by default
exec holos show buildplans
cmp stdout want/all-buildplans.yaml
# one = works in the selector
exec holos show buildplans --selector app.holos.run/name=empty1-label
cmp stdout want/buildplans.1.yaml
# double == works in the selector
exec holos show buildplans --selector app.holos.run/name==empty2-label
cmp stdout want/buildplans.2.yaml
# not equal != negates the selection
exec holos show buildplans --selector app.holos.run/name!=empty3-label
cmp stdout want/buildplans.3.yaml
exec holos show buildplans --selector app.holos.run/name!=something-else
cmp stdout want/buildplans.4.yaml
-- platform/empty.cue --
package holos
Platform: Components: {
empty1: _
empty2: _
empty3: _
empty4: _
}
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": "\(name)-label"
annotations: "app.holos.run/description": "\(name)-annotation empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/platform.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components:
- annotations:
app.holos.run/description: empty1-annotation empty test case
labels:
app.holos.run/name: empty1-label
name: empty1
path: components/empty
- annotations:
app.holos.run/description: empty2-annotation empty test case
labels:
app.holos.run/name: empty2-label
name: empty2
path: components/empty
- annotations:
app.holos.run/description: empty3-annotation empty test case
labels:
app.holos.run/name: empty3-label
name: empty3
path: components/empty
- annotations:
app.holos.run/description: empty4-annotation empty test case
labels:
app.holos.run/name: empty4-label
name: empty4
path: components/empty
-- want/empty.yaml --
-- want/all-buildplans.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.1.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.2.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.3.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
-- want/buildplans.4.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,64 @@
# https://github.com/holos-run/holos/issues/348
# when the optional kustomize patch name field is omitted
exec holos init platform v1alpha5 --force
# want a buildplan shown
exec holos show buildplans
cmp stdout buildplan.yaml
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
-- buildplan.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: example
spec:
artifacts:
- artifact: components/example/example.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/example/example.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
patches:
- patch: |
- op: add
path: /metadata/annotations/example
value: example-value
target:
kind: CustomResourceDefinition
name: ""
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,50 @@
# https://github.com/holos-run/holos/issues/366
# Build tags conditionally include CUE files.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cmp stdout want/empty.yaml
exec holos show platform -t foo
cmp stdout want/foo.yaml
-- platform/empty.cue --
@if(foo)
package holos
Platform: Components: foo: _
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": NAME
annotations: "app.holos.run/description": "\(NAME) empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/empty.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components: []
-- want/foo.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components:
- annotations:
app.holos.run/description: foo empty test case
labels:
app.holos.run/name: foo
name: foo
path: components/empty

View File

@@ -0,0 +1,144 @@
# https://github.com/holos-run/holos/issues/330
exec holos init platform v1alpha5 --force
exec helm template ./components/capabilities/vendor/0.1.0/capabilities
cmp stdout want/helm-template.yaml
exec holos render platform ./platform
# When no capabilities are specified
cmp deploy/components/capabilities/capabilities.gen.yaml want/when-no-capabilities-specified.yaml
# With APIVersions specified
cmp deploy/components/specified/specified.gen.yaml want/with-capabilities-specified.yaml
# With KubeVersion specified
cmp deploy/components/kubeversion1/kubeversion1.gen.yaml want/with-kubeversion-specified.yaml
# With both APIVersions and KubeVersion specified
cmp deploy/components/kubeversion2/kubeversion2.gen.yaml want/with-both-specified.yaml
-- want/with-both-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-kubeversion-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/when-no-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- platform/capabilities.cue --
package holos
import "encoding/json"
Platform: Components: capabilities: {
name: "capabilities"
path: "components/capabilities"
}
Platform: Components: specified: {
name: "specified"
path: "components/capabilities"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
Platform: Components: kubeversion1: {
name: "kubeversion1"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
}
Platform: Components: kubeversion2: {
name: "kubeversion2"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
-- components/capabilities/capabilities.cue --
package holos
import "encoding/json"
holos: Component.BuildPlan
Component: #Helm & {
Name: string @tag(holos_component_name, type=string)
Chart: name: "capabilities"
Chart: version: "0.1.0"
_APIVersions: string | *"[]" @tag(apiVersions, type=string)
APIVersions: json.Unmarshal(_APIVersions)
KubeVersion: string | *"v1.31.0" @tag(kubeVersion, type=string)
}
-- components/capabilities/vendor/0.1.0/capabilities/Chart.yaml --
apiVersion: v2
name: capabilities
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: "1.16.0"
-- components/capabilities/vendor/0.1.0/capabilities/templates/service.yaml --
apiVersion: v1
kind: Service
metadata:
{{- if .Capabilities.APIVersions.Has "foo/v1" }}
name: has-foo-v1
{{- else }}
name: has-foo-v1beta1
{{- end }}
annotations:
kubeVersion: {{ .Capabilities.KubeVersion }}
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
-- want/helm-template.yaml --
---
# Source: capabilities/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: has-foo-v1beta1
annotations:
kubeVersion: v1.31.0
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http

View File

@@ -0,0 +1,45 @@
# author.#Kubernetes
# Start in an empty directory.
cd $WORK
# Generate the directory structure we're going to work in.
exec holos init platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform
stderr -count=1 '^rendered platform'
# When author.#Kubernetes is empty
exec holos cue export --expression holos --out=yaml ./components/empty
cmp stdout want.txt
-- components/empty/empty.cue --
package holos
Kubernetes: #Kubernetes & {}
holos: Kubernetes.BuildPlan
-- want.txt --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
validators: []
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
resources:
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1

View File

@@ -0,0 +1,37 @@
# https://github.com/holos-run/holos/issues/357
exec holos init platform v1alpha5 --force
! exec holos render platform
stderr 'secret.kind: conflicting values "Forbidden. Use an ExternalSecret instead." and "Secret"'
-- validators.cue --
package holos
import "github.com/holos-run/holos/api/author/v1alpha5:author"
#ComponentConfig: author.#ComponentConfig & {
Validators: cue: {
kind: "Command"
command: args: ["holos", "cue", "vet", "./policy", "--path", "strings.ToLower(kind)"]
}
}
-- policy/validations.cue --
package validations
secret: kind: "Forbidden. Use an ExternalSecret instead."
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/secret.cue --
package holos
holos: Component.BuildPlan
Component: #Kubernetes & {
Resources: Secret: test: {
metadata: name: "test"
}
}

View File

@@ -2291,9 +2291,10 @@ First for prometheus.
<Tabs groupId="5062EB93-F5AA-4038-9CF8-67A5ECA085FD">
<TabItem value="command" label="Command">
```bash
cue import -p holos -o- -l '_Helm: Values:' \
projects/platform/components/prometheus/vendor/25.27.0/prometheus/values.yaml \
> projects/platform/components/prometheus/values.cue
cue import --package holos \
--path '_Helm: Values:' \
--outfile projects/platform/components/prometheus/values.cue \
projects/platform/components/prometheus/vendor/25.27.0/prometheus/values.yaml
```
</TabItem>
<TabItem value="output" label="values.cue">
@@ -3647,9 +3648,10 @@ Then for blackbox.
<Tabs groupId="843D706B-5BE0-46FE-978F-EA17BC1AD932">
<TabItem value="command" label="Command">
```bash
cue import -p holos -o- -l '_Helm: Values:' \
projects/platform/components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml \
> projects/platform/components/blackbox/values.cue
cue import --package holos \
--path '_Helm: Values:' \
--outfile projects/platform/components/blackbox/values.cue
projects/platform/components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
```
</TabItem>
<TabItem value="output" label="values.cue">
@@ -4310,11 +4312,10 @@ git diff
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:6115
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target
```
</TabItem>
</Tabs>
@@ -4578,9 +4579,10 @@ _Kustomize.BuildPlan
// https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/README.md
_Kustomize: #Kustomize & {
KustomizeConfig: Resources: "github.com/mccutchen/go-httpbin/kustomize": _
KustomizeConfig: Files: "resources.yaml": _
KustomizeConfig: Kustomization: {
commonLabels: "app.kubernetes.io/name": "httpbin"
images: [{name: "mccutchen/go-httpbin"}]
_patches: probe: {
target: kind: "Service"
target: name: "httpbin"
@@ -4593,6 +4595,55 @@ _Kustomize: #Kustomize & {
patches: [for x in _patches {x}]
}
}
```
</TabItem>
<TabItem value="projects/platform/components/httpbin/resources.yaml" label="resources.yaml">
Add a plain `resources.yaml` file containing resources for kustomize to process.
:::important
Holos knows this file is part of the BuildPlan from the `KustomizeConfig:
Files: "resources.yaml": _` line in the Component.
:::
```bash
mkdir -p projects/platform/components/httpbin
touch projects/platform/components/httpbin/resources.yaml
```
```yaml showLineNumbers
# https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/resources.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
template:
spec:
containers:
- name: httpbin
image: mccutchen/go-httpbin
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /status/200
port: http
readinessProbe:
httpGet:
path: /status/200
port: http
resources: {}
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
appProtocol: http
```
</TabItem>
</Tabs>

View File

Before

Width:  |  Height:  |  Size: 690 KiB

After

Width:  |  Height:  |  Size: 690 KiB

View File

Before

Width:  |  Height:  |  Size: 997 KiB

After

Width:  |  Height:  |  Size: 997 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 287 KiB

After

Width:  |  Height:  |  Size: 287 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1009 KiB

After

Width:  |  Height:  |  Size: 1009 KiB

View File

Before

Width:  |  Height:  |  Size: 617 KiB

After

Width:  |  Height:  |  Size: 617 KiB

View File

Before

Width:  |  Height:  |  Size: 706 KiB

After

Width:  |  Height:  |  Size: 706 KiB

View File

Before

Width:  |  Height:  |  Size: 794 KiB

After

Width:  |  Height:  |  Size: 794 KiB

View File

@@ -326,35 +326,34 @@ like this.
```mermaid
---
title: Figure 1 - Render Pipeline
title: Figure 1 - v1alpha4 Rendered Manifest Pipeline
---
graph LR
PS[<a href="/docs/api/author/v1alpha3/#Platform">Platform</a>]
HC[<a href="/docs/api/author/v1alpha3/#ComponentFields">Components</a>]
BP[<a href="/docs/api/core/v1alpha3#BuildPlan">BuildPlan</a>]
Platform[<a href="/docs/api/author/v1alpha4/#Platform">Platform</a>]
Component[<a href="/docs/api/author/v1alpha4/#ComponentConfig">Components</a>]
H[<a href="/docs/api/author/v1alpha3/#Helm">Helm</a>]
K[<a href="/docs/api/author/v1alpha3/#Kustomize">Kustomize</a>]
O[<a href="/docs/api/author/v1alpha3/#Kubernetes">Kubernetes</a>]
Helm[<a href="/docs/api/author/v1alpha4/#Helm">Helm</a>]
Kustomize[<a href="/docs/api/author/v1alpha4/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/api/author/v1alpha4/#Kubernetes">Kubernetes</a>]
P[<a href="/docs/api/core/v1alpha3#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
BuildPlan[<a href="/docs/api/core/v1alpha4/#buildplan">BuildPlan</a>]
C[Kube API Server]
ResourcesArtifact[<a href="/docs/api/core/v1alpha4/#artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/api/core/v1alpha4/#artifact">GitOps<br/>Artifact</a>]
PS --> HC --> BP
BP --> H --> P
BP --> K --> P
BP --> O --> P
Generators[<a href="/docs/api/core/v1alpha4/#generators">Generators</a>]
Transformers[<a href="/docs/api/core/v1alpha4/#transformer">Transformers</a>]
Files[Manifest<br/>Files]
P --> Y --> FS
P --> G --> FS
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
FS --> ArgoCD --> C
FS --> Flux --> C
FS --> kubectl --> C
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Files
```
### Why do we render the platform? {#why-render-the-platform}

View File

Before

Width:  |  Height:  |  Size: 934 KiB

After

Width:  |  Height:  |  Size: 934 KiB

View File

Before

Width:  |  Height:  |  Size: 703 KiB

After

Width:  |  Height:  |  Size: 703 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 728 KiB

After

Width:  |  Height:  |  Size: 728 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 854 KiB

After

Width:  |  Height:  |  Size: 854 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 624 KiB

After

Width:  |  Height:  |  Size: 624 KiB

View File

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 116 KiB

View File

@@ -1,6 +1,5 @@
---
description: Holos Documentation
slug: /
description: Introduction
---
# Introduction

Some files were not shown because too many files have changed in this diff Show More