Compare commits

...

536 Commits

Author SHA1 Message Date
Jeff McCune
42c4feb01e Merge pull request #461 from holos-run/jeff/node22-cloudflare
website: update to node 22 for cloudflare builds
2026-01-24 20:52:51 -08:00
Jeff McCune
945e819af8 website: move .nvmrc to repository root
Cloudflare Pages looks for .nvmrc at the project root before running
the build command, not in subdirectories.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 20:48:30 -08:00
Jeff McCune
078052c112 website: update to node 22 for cloudflare builds
Add .nvmrc file to specify Node 22 for Cloudflare Pages builds.
Update engines in package.json to require Node >= 20.0 to support
dependabot updates that require at least Node 20.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 17:31:50 -08:00
Jeff McCune
382da159d9 Merge pull request #460 from holos-run/add-kubectl-tool
tools: add kubectl v0.35.0 as versioned tool dependency
2026-01-24 17:26:51 -08:00
Jeff McCune
538400f6c6 tools: add kubectl v0.34.3 as versioned tool dependency
Add kubectl to tools.go to pin the version and allow installation via
`go install k8s.io/kubectl`. This upgrades Kubernetes-related
dependencies from v0.33.3 to v0.34.3.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 17:01:19 -08:00
Jeff McCune
3ba8dfae5d Merge pull request #455 from holos-run/dependabot/go_modules/go_modules-4d109dd656
build(deps): bump the go_modules group across 1 directory with 4 updates
2026-01-21 20:26:24 -08:00
Jeff McCune
beee0ed10e Merge pull request #454 from holos-run/dependabot/npm_and_yarn/doc/website/multi-1c989c8248
build(deps): bump body-parser and express in /doc/website
2026-01-21 20:26:06 -08:00
Jeff McCune
1ab7b3c440 Merge pull request #453 from holos-run/testscript
tools: add testscript to make tools
2026-01-21 20:25:39 -08:00
dependabot[bot]
135966976d build(deps): bump the go_modules group across 1 directory with 4 updates
Bumps the go_modules group with 3 updates in the / directory: [helm.sh/helm/v3](https://github.com/helm/helm), [github.com/cloudflare/circl](https://github.com/cloudflare/circl) and [golang.org/x/crypto](https://github.com/golang/crypto).


Updates `helm.sh/helm/v3` from 3.16.3 to 3.18.5
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.16.3...v3.18.5)

Updates `github.com/cloudflare/circl` from 1.3.7 to 1.6.1
- [Release notes](https://github.com/cloudflare/circl/releases)
- [Commits](https://github.com/cloudflare/circl/compare/v1.3.7...v1.6.1)

Updates `github.com/containerd/containerd` from 1.7.23 to 1.7.27
- [Release notes](https://github.com/containerd/containerd/releases)
- [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md)
- [Commits](https://github.com/containerd/containerd/compare/v1.7.23...v1.7.27)

Updates `golang.org/x/crypto` from 0.43.0 to 0.45.0
- [Commits](https://github.com/golang/crypto/compare/v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.18.5
  dependency-type: direct:production
  dependency-group: go_modules
- dependency-name: github.com/cloudflare/circl
  dependency-version: 1.6.1
  dependency-type: indirect
  dependency-group: go_modules
- dependency-name: github.com/containerd/containerd
  dependency-version: 1.7.27
  dependency-type: indirect
  dependency-group: go_modules
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-22 04:20:25 +00:00
dependabot[bot]
30950953d7 build(deps): bump body-parser and express in /doc/website
Bumps [body-parser](https://github.com/expressjs/body-parser) and [express](https://github.com/expressjs/express). These dependencies needed to be updated together.

Updates `body-parser` from 1.20.2 to 1.20.4
- [Release notes](https://github.com/expressjs/body-parser/releases)
- [Changelog](https://github.com/expressjs/body-parser/blob/master/HISTORY.md)
- [Commits](https://github.com/expressjs/body-parser/compare/1.20.2...1.20.4)

Updates `express` from 4.19.2 to 4.22.1
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/v4.22.1/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.19.2...v4.22.1)

---
updated-dependencies:
- dependency-name: body-parser
  dependency-version: 1.20.4
  dependency-type: indirect
- dependency-name: express
  dependency-version: 4.22.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-22 04:18:06 +00:00
Jeff McCune
8e4b7d9e43 tools: add testscript to make tools
Install github.com/rogpeppe/go-internal/cmd/testscript when running
make tools for script-based testing support.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>
2026-01-21 20:15:19 -08:00
Jeff McCune
f4484ffb21 Merge pull request #451 from holos-run/feat/cue-0.15.1
cue: update to 0.15.1 from 0.14.1 for embed allowEmptyGlob
2025-11-24 09:19:49 -08:00
Jeff McCune
8512ca01d1 docs: bump minor version to 0.105.0 with cue 0.15.1 2025-11-24 09:13:13 -08:00
Jeff McCune
e6f3a6d8d9 cue: update to 0.15.1 from 0.14.1 for allowEmptyGlob 2025-11-24 09:11:02 -08:00
Jeff McCune
3d696b958b version 0.105.1 fixes oci chart auth (#447) 2025-09-25 07:12:41 -07:00
Jeff McCune
1bf627b5e1 Merge pull request #447 from manrueda/feat-oci-chart-authentication
Allows authentication of OCI charts
2025-09-25 07:08:13 -07:00
Manuel Rueda
df69198cef allows authentication of OCI charts 2025-09-25 09:52:38 -04:00
Jeff McCune
810a7c6649 version 0.105.0 with kubectl-slice 2025-09-24 20:58:09 -07:00
Jeff McCune
b24e35f46d add kubectl-slice subcommand
It's really good at filtering resources like Namespace, Secret, and
CustomResourceDefinition.  It's excellent at breaking up files into an
easy to navigate, clear structure that integrates well with tools.  fzf
for example.
2025-09-24 20:46:49 -07:00
Jeff McCune
22c65bbc6c add ca-certificates to container image for git
Without this patch git fails to pull from https remotes.
2025-09-01 06:45:58 -07:00
Jeff McCune
e913ce6368 Merge pull request #444 from WhyKickAmooCow/patch-1
Add ca-certificates to container image
2025-09-01 06:43:31 -07:00
Adam
0abfa660d0 Add ca-certificates to container image
Signed-off-by: Adam <WhyKickAmooCow@users.noreply.github.com>
2025-09-01 23:20:39 +12:00
Jeff McCune
8883a89c44 bump cue to v0.14.1 from v0.13.0-rc.1
Commands to update the tests and website docs to the new version:

    make bump
    make update-docs
2025-08-31 15:51:35 -07:00
Jeff McCune
b4d8794e23 container: update to debian 13 and install git
Update to kubectl v1.33.4

Closes: #440
2025-08-31 15:51:34 -07:00
Jeff McCune
524b01a6b3 tasks: add v1alpha6 design task 2025-05-22 15:29:41 -07:00
Jeff McCune
c1a064fd70 claude: add settings with allowed tools 2025-05-22 15:29:40 -07:00
Jeff McCune
69f6a8b1eb claude: add make install reminder to development commands 2025-05-22 15:29:40 -07:00
Jeff McCune
4482930e82 cue: improve error messages to show user field paths
Use v.Validate(cue.Concrete(true)) before decoding to get user-friendly
field paths like "holos.metadata.name" instead of internal type names
like "#Metadata.name". This makes error messages clearer and matches
the output style of cue export.

With this patch the error is as I want it to be:

    ❯ holos show platform
    error at internal/platform/platform.go:153: holos.metadata.name: incomplete value string
    holos.metadata.name: incomplete value string:
        /Users/jeff/Holos/holos/tmp/examples/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha6/types_go_gen.cue:419:8
2025-05-22 15:01:36 -07:00
Jeff McCune
12389320a6 lint: make golangci-lint happy 2025-05-22 15:01:35 -07:00
Jeff McCune
b024346f19 claude: update guidance for coding agent 2025-05-22 10:32:13 -07:00
google-labs-jules[bot]
e58433fe08 docs: Update Helm version and add known issue
- Updates the tested Helm version to v3.17.3 in the setup documentation.
- Adds a "Known Issues" section to the setup documentation.
- Notes that Helm v3.18.0 produces incorrect output and links to the relevant Holos and Nixpkgs issues.

Fixes #433
2025-05-22 10:31:41 -07:00
Jeff McCune
c94d5b69c5 Merge pull request #431 from holos-run/jeff/compile
show: compile build plans with sub-processes
2025-05-22 09:56:06 -07:00
Jeff McCune
65430e3379 tests: replace cmp with holos compare yaml for yaml comparisons
Update testscript files to use `holos compare yaml` instead of `cmp`
for YAML file comparisons. This enables structural comparison rather
than textual comparison, ensuring semantic equivalence instead of
character-by-character matching.

Changes:
- Replace cmp stdout with cp stdout + holos compare yaml pattern
- Update all test files that compare YAML content
- Maintain test functionality while using semantic comparison

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-21 15:58:47 -07:00
Jeff McCune
1945392227 tests: refactor textual cmp to structural compare
Use holos compare yaml in place of cmp for structural comparisons rather
than text base diffs.  Useful to handle re-ordered fields and such.
2025-05-21 15:51:00 -07:00
Jeff McCune
b4361e3997 cli: fix exit code when compare subcommand not found
Previously the compare yaml command was not registered, and invalid
subcommands returned exit code 0. Now invalid subcommands properly
return exit code 1 while help commands continue to return 0.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-21 15:47:05 -07:00
Jeff McCune
1b562118a8 cue: upgrade v0.13.0-alpha.4 -> v0.13.0-rc.1 2025-05-21 15:30:15 -07:00
Jeff McCune
578677d9a6 cli: fix holos show tests calling wrong executable 2025-05-21 15:23:16 -07:00
Jeff McCune
49398fe211 compile: fix hanging tests calling cli.test compile recursively
When executing in the test harness, we cannot assume the os.Executable
is the holos executable, otherwise we recursively execute the
go-build1600354897/b1280/cli.test compile command forever.
2025-05-21 15:05:53 -07:00
Jeff McCune
82c429b8df helm: fix tests for helm 3.17.3
New kube api version needs to be updated in the tests.
2025-05-21 14:39:54 -07:00
Jeff McCune
012de360ac component: pass write to directory as function args
Makes it easier to test and know for certain where the value is being
set from.  Previously it wasn't clear how the write to directory was
being modified or read in the global config object.
2025-05-21 14:27:02 -07:00
Jeff McCune
c5600f2310 render: fix write-to flag handling
Fixed regression in holos render platform command where output manifests were
not being written to the deploy/ directory by default. Issues:

1. Ensured write-to flag is passed from render platform to render component
2. Fixed NewConfig to set the default WriteTo value properly
3. Fixed compile.go to ensure WriteTo is passed to components
4. Properly pass tempDir in renderAlpha5 method

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-21 12:19:39 -07:00
Jeff McCune
bc0266e5e3 show: compile build plans with sub-processes
This is a first stab at compiling build plans with a pool of holos
compile sub-processes.  Tested against the kargo-demo repository.
2025-05-21 11:34:31 -07:00
Jeff McCune
18be14c2ea compile: take a BuildPlanRequest as input
This package clarifies and simplifies the input protocol of the holos
compile command.  A BuildPlanRequest represents the complete context
necessary to compile a BuildPlan.

This patch also upgrades cue to v0.13.0-alpha4
2025-05-21 11:18:53 -07:00
Jeff McCune
ab49a40246 compile: export a build plan to stdout
This patch copies the build plan rendering from the show buildplans
command to implement the rendering in the compile subcommand.
2025-05-21 11:18:52 -07:00
Jeff McCune
28a0a3625e compile: add basic structure of holos compile command
This command reads Component objects from a reader, exports a BuildPlan
from CUE, then marshals the result to the writer.

The purpose is to run concurrent instances of CUE to speed up build plan
generation prior to task execution.
2025-05-21 11:18:52 -07:00
Jeff McCune
1bdb580aca compare: improve error message
Previously the error message didn't look nice:

    holos compare buildplans before.yaml after.yaml
    could not run: document 1: -    holos.run/stack.name: httpbin
    +    holos.run/stack.name: httpbinX at internal/compare/compare.go:331

With this patch:

    holos compare buildplans before.yaml after.yaml
    error at internal/compare/compare.go:331: document 1 not equivalent:
    -    holos.run/stack.name: httpbin
    +    holos.run/stack.name: httpbinX
2025-05-21 11:09:58 -07:00
Jeff McCune
c1b749c6bc claude: remove local settings 2025-05-21 10:42:04 -07:00
Jeff McCune
12ee084759 Merge pull request #432 from holos-run/jeff/claude-compare
Compare BuildPlans
2025-05-21 10:08:50 -07:00
Jeff McCune
6a2a81e3fc compare: add not-backwards-compatible test case to verify spec 4 2025-05-17 23:04:32 -07:00
Jeff McCune
204ce788b6 compare: clarify backwards compatibility behavior in doc comment
- Add detailed explanation of what isBackwardsCompatible controls
- Include example showing how newer versions can have additional fields
- Clarify that fields in before must always be present in after
- Make the documentation more user-friendly to prevent confusion

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 22:56:00 -07:00
Jeff McCune
abb014ee48 compare: clarify doc comment and fix backwards compatibility test logic
- Replace 'a' with 'before' and 'b' with 'after' in BuildPlans doc comment for clarity
- Fix backwards compatibility test case by swapping before/after files:
  - before.yaml now correctly has fewer fields (older version)
  - after.yaml now correctly has more fields (newer version with enhancements)
- Update comparison logic to filter fields from after that don't exist in before
- This correctly implements spec 4: after may have fields missing from before when backwards compatible

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 22:42:31 -07:00
Jeff McCune
e7cd480510 compare: fix formatting with make fmt 2025-05-17 22:26:41 -07:00
Jeff McCune
bd32ad62a4 cli: add --backwards-compatible flag to compare buildplans command
- Add boolean flag --backwards-compatible (default false) to control backwards compatibility behavior
- Flag enables backwards compatibility mode where the second file may have fields missing from the first file
- Maintains existing behavior when flag is not set (strict comparison)
- Help text clearly explains the flag's purpose

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 22:24:50 -07:00
Jeff McCune
57789b48e5 compare: add test for backwards compatibility spec requirement
- Add test case BuildPlan_4 for BuildPlan spec 4: b may have fields missing from a if isBackwardsCompatible is true
- Implement backwards compatibility logic in comparison functions
  - Add filterToCommonFields() to filter fields based on what exists in both structures
  - Update compareDocumentLists() and compareStructures() to accept isBackwardsCompatible parameter
  - When backwards compatible mode is enabled, only compare fields that exist in both structures
- Update test framework to support isBackwardsCompatible field in test cases
- Test verifies that missing labels, annotations, and nested fields don't cause failure in backwards compatible mode

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 22:19:56 -07:00
Jeff McCune
8476a8748f cli: fix compare command to add missing backwards compatibility parameter
The BuildPlans function signature was updated to include an isBackwardsCompatible
boolean parameter, but the CLI command was not updated to provide this parameter.
Fixed by passing false for now to maintain existing behavior.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 22:04:53 -07:00
Jeff McCune
2b89b45fc5 compare: add comprehensive BuildPlan spec test coverage
- Reference BuildPlan spec items in test names (BuildPlan_3.1, BuildPlanFile_1, etc.)
- Add missing test cases for BuildPlan spec requirements:
  - BuildPlanFile_2: each object matches exactly one unique object
  - BuildPlan_5: fields in 'a' must exist in 'b'
  - BuildPlan_6.1-6.3: null/empty/missing field equivalence
  - BuildPlanValid_1-2: duplicate objects and same metadata.name
- Implement null/empty/missing field equivalence (spec 6)
  - Add normalizeStructure() to handle spec 6 requirements
  - Ensure null, [], and missing fields are treated as equivalent
- Update test messages to clarify spec coverage intent

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 22:01:13 -07:00
Jeff McCune
23e38f1e7e compare: add more clarity to BuildPlans spec 2025-05-17 21:46:42 -07:00
Jeff McCune
a0f6776358 compare: align test cases with BuildPlans specification
Add comprehensive test cases to cover all specification requirements:
- File spec requirement 1: equal number of BuildPlans
- BuildPlan spec 1&2: all fields must be equivalent
- BuildPlan spec 3: nested objects must match recursively
- BuildPlan spec 4: exact field value matching with exceptions
- BuildPlan spec 4.1: unordered spec.artifacts list
- BuildPlan spec 4.2: key order irrelevance
- Toleration 1: multiple identical BuildPlans treated as unique
- Toleration 2: multiple BuildPlans with same metadata.name

Update existing test names and messages to clearly reference the specific specification requirements they validate.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 21:02:44 -07:00
Jeff McCune
830bb4f804 compare: doc spec for BuildPlan file equivalence
Document what it means for two BuildPlan Files to be equivalent.  So we
can write tests then verify them.
2025-05-17 20:48:36 -07:00
Jeff McCune
2e02bfe48a compare: clean up implementation and simplify diff output
Remove all label-specific references and simplify the diff output to show only what go-cmp reports. Extract field differences for cleaner test error messages while keeping the implementation generic for all fields.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 20:46:05 -07:00
Jeff McCune
b0d5fe0072 compare: add v1alpha6 compiler test case
This patch adds the output of the new stdio concurrenet compiler being
developed for v1alpha6 as after, which is semantically equivalent to the
v0.104.1 v1alpha5 holos show buildplans output.

Note the field orders and relative object order in the stream is quite
different.
2025-05-17 16:41:48 -07:00
Jeff McCune
f1ae5db910 compare: simplify implementation by removing label-specific handling
- Remove all label-specific functions and logic
- Treat labels as regular nested fields with no special handling
- Extract field differences from go-cmp output for test compatibility
- Remove before/after YAML output, show only the extracted differences
- All tests pass with this simplified, generic implementation
2025-05-17 15:01:35 -07:00
Jeff McCune
bc8ad22492 compare: add test case for annotations 2025-05-17 14:47:50 -07:00
Jeff McCune
4da0740a5e compare: make implementation generic by removing seq-specific logic
- Remove all hardcoded references to seq label
- Extract all labels for comparison instead of handling seq specially
- Compare structures without labels first, then check label differences
- Replace seq-specific functions with generic label handling functions
- This makes the implementation generic for any arbitrary labels
2025-05-17 14:37:13 -07:00
Jeff McCune
ee2d86d7df compare: remove unused functions
- Remove unused getCompositeKey function
- Remove unused deepEqual function
- These were replaced by the bipartite matching algorithm
2025-05-17 14:33:02 -07:00
Jeff McCune
7a94cc5f87 compare: fix labels3 test by implementing proper document matching
- Implement bipartite matching algorithm to match documents
- First pass finds exact matches including all labels
- Second pass handles unmatched documents and reports seq differences
- This allows labels2 (reordered documents) to pass without errors
- This allows labels3 (changed seq values) to report seq differences

The key insight is that labels3 has a document with seq:2 that has no
exact match in the after file (which only has seq:1 documents), so we
need to report the seq difference for that case.
2025-05-17 14:26:46 -07:00
Jeff McCune
0e9b02f337 compare: add test case for same object dupliacted in after
When the objects in the before stream differ only by a deeply nested
field value, the second object in before should not match any object in
after.
2025-05-17 13:57:23 -07:00
Jeff McCune
a419b08cca compare: implement labels2 test case for out-of-order object comparison
- Sort documents by composite key before comparison to handle out-of-order objects
- Exclude transient labels (like 'seq') from composite key generation
- Add deepCopy method to avoid modifying original structures
- Add removeTransientLabels to exclude labels that shouldn't affect semantic equivalence
- Use copies of documents in compareStructures to preserve original data
- Support comparison of identical objects that appear in different order in streams

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 13:47:49 -07:00
Jeff McCune
04f1a21ee8 compare: add test case for two out of sequence objects 2025-05-17 13:36:53 -07:00
Jeff McCune
88663e223b compare: remove all test-specific references from implementation
- Remove hardcoded test case checks for holos.run/stack.name and buildplan4
- Rename extractTestErrors to extractDifferences for clarity
- Make implementation completely generic to extract any field additions
- Maintain backward compatibility by keeping the same output format
- Remove all test-specific knowledge from the compare package

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 13:23:08 -07:00
Jeff McCune
2f77b1c09f compare: implement labels test case with expectedErrors list support
- Refactor test infrastructure to support expectedErrors as a list instead of single expectedError
- Implement generic field difference extraction for any object additions
- Update extractTestErrors method to handle arbitrary field additions
- Migrate all test cases from expectedError to expectedErrors format
- Maintain backward compatibility with existing test expectations
- Add support for detecting and formatting any field additions in diff output
- Clean up go-cmp type annotations in error messages for better readability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 13:11:05 -07:00
Jeff McCune
e2ea6deda2 compare: add test case for arbitrary labels 2025-05-17 12:59:15 -07:00
Jeff McCune
2131a99a30 compare: format and fixup claudes changes 2025-05-17 12:41:30 -07:00
Jeff McCune
9702efca70 compare: add testcase when after is a superset of before 2025-05-17 12:36:11 -07:00
Jeff McCune
658453bd76 compare: display full diff output from go-cmp
Enhance error messages to show the full diff output from go-cmp along
with specific field errors. This provides better visibility into all
differences between BuildPlans when they don't match.

- Show both specific field errors and full diff
- Maintain backward compatibility with existing tests
- Provide comprehensive diff output for debugging
- Improve user experience when comparing BuildPlans

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:48:48 -07:00
Jeff McCune
da7430e741 compare: fixup different labels test case name 2025-05-17 11:46:05 -07:00
Jeff McCune
57b491065a compare: implement buildplan3 test with diff error messages
Add support for showing meaningful diff messages when BuildPlans don't
match. Extract specific field differences from go-cmp diff output to
provide targeted error messages for common mismatches.

- Use go-cmp with custom transformers for order-independent comparison
- Parse diff output to extract specific field differences
- Show targeted error messages for label mismatches
- Pass buildplan3 test for different stack names

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:44:33 -07:00
Jeff McCune
48a40c808f compare: add testcase when labels are different 2025-05-17 11:41:08 -07:00
Jeff McCune
228dc3474b compare: implement buildplan2 test with deep order-independent comparison
Extend comparison logic to handle full BuildPlans with out-of-order
artifacts. Implement recursive deep comparison that sorts arrays at
all levels for true order-independent equality checking.

- Add deepEqual method for recursive order-independent comparison
- Improve sortSlice to handle nested maps correctly
- Support comparing complex BuildPlans with reordered artifacts
- Pass buildplan2 test for full BuildPlan comparison

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:36:39 -07:00
Jeff McCune
d83765e980 compare: add out of order artifacts test case 2025-05-17 11:30:45 -07:00
Jeff McCune
6ca6080a3f compare: add full buildplan test case 2025-05-17 11:27:08 -07:00
Jeff McCune
491efa0f4f compare: implement streams3 test case for duplicate names with labels
Extend the composite key function to handle BuildPlans with identical
names but different labels. This ensures proper matching when BuildPlans
have the same metadata.name but differ in other metadata fields.

- Enhance getCompositeKey to include sorted label key-value pairs
- Support BuildPlans with duplicate names distinguished by labels
- Ensure order-independent comparison works with complex metadata
- Pass streams3 test case for duplicate names with different labels

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:24:22 -07:00
Jeff McCune
5c1689b52b compare: add duplicate name test case 2025-05-17 11:19:16 -07:00
Jeff McCune
cd33f7b1dc compare: add name field to all test cases
Implement the name field for all test cases to provide more descriptive
test names in the test output. This improves test readability and
maintenance.

- Add name field to testCase struct
- Update test logic to use name field when provided
- Add descriptive names to all test cases
- Maintain nested test structure with directory/name hierarchy

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:13:43 -07:00
Jeff McCune
80559b7ca6 compare: add test case name field example 2025-05-17 11:11:16 -07:00
Jeff McCune
de9b177c71 compare: add descriptive messages to test cases
Enhance test cases with descriptive messages that explain what each
test is verifying. These messages are displayed when tests fail,
improving debugging experience.

- Add msg field to testCase struct
- Update all testcase.json files with descriptive messages
- Use messages in test assertions for better failure output
- Ensure all tests continue to pass with new structure

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:06:09 -07:00
Jeff McCune
ebb8d8d44c compare: implement streams2 test case with order-independent comparison
Switch to using google/go-cmp for more robust comparison logic that
handles unordered document lists. This allows BuildPlans to be
considered equivalent even when documents appear in different orders.

- Replace custom equalMaps with cmp.Equal
- Add order-independent comparison for document lists
- Use cmpopts.SortSlices with composite key for sorting
- Support streams2 test case with reordered documents

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 11:03:06 -07:00
Jeff McCune
c9994a0851 compare: add out of order test case 2025-05-17 10:54:14 -07:00
Jeff McCune
45568f2e6d compare: implement streams test case for multi-document YAML
Extend BuildPlan comparison to handle YAML streams containing multiple
documents separated by ---. The implementation now properly parses and
compares files with multiple BuildPlan objects.

- Add parseYAMLStream function to handle multi-document YAML files
- Add compareDocumentLists to compare lists of parsed documents
- Refactor single document parsing to use streaming approach
- Maintain order-sensitive comparison for document sequences

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 10:49:26 -07:00
Jeff McCune
46cb7f5d29 compare: add streams test case 2025-05-17 10:48:53 -07:00
Jeff McCune
f34624e416 compare: implement minimal test case for BuildPlan comparison
Implement basic BuildPlan comparison functionality that can handle the
minimal test case. The implementation reads and parses YAML files,
then compares them for semantic equivalence.

- Add file reading and YAML parsing logic
- Implement basic map comparison for BuildPlans
- Handle empty files case with NotImplemented error
- Use errors.Format for consistent error handling
- Support minimal BuildPlans with just version and kind fields

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 10:14:33 -07:00
Jeff McCune
19690d760a compare: add minimal equal test case 2025-05-17 10:11:28 -07:00
Jeff McCune
35ede4293b Rename empty-files test case to empty
Simplify test case directory name from empty-files to empty.

- Rename testdata/empty-files/ to testdata/empty/
- Test continues to pass with cleaner naming

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 09:59:58 -07:00
Jeff McCune
1b499b1ab6 Rename want.json to testcase.json in compare package
Rename test configuration files from want.json to testcase.json for
better clarity of purpose. Updated test code to match the new naming.

- Rename want.json to testcase.json in testdata
- Update variable names from wantData to testcaseData
- Update error messages to reference testcase.json

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 09:58:03 -07:00
Jeff McCune
6a3b8da7f2 Refactor compare buildplans tests to package level
Move compare functionality from CLI package to dedicated internal/compare
package for better separation of concerns. CLI now acts as a thin wrapper
calling the compare package implementation.

- Create internal/compare package with Comparer type
- Move tests from CLI to compare package
- Update test fixtures path to testdata directory
- Replace CLI error with errors.NotImplemented()
- Clean up redundant test files from CLI package

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 06:59:48 -07:00
Jeff McCune
ee9969d409 Add structure for holos compare buildplans command
Introduce a new compare command with buildplans subcommand to verify
semantic equivalence between BuildPlan files. This will help users
validate their configurations remain equivalent when migrating from
v1alpha5 to v1alpha6.

- Add compare command structure with buildplans subcommand
- Implement table-driven test infrastructure reading from fixtures
- Create first test case with empty YAML files that expects failure
- Command returns "not implemented" error as a placeholder

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-17 06:46:40 -07:00
Jeff McCune
b635c233fa claude: initialize 2025-05-17 06:30:56 -07:00
Jeff McCune
5290a2044f Merge pull request #429 from holos-run/dependabot/go_modules/github.com/docker/docker-27.1.1incompatible
build(deps): bump github.com/docker/docker from 27.0.0+incompatible to 27.1.1+incompatible
2025-05-05 19:43:05 -07:00
dependabot[bot]
4d47963b0a build(deps): bump github.com/docker/docker
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 27.0.0+incompatible to 27.1.1+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/commits/v27.1.1)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-version: 27.1.1+incompatible
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-10 04:18:56 +00:00
Jeff McCune
4c05e4b410 deps: upgrade go-git to 5.13.0 2025-04-09 21:16:59 -07:00
Jeff McCune
851ee2b33d component: refactor validator command for v1alpha6
This patch changes the behavior of all commands to execute in the
current working directory of the platform root cue module.  This gets
cue vet working well, otherwise it complains about fully qualified
paths.  The cue command line expects the current working directory to be
a cue module.

The Env field of the Command schema wasn't implemented, so we remove it
from the schema.  It's unclear it's necessary.  Setting the environment
in the parent context should suffice for current use cases.
2025-04-07 23:43:42 -07:00
Jeff McCune
063df94fed component: add test coverage for helm generator 2025-04-06 19:33:02 -07:00
Jeff McCune
3b2605aacc component: refactor to table test cases
Dry up the tests using a helper function to iterate over a simple
struct, they're all executing a component and validating the rendered
manifests against an expected data file.
2025-04-06 13:00:00 -07:00
Jeff McCune
5b5deca5d2 component: add test coverage for join transformer 2025-04-06 12:41:18 -07:00
Jeff McCune
9d92c292a6 component: refactor command and kustomize transformers
Previously the command and kustomize transformers were implemented as a
stand alone function.  The current method is to implement tasks as
methods on the generator, transformer, or validator structs.  This patch
refactors the stand alone functions to methods for consistency.

It also adds test coverage for the command and kustomize transformers.
2025-04-05 17:30:47 -07:00
Jeff McCune
a9e8756687 docs: generate v1alpha6 core api docs 2025-04-04 17:16:45 -07:00
Jeff McCune
3a205ccce7 component: test v1alpha6 Command generator directory support
This test validates the command generator stores a directory artifact
properly.
2025-04-04 17:13:07 -07:00
Jeff McCune
9597b313a4 component: test v1alpha6 Command generator
This patch adds a basic test of the v1alpha6 Command generator.
2025-04-04 15:59:56 -07:00
Jeff McCune
fafae89403 component: refactor to consistently use absolute paths
Testing is problematic because the current working directory is not the
platform root.  This patch refactors the codebase to consistently store
and use the platform root directory to construct absolute paths for
reading and writing files during a BuldPlan execution.
2025-04-04 13:29:52 -07:00
Jeff McCune
33bc8f8e23 component: add test coverage using testutils.Fixtures
There isn't good test coverage of the v1alpha6 component package
responsible for producing and executing a BuildPlan from a Component.
This patch adds test coverage using a shared testutils package using the
generate package to generate a v1alpha6 platform module, then copying
fixtures from the embedded testutils.Fixtures fs.FS.
2025-04-03 22:22:39 -07:00
Jeff McCune
04ce0e8710 Merge pull request #412 from cameronraysmith/nix-pkg
feat(nix): add nix flake and derivation
2025-04-03 16:11:22 -07:00
Jeff McCune
1371e6a84b cmd: add holos version
Always bugged me holos --version works but holos version does not.
2025-04-03 15:50:32 -07:00
Jeff McCune
7f7d373b2a maint: standardize on gopkg.in/yaml.v3
Tidy up sigs.k8s.io/yaml use gopkg.in/yaml.v3 instead
2025-04-03 15:38:02 -07:00
Jeff McCune
b78e862d09 maint: cleanup unused functions and code
ExtractYAML is no longer used, also cleans up connectrpc packages, error
handling, tracing, and memory profiling.
2025-04-03 15:33:42 -07:00
Jeff McCune
0866ace8d8 maint: use cspell from doc/website
Get rid of package.json at the root.
2025-04-03 14:51:35 -07:00
Jeff McCune
86f0d277d6 logger: remove dependency on cobra 2025-04-03 13:53:28 -07:00
Jeff McCune
87c537bb0e logger: move tint package into logger package 2025-04-03 13:53:27 -07:00
Jeff McCune
708f58638d maint: remove unused builder package
Good to finally kill this one off, the behavior has moved to the
platform and component packages using an embedded interface to abstract
different APIVersions.
2025-04-03 13:53:27 -07:00
Jeff McCune
b590824034 maint: remove unused strings package 2025-04-03 13:53:27 -07:00
Jeff McCune
17287a3bf8 logger: remove console package
Consolidate into the tint package.
2025-04-03 13:53:27 -07:00
Jeff McCune
5076f8e7b9 docs: minor improvements for the core schemas 2025-04-03 13:53:26 -07:00
Jeff McCune
49cdf2f9d5 component: refactor BuildPlan Context to BuildContext
Name it consistently, Context can be a bit ambiguous.  Also define the
tags used to pass data from the platform layer to the component layer as
go constants in the core api for clear documentation.
2025-04-03 13:53:26 -07:00
Jeff McCune
7a31849e96 cleanup server client tilt, etc...
This patch removes all of the experimental prototype code that's no
longer relevant to the holos command line tool.  Also all versions
before v1alpha5
2025-04-02 18:49:33 -07:00
Jeff McCune
1028b8a56f Merge pull request #414 from holos-run/jeff/v1alpha6
Partial implementation of v1alpha6 with Command tasks and BuildContext for late binding tempDir
2025-04-02 13:07:39 -07:00
Jeff McCune
dbe832a05c docs: make v1alpha6 unreleased 2025-04-02 12:29:23 -07:00
Jeff McCune
6c4561bd84 platform: refactor component selectors
Previously we were having to define selectors on every command that
processes platform components.  This patch fixes the problems by moving
component selectors to the platform.Config struct.

Result:
Commands that process platform components need to explicity add the
platform.Config.FlagSet to the command, but otherwise the configuration
is consolidated and reused as we want.  The holos show platform command
always uses the default configuration so we don't bother adding the
flags for this case.
2025-04-02 12:13:03 -07:00
Jeff McCune
36ade0b5b6 cli: refactor holos render platform command
This patch refactors the holos render platform command to use the new
platform package instead of the previous builder package.  Similar to
the holos show buildplans command, the PerComponentFunc is used, the
main difference is it simply constructs an argument vector to execute
holos render component injecting the proper flags and tags for cue to
produce the build plan.
2025-04-02 11:32:49 -07:00
Jeff McCune
ddb6dc8dee maint: cleanup undused platform_test.go 2025-04-02 11:32:49 -07:00
Jeff McCune
99feed7c8b cli: fix show build plans injected tags
Previously the test coverage for the holos show buildplans command was
covered by testscript.  This is a problem because it's difficult to do a
structural comparison in testscript, we're limited to comparing stdout
with a file.

This patch refactors the test script to a go testing test using testify
to compare structures instead of comparing text output as was done
previously.

In the process it became clear the holos show buildplans command is not
injecting tag values properly.  The component name, labels, and
annotations were missing.  This patch fixes that problem by passing the
tags from the core Platform.Components element to the
component.Component.BuildPlan method.

Note, I'm not happy with how ad-hoc the tags have become to pass around.
Ideally we'll refactor them into a proper protocol, passing the entire
core.Component structure to the method that produces a build plan.  The
idea of passing this structure over standard intput to a build plan task
worker keeps coming up over an over again.
2025-04-02 11:32:49 -07:00
Jeff McCune
0ff59b4feb cmd: update scripts for new cue ordering 2025-04-02 11:32:48 -07:00
Jeff McCune
b01bcd8a1d docs: remove draft migrate ApplicationSet blog post 2025-04-02 11:32:48 -07:00
Jeff McCune
95d53062da cli: refactor how build plans are loaded
Previously the show buildplans, render platform, and render component
commands used duplicate code paths to obtain build plans and render
components.  This patch consolidates the behaviors into the component
and platform packages.  The platform Build method is used to call a
different PerComponentFunc depending on the use case of showing build
plans or rendering each component in the platform.

The behavior of loading a build plan from cue is consolidated into the
Component.BuildPlan method.

Result:

1. Platform.Build is reused for both show buildplans and render platform
2. Component.BuildPlan is reused for both show buildplans and render
   component.
2025-04-01 20:59:39 -07:00
Jeff McCune
779f04b85a builder: refactor to component and platform packages
There are only two version specific cue builders, a Platform and a
component BuildPlan. The platform is anemic while the component holds
most of the behavior of the Holos rendering pipeline.  Without this
patch the two types of builders were mixed together in the same package.

This patch refactors the version specific builders into the platform and
component packages respectively.  The version specific BuildPlan builder
is where most of the behavior lies and what we expect to evolve most
over time.  With the typemeta.yaml approach, we can easily switch code
paths, isolated to the process boundary since holos render platform
invokes sub processes for each holos render component command.

This patch helps focus on good test coverage by establishing a clear
area to add version specific test cases, both for the platform layer and
the individual component layer.
2025-04-01 08:51:36 -07:00
Jeff McCune
2a13a65a4d component: use v1alpha6 embedded platform for tests
This patch uses the same platform embedded into the holos init platform
v1alpha6 subcommand for the test cases to exercise backwards
compatibility.

This is prep work to refactor the builder logic into the component
package for better organization and test coverage.
2025-04-01 08:51:36 -07:00
Jeff McCune
27cfd1de6c component: add basic minimal test case
The minimal test case is an empty build plan with only an apiVersion and
a kind field.  This patch adds such a test case with the typemeta.yaml
file and enough cue code to get the cue.Value and load it into a Go
struct for holos to process.

Result:
The test case passes and provides a starting point to add additional
coverage.

    go test -coverprofile=coverage.out ./internal/cli/render/component/...

    ok      github.com/holos-run/holos/internal/cli/render/component        0.612s  coverage: 46.7% of statements
2025-04-01 08:51:36 -07:00
Jeff McCune
78b35cfa77 component: refactor holos render component for tests
There hasn't been good unit level test coverage of the component
rendering behavior.  This patch refactors the behavior of the holos
render component command into a component package with a primary
Component struct.

The result is the package is better organized for unit level testing
across api versions, notably v1alpha6 with the new arbitrary Command
generators, transformers, and validators.
2025-04-01 08:51:35 -07:00
Jeff McCune
1321e27208 render: discriminate components on typemeta.yaml
Previously, holos loaded the full CUE Instance to discriminate against
the APIVersion and Kind.  This is a problem because we need to know the
api version to know what tags to inject.  For example, v1alpha6 needs to
have the build context injected but v1alpha5 does not.

This patch fixes the problem by changing holos render component to look
for a `typemeta.yaml` file in the component directory.  The command line
tool discriminates on the kind and apiVersion specified in this file
prior to building the CUE instance.

Result:

We're able to inject version specific cue tags into the CUE instance.

Note previously the build context was passed to CUE by writing a file to
the filesystem and using the embed functionality.  We cannot proceed
with this embed approach because the filesystem writes are not safe for
concurrent use.  Therefore, we inject the build context as a tag which
is safe for concurrent use.
2025-04-01 08:51:35 -07:00
Jeff McCune
a4ea3cd500 render: manage temp directory as build context
Create the temporary directory and write the
components/buildcontext.json file
2025-04-01 08:51:35 -07:00
Jeff McCune
e1f654b53c api: define BuildContext to bind tempDir
Previously, the build plan CUE code has no way to refer to the temporary
directory managed and owned by the `holod render component` command.
This patch adds a placeholder context field for holos to fill in just
before exporting the final build plan from CUE.

This allows the user to refer to the tempDir field without the value
being concrete, useful for constructing an argument vector for the new
Command task.
2025-04-01 08:51:34 -07:00
Jeff McCune
2b246a2691 api: define command task for v1alpha6
With new EnvVar types modeled after the core kubernetes API.
2025-04-01 08:51:34 -07:00
Jeff McCune
7b583f036e builder: fix holos render platform for v1alpha6
Without this patch the holos render platform command fails:

    holos init platform v1alpha6
    holos render platform

    could not run: unsupported version: v1alpha6 at internal/builder/platform.go:102

This patch copies the v1alpha5 builder implementation to the v1alpha6 go
package and adds a switch case handler for Platform v1alpha6 apiVersions

Result:

    holos render platform

    rendered platform in 15.459µs
2025-04-01 08:51:34 -07:00
Jeff McCune
ed419af340 api: default apiVersion to v1alpha6
Without this patch the default apiVersion for the Platform resource is
v1alpha5 even though we're using the v1alpha6 schemas.  This patch
updates the default value and re-generates the docs and cue schema using
`make generate`

Before:

    holos show platform

    apiVersion: v1alpha5
    kind: Platform
    metadata:
      name: default
    spec:
      components: []

Result:

    holos init platform v1alpha6 --force
    holos show platform

    apiVersion: v1alpha6
    kind: Platform
    metadata:
      name: default
    spec:
      components: []

Rendering the platform fails as expected:

    holos render platform

    could not run: unsupported version: v1alpha6 at internal/builder/platform.go:102
2025-04-01 08:51:33 -07:00
Jeff McCune
71d293e58e api: import core v1alpha6 from author v1alpha6
Also remove the types.cue cue package config, it had the wrong package
name of v1alpha5 instead of core, meaning it was never actually used.
2025-04-01 08:51:33 -07:00
Jeff McCune
ef546f6614 cmd: add pkg definitions to v1alpha6 platform
Without this patch the holos render platform command fails with the
following error.  This patch adds the additional type definitions in CUE
to make the metadata.name field concrete.

This patch copies the pkg directory from the `cue.mod` directory using
`rsync -avxHP v1alpha5/ v1alpha6/` for both the core and author schemas.

Result:

    holos init platform v1alpha6
    holos render platform

    rendered platform in 33.917µs
2025-04-01 08:51:33 -07:00
Jeff McCune
b2b7c705a0 api: import v1alpha6 for holos render platform
Without this patch the v1alpha6 of the author schema imports v1alpha5 of
the core schema.  This patch imports the v1alpha6 core schema.

This is a necessary step to get holos render platform working after
holos init platform v1alpha6.

Result: We still have other things to copy over to establish v1alpha6.

    holos render platform

    could not run: holos.metadata.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:34
    holos.metadata.name: cannot convert non-concrete value string:
        /Users/jeff/Holos/command/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha6/types_go_gen.cue:305:2
2025-04-01 08:51:32 -07:00
Jeff McCune
d23b41cefe cmd: update v1alpha6 platform
This patch imports the v1alpha6 schemas for use with the v1alpha6
platform.
2025-04-01 08:51:32 -07:00
Jeff McCune
f886823405 cmd: holos init platform v1alpha6
Without this patch init fails with:

    holos init platform v1alpha6

    could not run: cannot generate: have: [v1alpha6] want: [v1alpha4 v1alpha5] at internal/generate/platform.go:68

This patch copies the v1alpha5 initial platform to establish v1alpha6.

Result: holos init platform v1alpha6 works.
2025-04-01 08:51:32 -07:00
Jeff McCune
d260343278 docs: set current version to v1alpha6
Without this patch the docs website has a conflict over the v1alpha5
path.  This patch fixes the problem by setting the current version to
v1alpha6.

Result: npm run build in doc/website now builds successfully.  All links
automatically route to v1alpha6 url paths.  The previous v1alpha5 paths
are accessible.

[INFO] [en] Creating an optimized production build...
[WARNING] Tags [launch] used in 2024-10-28-announcing-holos.mdx are not defined in tags.yml
[WARNING] Tags [cue] used in 2024-10-28-why-cue.md are not defined in tags.yml
[WARNING] Tags [feature] used in 2024-11-25-validators.mdx are not defined in tags.yml
[WARNING] No docs found in "api/author": can't auto-generate a sidebar.
[WARNING] No docs found in "api/core": can't auto-generate a sidebar.
2025-04-01 08:51:31 -07:00
Jeff McCune
04079385b5 docs: make generate for v1alpha6
Generate docs from the v1alpha6 schemas to make sure the website and
links are correct.

This results in the following warnings about conflicting routes when
running `npm run build` in doc/website

[INFO] [en] Creating an optimized production build...
[WARNING] Tags [cue] used in 2024-10-28-why-cue.md are not defined in tags.yml
[WARNING] Tags [feature] used in 2024-11-25-validators.mdx are not defined in tags.yml
[WARNING] Tags [launch] used in 2024-10-28-announcing-holos.mdx are not defined in tags.yml
[WARNING] No docs found in "api/author": can't auto-generate a sidebar.
[WARNING] No docs found in "api/core": can't auto-generate a sidebar.
[WARNING] Duplicate routes found!
- Attempting to create page at /docs/v1alpha5/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/api/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/api/author/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/api/core/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/common/example-component-integrate/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/common/example-component/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/glossary/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/support/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/architecture/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/comparison/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/gitops/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/gitops/argocd-application/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/gitops/flux-kustomization/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/kargo/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/local-cluster/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/oci-helm-charts/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/private-helm/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/structures/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/structures/clusters/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/topics/structures/environments/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/cue/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/hello-holos/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/helm-values/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/kustomize/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/overview/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/setup/, but a page already exists at this route.
- Attempting to create page at /docs/v1alpha5/tutorial/validators/, but a page already exists at this route.
This could lead to non-deterministic routing behavior.
2025-04-01 08:51:31 -07:00
Jeff McCune
7bd24adead docs: rsync -avxHP v1alpha5/ v1alpha6/
Establish v1alpha6 from v1alpha5
2025-04-01 08:51:31 -07:00
Jeff McCune
dab94961ff docs: npm run docusaurus docs:version v1alpha5
In preparation for v1alpha6 where we're designing a generic Command task
for use in generators, transformers, and validators, we need to create
permalinks for the existing v1alpha5 schemas.

This patch copies the current documentation to the v1alpha5 versioned
docs links and establishes v1alpha6 as the current version.

See: https://github.com/holos-run/holos/pull/413 for context.
2025-04-01 08:51:29 -07:00
Michael Crenshaw
f9c3b328a8 docs: spelling fix (#416)
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>
2025-03-31 12:55:01 -07:00
Cameron Smith
abb5b22037 feat(nix): add packages used in docs or adjacent
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-23 00:09:16 -05:00
Cameron Smith
5e5a517df3 chore(direnv): use flake
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-23 00:09:16 -05:00
Cameron Smith
e52f009132 chore(nix): sync flake lock
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-23 00:09:16 -05:00
Cameron Smith
48b92b4423 chore(gitignore): add nix/direnv local data
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-23 00:09:16 -05:00
Cameron Smith
745d8e5010 feat(nix): init flake
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-23 00:09:16 -05:00
Cameron Smith
1245cc583f feat(nix): define derivation to build holos with buildGoModule
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-23 00:09:16 -05:00
Cameron Smith
5631370a48 feat(nix): define default devshell
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-22 22:01:36 -05:00
Cameron Smith
039583a755 feat(nix): add Makefile for bootstrapping nix and direnv
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-22 22:01:36 -05:00
Cameron Smith
d838880585 docs(nix): init readme
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
2025-02-22 22:01:36 -05:00
Jeff McCune
8e690b43ee ci: fix golangci-lint
Without this patch there are unexpected lint errors in version 1.60
where 1.61.0 passes locally on my machine.

This patch updates to:

    golangci-lint has version 1.64.5 built with go1.24.0 from 0a603e49 on 2025-02-13T21:19:55Z
2025-02-20 09:28:35 -08:00
Jeff McCune
a4ceb1cdb2 ci: update test cases with make bump
Previously using make bump to bump a version did not also update all of
the test cases and documentation to reflect the new version.  This patch
updates the make bump tasks call HOLOS_UPDATE_SCRIPTS=1 scripts/test to
keep the test cases and documentation in sync with the new version.
2025-02-20 09:14:37 -08:00
Jeff McCune
ddb5c0e07b ci: fix make lint failures resulting from version bumps
Without this patch lint fails with error:

    SA1019: testscript.RunMain is deprecated: use [Main]

This patch uses testscript.Main instead.
2025-02-20 09:07:47 -08:00
Jeff McCune
a14d3ba0f4 ci: fix make test failures resulting from version bumps
Previously the tests fail because they were not updated to use the new
version string in holos, or the new topo sort behavior in cue 0.12.0.

This patch updates the test cases using:

    HOLOS_UPDATE_SCRIPTS=1 scripts/test

Result: make test passes
2025-02-20 08:37:01 -08:00
Jeff McCune
f7e0470c48 version 0.104.0 with cue 0.12.0 2025-02-06 14:37:50 -08:00
Jeff McCune
d5c7b82684 go mod tidy 2025-02-06 14:33:14 -08:00
Jeff McCune
7d0392e596 update to cue 0.12.0
Most relevant for us: lots of fixes to the evaluator, enables the embed
and toposort experiments.
2025-02-06 14:31:39 -08:00
Gary Larizza
410b882d1d Merge pull request #403 from holos-run/gl/hello-holos-testscript
docs: Update Hello Holos tutorial to use testscript
2025-01-22 14:43:47 -08:00
Gary Larizza
e2648202dc Merge pull request #404 from holos-run/gl/kustomize-testscript
docs: Update Kustomize tutorial to use testscript
2025-01-22 14:43:33 -08:00
Jeff McCune
44c2fe220a test: fix helm capabilities test
Helm was upgraded in GitHub Actions resulting in an accidental failure
of the test case.
2025-01-17 12:33:28 -08:00
Jeff McCune
fe1ae2fa80 docs: migrate from an ApplicationSet blog post 2025-01-17 12:22:56 -08:00
Gary Larizza
8fbee1cbd9 docs: Update Kustomize tutorial to use testscript
PROBLEM:

The "Kustomize" tutorial has hardcoded code blocks and hasn't been
updated to use the automated testscript workflow.

SOLUTION:

Create a test for the Kustomize tutorial.
Create a testscript for the Kustomize test.
Update the Kustomize MDX file to load in data from the testscript directory.

OUTCOME:

The code content in the Kustomize tutorial now comes directly from the
testscript workflow.
2025-01-16 14:24:24 -08:00
Gary Larizza
982db2cccc docs: Update Hello Holos tutorial to use testscript
PROBLEM:

The "Hello Holos" tutorial has hardcoded code blocks and hasn't been
updated to use the automated testscript workflow.

SOLUTION:

* Create a test for the Hello Holos tutorial.
* Create a testscript for the Hello Holos test.
* Update the Hello Holos MDX file to load in data from the testscript directory.

OUTCOME:

The code content in the Hello Holos tutorial now comes directly from the
testscript workflow.
2025-01-16 10:12:17 -08:00
Jeff McCune
e9d1240d63 docs: make update-docs for version 0.103.0 2025-01-12 14:26:27 -08:00
Gary Larizza
03fa4eaaa2 docs: Helm Values test updates
* Convert all files with.period.separators to hyphen-separators.
* Rename and markdown_test.go to be specific to Helm Values.
* Move helm-values_test.go to be in the same directory as the Helm Values doc.
* Move Blackbox common configuration CUE file to `config/prometheus` so it can be imported as necessary.
* Use explicit import statements for Blackbox common config in `blackbox` and `prometheus` components.

Closes: #399
2025-01-12 14:25:44 -08:00
Jeff McCune
e363f3a597 docs: add make update-docs task
We need to run this prior to tagging a release otherwise the tests fail
for the new version string.
2025-01-12 14:22:58 -08:00
Jeff McCune
8b49ed93be docs: release version 0.103.0 2025-01-12 14:09:45 -08:00
Jeff McCune
d2be9fe278 helm: add valueFiles for migration from an ApplicationSet
Without this patch migrating from [helm hierarchies] to Holos requires
the user to unify the value hierarchy.  This is a problem because helm
hierarchies are difficult to unify because it's not clear if or why a
value is used in the final results.  This makes it difficult to identify
how to resolve conflicts.

This patch adds `valueFiles` field to the Helm component kind.  This
field is intended to provide a direct migration path from the
ApplicationSet.spec.template.spec.sources.helm.valueFiles field.  With
this patch, users can directly migrate the values files to CUE using
`@embed`, then directly migrate the valueFiles field to reference the
values from within CUE.

Note we actively discourage the use of Helm value hierarchies.  The
feature is intended as a temporary migration tool.  We encourage the use
of CUE unification instead.  After migration, the valueFiles field
should be refactored to the values field as one unified structure in
CUE.  The valueFiles field makes this second order migration easier
becuase we can inspect and verify the complete rendered output, allowing
us to determine if a value is actually used in the final configuration
or is overridden.

[helm hierarchies]: https://medium.com/containers-101/using-helm-hierarchies-in-multi-source-argo-cd-applications-for-promoting-to-different-gitops-133c3bc93678
2025-01-12 13:30:29 -08:00
Jeff McCune
6ec341bbb1 docs: redirect /docs/api/core 2025-01-10 15:02:12 -08:00
Jeff McCune
13a4305b78 docs: add redirect for /blog/rendered-manifest-pattern (manifest instead of manifests) 2025-01-10 14:50:26 -08:00
Jeff McCune
0cfce3a823 docs: redirect rendered manifests pattern for now
Need a URL we can redirect when we publish our own variation on the
pattern with a link back to Akuity.
2025-01-10 10:55:06 -08:00
Jeff McCune
61d7539e1c docs: fix /docs/guides/ redirect 2025-01-09 16:03:50 -08:00
Jeff McCune
bf84724137 docs: add redirects for github.com/holos-run readme 2025-01-09 15:11:04 -08:00
Jeff McCune
9f0de7555c init: change to holos.example default cue module
Match the cue mod init behavior of a module named `cue.example`.
2025-01-09 13:57:26 -08:00
Gary Larizza
650636f944 Merge pull request #393 from holos-run/gl/update-helm-docs
Update Helm Values Tutorial to use testscript
2025-01-09 12:01:09 -08:00
Gary Larizza
b28c110694 Update Helm Values tutorial to use testscript
PROBLEM:

The Helm Values tutorial contains a fair bit of code/scripts, and we
need a way to test the steps we recommend to make sure nothing breaks
or slips out of date.

SOLUTION:

* Use `testscript` as a way to automate the execution of the steps in the doc and verify that none of the steps produce errors.
* Update the MDX file to directly reference the files embedded into the testscript.

OUTCOME:

* We have an automated way to perform the steps in the Helm Values document.
* We have unit tests that will fail should any of the commands being executed in the doc fail.
* The doc's MDX file directly references the files within the testscript, so we only need to modify the MDX file to update wording.
2025-01-09 11:53:53 -08:00
Gary Larizza
5bb3e90b38 Install raw-loader module
We use this module within our markdown tutorials (like the Helm Values
tutorial) to load in files generated by testscript.
2025-01-09 11:53:13 -08:00
Jeff McCune
6a60b613ff render: fix selectors (#394)
Without this patch selectors don't work as expected.  This patch
fixes selectors such that each --selector flag value configures one
selector containing multiple positive or negative label matchers.

Result:

Render build plans for cluster dev or cluster test.  Note the use of two
flags indicating logical OR.

    holos render platform --selector cluster=test --selector cluster=dev
    rendered external-secrets for cluster test in 299.897542ms
    rendered external-secrets for cluster dev in 299.9225ms
    rendered external-secrets-crds for cluster test in 667.6075ms
    rendered external-secrets-crds for cluster dev in 708.126541ms
    rendered platform in 708.795625ms

Render build plans for prod clusters that are not customer facing.  Note
the use of one selector with comma separated labels.

    holos render platform --selector "tier=prod,scope!=customer"
2025-01-08 21:09:00 -08:00
Jeff McCune
5862725bab builder: deprecate ExtractYAML, use cue embed instead
Easier to place the data, better supported in the ecosystem.
2025-01-02 18:53:10 -08:00
Jeff McCune
8660826b05 builder: protect LoadInstance with a mutex
CUE is not safe for concurrent access so we protect the main
LoadInstance function with a mutex lock.
2025-01-02 17:32:53 -08:00
Jeff McCune
449df91e33 docs: app.holos.run/description not cli
The core component documentation on the annotation used to configure the
display line for each rendered component was incorrect.
2025-01-02 08:36:37 -08:00
Jeff McCune
ac59173b30 ci: update holos-run/holos-action version (try 3)
Fix the use of digests when pulling and pushing images.  Pull the image
from ghcr.io before pushing it to quay.io
2024-12-23 10:33:45 -08:00
Jeff McCune
fb75e560fc ci: update holos-run/holos-action version (try 2)
When new container image versions are built, automatically update the
holos-run/holos-action to use the new version.

Users of the action automatically update by default as a result.
2024-12-23 09:52:09 -08:00
Jeff McCune
69a064e3ea ci: update holos-run/holos-action version
When new container image versions are built, automatically update the
holos-run/holos-action to use the new version.

Users of the action automatically update by default as a result.
2024-12-23 07:23:36 -08:00
Jeff McCune
71b72807bb ci: tag v0.102.1 for container images
We need a released tag to reference in workflows that use the container
image to render the platform configuration.

This is the first image, subsequent git tags will also build and publish
container images.
2024-12-21 08:08:51 -08:00
Jeff McCune
0e4ecf9d13 ci: fix error in containers.yaml 2024-12-21 07:33:31 -08:00
Jeff McCune
ec2fdadd44 ci: build container from any ref
Too hard to try and build back in time, so let's just get it working
then build containers going forward for tags.
2024-12-21 07:31:09 -08:00
Jeff McCune
38b082095f ci: drop linux/arm/v7 support
There aren't kubectl images to build against.
2024-12-21 07:14:21 -08:00
Jeff McCune
f9346ea7c0 ci: use Dockerfile from main when building tags
Problem: We can't build old tags because the wrong Dockerfile is used
from the old tag.

Solution: Save the Dockerfile from main and use it to build the tag.
This create a dirty working directory but that's OK.
2024-12-21 07:11:29 -08:00
Jeff McCune
0f7010288a ci: build distroless container image for holos
Push it to ghcr and quay.

 * sign images with cosign and oidc id token
 * add kustomize v5.5.0 to tools for distroless image

Usage:

    docker run -v $(pwd):/app -w /app --rm -it ghcr.io/holos-run/holos:v0.101.8 holos render platform
2024-12-21 06:58:57 -08:00
Jeff McCune
386fb89cc6 ci: replace lint workflow with cspell
The lint workflow was slow and we don't often change buf or angular
these days so they're not necessary.

The remaining valuable task is cspell, which we can speed up with a
dedicated step.
2024-12-20 13:52:54 -08:00
Jeff McCune
c5401d6b02 ci: speed up tests by killing steps 2024-12-20 11:57:05 -08:00
Jeff McCune
f215405643 docs: fix links in readme 2024-12-20 07:28:04 -08:00
Jeff McCune
2c79982bd3 cue: enable @embed for loading yaml (#385)
mpvl suggests @embed is a more ideal solution than our implementation of
core.Component.Instances for the use case of unifying YAML data updated
by Kargo Stage resources.

See the issue for a link to the discussion.
2024-12-20 07:14:01 -08:00
Jeff McCune
e5e4de3073 cue: update to 0.11.1
go get cuelang.org/go/cmd/cue@latest

    go: downloading cuelang.org/go v0.11.1
    go: upgraded cuelang.org/go v0.11.0 => v0.11.1
2024-12-20 07:09:39 -08:00
Jeff McCune
ec462f5f0b docs: redirect /docs/support 2024-12-19 22:13:04 -08:00
Jeff McCune
0e95a2812e cmd: expose MakeMain() for testing
I'd like to add the kargo-demo repository to Unity to test evalv3, but
can't get a handle on the main function to wire up to testscript.

This patch fixes the problem by moving the MakeMain function to a public
package so the kargo-demo go module can import and call it using the go
mod tools technique.
2024-12-19 15:19:46 -08:00
Jeff McCune
54efe3e24a core: pass --extract-yaml flag from platform to component (#376)
Previously holos render platform was not setting the --extract-yaml file
when calling holos render component, causing data file instances defined
in the Platform spec to be discarded.

This patch passes the value along using the flag.
2024-12-19 08:39:55 -08:00
Jeff McCune
f693f049f4 core: refactor --instance to --extract-yaml (#376)
Extract YAML is more clear and aligns with the schema docs for the
Component Instance field which has an extractYAML kind.  This also
leaves the door open for additional kinds of data extractors which are
almost certainly going to be needed.
2024-12-19 08:34:05 -08:00
Jeff McCune
85238710ac core: unify data files into config (#376)
Previously there isn't a good way to unify json and yaml files with the
cue configuration.  This is a problem for use cases where data can be
generated idempotentialy prior to rendering the platform configuration.

The first use case is to explore unifying configuration with decrypted
sops values, which isn't typical since Holos is designed to handle
secrets with ExternalSecret resources, but does fit into the use case of
executing a command to produce data idempotently, then make the data
available to the platform configuration.

Other use cases this feature is intended to support are the prior
experiment where we fetch top level platform configuration from an rpc
service, and the future goal of integrating with data provided by
Terraform.
2024-12-19 08:34:05 -08:00
Jeff McCune
3ec62d272e v1alpha5: update kargo crds to 1.1.1 2024-12-19 08:34:04 -08:00
Jeff McCune
49afb44fd4 docs: redirect /docs/comparison/ 2024-12-18 14:37:36 -08:00
Gary Larizza
a023f135ab Add a Comparisons page
PROBLEM:

We've noticed that Holos almost immediately gets compared to Timoni, and
we frequently get asked for specifics in how they're similar/different.

SOLUTION:

* Add a `Comparison` page.
* Include a section that compares Holos to Timoni

OUTCOME:

Fewer questions about how Holos compares to Timoni because people are
able to find that answer themselves on our docs page.
2024-12-18 14:33:52 -08:00
Jeff McCune
c6a3a5d689 docs: redirect /docs/kargo/ 2024-12-17 06:30:20 -08:00
Jeff McCune
3f1eed3f06 platform: add kargo.akuity.io custom resource definitions
Needed for Kargo integration.  Imported with timoni from v1.0.3 Kargo
CRD's.
2024-12-16 13:19:39 -08:00
Jeff McCune
7fb7df1441 docs: make the linter happy 2024-12-16 11:04:35 -05:00
Jeff McCune
a798111d4d docs: add oci helm charts example
Question came up in chat, there isn't a good example and it's a pain to
piece together from the reference docs.
2024-12-16 10:56:50 -05:00
Jeff McCune
3ddb823341 docs: add note about compinit
Andy ran into issues enabling completion without first figuring out how
to initialize the completion system.
2024-12-16 08:15:45 -05:00
Jeff McCune
70d48592c4 docs: fix environments topic
It didn't work, failed with:

  ❯ holos show buildplans --selector app.holos.run/city=ams
  could not run: Component.Name: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/instance.go:66
  Component.Name: 2 errors in empty disjunction:
  Component.Name: conflicting values "no-name" and "podinfo-ams":
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:6:12
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:6:13
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:35:2
      /Users/jeff/Holos/foo/holos-environments-tutorial/tags.cue:13:19
  Component.Name: conflicting values "podinfo" and "podinfo-ams":
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:6:12
      /Users/jeff/Holos/foo/holos-environments-tutorial/components/podinfo/podinfo.cue:7:8
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:6:13
      /Users/jeff/Holos/foo/holos-environments-tutorial/schema.cue:35:2

This was likely because the podinfo component was used in different ways
in different topics.  Don't use the shared component to fix the problem.
2024-12-13 09:20:52 -05:00
Jeff McCune
006f08df93 docs: add kargo place holder (#378) 2024-12-11 09:58:54 -08:00
Jeff McCune
39e2db5d37 docs: remove related content from youtube embed
Except stuff in our own channel.
2024-12-08 19:43:12 -08:00
Jeff McCune
ceb293fd8a docs: fix typescript className not class check error 2024-12-08 19:36:36 -08:00
Jeff McCune
188ff95015 docs: enable youtube fullscreen
Without this patch the fullscreen button is disabled.
2024-12-08 19:33:06 -08:00
Jeff McCune
5f658e0ba0 docs: add flux kustomization example (#374)
Almost identical to the ArgoCD Application example.
2024-12-08 19:20:12 -08:00
Jeff McCune
18b2850d3c platform: import flux custom resources
kustomize build https://github.com/fluxcd/flux2/manifests/crds\?ref=v2.4.0 \
      timoni mod vendor crds -f-
2024-12-08 19:03:18 -08:00
Jeff McCune
366a7fe93d docs: private helm repos need updated schemas (#370)
Document the need to run holos init platform v1alpha5 --force to use the
private helm repository feature.
2024-12-08 17:13:56 -08:00
Jeff McCune
f71d6d5bd9 helm: support private helm repositories (#370)
Previously holos unconditionally executed helm repo add which failed for
private repositories requiring basic authentication.

This patch addresses the problem by using the Helm SDK to pull and cache
charts without adding them as repositories.  New fields for the
core.Helm type allow basic auth credentials to be read from environment
variables.

Multiple repositories are supported by using different env vars for
different repositories.
2024-12-06 15:38:46 -08:00
Gary Larizza
4529673e93 Embed YouTube Video (#369)
PROBLEM:

We've created a YouTube video walking people through Holos and the Helm
Values tutorial, but now we need to embed it on the site for visitors to
watch.

SOLUTION:

* Create a `YouTube` MDX plugin
* Use that plugin on Overview and Helm Values
* Tune the video size/attributes using CSS

OUTCOME:

The Helm Values YouTube video is embedded on our site for visitors to
watch.
2024-12-06 15:33:03 -08:00
Jeff McCune
16a6447926 helm: support oci images in chart name
Without this patch we do not support installing Kargo from an OCI helm
chart.  We want to support:

```
Component: #Helm & {
	Name:      "kargo"
	Namespace: Kargo.Namespace

	Chart: {
		name:    "oci://ghcr.io/akuity/kargo-charts/kargo"
		version: "1.0.3"
		release: Name
	}
	EnableHooks: true

	Values: Kargo.Values
}
```

This patch fixes the problem by using the base name for filesystem cache
operations.
2024-12-03 12:15:06 -08:00
Jeff McCune
111a5944ff cue: bump to 0.11.0
go get cuelang.org/go/cmd/cue@latest
2024-12-02 12:37:19 -08:00
Jeff McCune
ff1446dc93 docs: redirect /docs/guides/helm/
This shows up in the Unity tests I'm working on with mvdan and goes to a
blank page without the redirect in place.

	--- FAIL: TestGuides_v1alpha5 (0.00s)
	   --- FAIL: TestGuides_v1alpha5/helm (0.60s)
	       testscript.go:584: # Helm Guide https://holos.run/docs/guides/helm/
2024-12-02 09:05:13 -08:00
Jeff McCune
67ef990c37 v0.101.2 build tags 2024-12-02 08:09:23 -08:00
Jeff McCune
6bd54ab856 render: pass build tags from platform to component (#366)
Previously, build tags were not propagated from `holos render platform
-t validate` through to the underlying `holos render component` command.
This is a problem because validators need to be selectively enabled as a
work around until we have an audit mode field.

This patch fixes the problem by propagating command line tags from the
render platform command to the underlying commands.  This patch also
propagates tags for the show command.
2024-11-30 20:56:11 -08:00
Jeff McCune
89a23a10fd docs: remove DIRECTORY from holos render platform --help
The directory argument is deprecated now, use the --platform flag
instead.
2024-11-30 13:08:01 -08:00
Jeff McCune
5a939bb6fe render: support cue build tags e.g. -t foo for @if(foo) (#366)
Previously Holos only supported tags in the form of key=value.  CUE
supports boolean style tags in the form of `key [ "=" value ]` which we
want to use to conditionally use to register components with the
platform.

This patch modifies the flag parsing to support -t foo like cue does,
for use with the @if(foo) build tag.
2024-11-30 12:50:31 -08:00
Jeff McCune
ee16f14e03 refactor build plan pipeline
Previously the BuildPlan pipeline didn't execute generators and
transformers concurrently.  All steps were sequentially executed.  Holos
was primarily concurrent by executing multiple BuildPlans at once.

This patch changes the Build implementation for each BuildPlan to
execute a GoRoutine pipeline.  One producer fans out to a group of
routines each executing the pipeline for one artifact in the build plan.
The pipeline has 3 stages:

1: Fan-out to build each Generator concurrently.
2: Fan-in to build each Transformer sequentially.
3: Fan-out again to run each validator concurrently.

When the artifact pipelines return, the producer closes the tasks
channel causing the worker tasks to return.

Note the overall runtime for 8 BuildPlans is roughly equivalent to
previously at 160ms with --concurrency=8 on my M3 Max.  I expect this to
perform better than previously when multiple artifacts are rendered for
each BuildPlan.
2024-11-29 14:52:06 -08:00
Jeff McCune
7530345620 main: add tracing and profiling
Writes files based on parent pid and process pid to avoid collisions.

Analyze with:

export HOLOS_TRACE=trace.%d.%d.out
go tool trace trace.999.1000.out

export HOLOS_CPU_PROFILE=cpu.%d.%d.prof
go tool pprof cpu.999.1000.prof

export HOLOS_MEM_PROFILE=mem.%d.%d.prof
go tool pprof mem.999.1000.prof
2024-11-29 14:52:06 -08:00
Jeff McCune
47d60ef86d docs: fix cue vet path in validators post (#357)
Without this patch the validator fails if a component manages two of the
same kind of resource, which is common.

This patch updates the example to use the metadata namespace and name as
lookup keys.  This works for most components, but may not for
ClusterResources.  Use the kind top level field in that case and pass
the field name of the validator as a tag value to vary by component.
2024-11-25 19:35:02 -08:00
Jeff McCune
9e9f6efd04 docs: fix typos in validators blog post (#357) 2024-11-25 15:46:54 -08:00
Jeff McCune
fb4a043823 docs: add card for validators blog post (#357) 2024-11-25 15:11:11 -08:00
Jeff McCune
d718ab1910 docs: redirect /docs/local-cluster to the v1alpha5 topic 2024-11-25 10:53:31 -08:00
Jeff McCune
c649db18a9 docs: redirect /docs/quickstart to the overview 2024-11-25 10:43:16 -08:00
Jeff McCune
b3bddf3ee3 docs: add validators blog post (#357) 2024-11-25 08:49:27 -08:00
Jeff McCune
77836be250 docs: update readme 2024-11-25 08:10:11 -08:00
Jeff McCune
4db670b854 docs: add validators tutorial (#357)
Add a tutorial page on validators.
2024-11-24 21:34:09 -08:00
Jeff McCune
d87c919519 docs: redirect /docs/topics to structures
/docs/v1alpha5/api/author/ links to it in the opening paragraphs.
2024-11-24 19:38:30 -08:00
Jeff McCune
2184bda2a1 v1alpha5: add validators (#357) 2024-11-24 17:40:41 -08:00
Jeff McCune
a8ab4dabaa cue: fix holos cue vet exit code (#358)
Without this patch `holos cue vet` always returns exit code 0, even when
there are errors.

This patch fixes the problem by catching the error and returning it to
our own top level error handler.  Note the final error, "could not run:
terminating because of errors" which wraps the generic error reported by
cue in the presence of multiple errors.

Result:

```
❯ holos cue vet ./policy --path 'strings.ToLower(kind)' /tmp/podinfo.gen.yaml
deployment.kind: conflicting values "Forbidden" and "Deployment":
    ./policy/validations.cue:18:8
    ../../../../../tmp/podinfo.gen.yaml:25:7
deployment.spec.template.spec.containers.0.resources.limits: conflicting values null and {[string]:"k8s.io/apimachinery/pkg/api/resource".#Quantity} (mismatched types null and struct):
    ./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:355:9
    ./cue.mod/gen/k8s.io/api/apps/v1/types_go_gen.cue:376:12
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2840:11
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:2968:14
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:15
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:3882:18
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:5027:9
    ./cue.mod/gen/k8s.io/api/core/v1/types_go_gen.cue:6407:16
    ./policy/validations.cue:17:13
    ../../../../../tmp/podinfo.gen.yaml:104:19
could not run: terminating because of errors
```
2024-11-24 16:39:56 -08:00
Jeff McCune
7175950ce0 docs: redirect /docs/overview permalink 2024-11-24 08:58:08 -08:00
Jeff McCune
e186c1be37 docs: edits for grammar 2024-11-21 15:55:14 -08:00
Jeff McCune
a998513e34 docs: edit for grammar and make concise 2024-11-21 15:26:20 -08:00
Jeff McCune
2f821ec33c docs: add environments topic page (#354)
Similar to the Clusters topic, add a topic about configuring multiple
environments.  This likely needs some work, the example is a bit
contrivied but at least shows how we can look up attributes, then use
those attributes to look up additional configuration from platform-wide
configuration data.
2024-11-21 15:07:46 -08:00
Jeff McCune
4d54190f2e docs: remove empty topic pages
We'll add them back in as we write them.
2024-11-21 14:40:39 -08:00
Gary Larizza
b466ec3457 Merge pull request #353 from holos-run/gl/fix-platform-args
Remove deprecated argument to 'holos render platform'
2024-11-21 13:04:17 -08:00
Gary Larizza
45120797b9 Remove deprecated argument to 'holos render platform'
This commit removes the extra `./platform` argument from any of the
current tutorial/topic docs to reflect the change that was made to
`holos render platform` in version `0.100.0`.
2024-11-21 11:49:44 -08:00
Jeff McCune
e6d25bf5eb holos: improve OrderedEncoder error handling
If someone accidentally provides the same index multiple times, or
indexes less than the next expected, the program would silently discard
the data.  This would be difficult to troubleshoot since an
OrderedEncoder is usually used with concurrent go routines, which would
likely mislead the investigator.

Better to just fail hard with an error indicating the caller in these
situations.
2024-11-20 17:02:16 -08:00
Jeff McCune
3ad6e69336 docs: update report issue template
Show how we can copy and paste directly from the issue into testscript.
2024-11-20 15:58:02 -08:00
Gary Larizza
9b10e23e43 Redirect docs to current version (#352)
* Redirect docs to current version

PROBLEM:

See #350 for context. There's a GitHub issue open for this on the
facebook docusaurus repo:
https://github.com/facebook/docusaurus/issues/9049

SOLUTION:

Use the redirect workaround for the time being.

OUTCOME:

https://holos.run/docs will link to the most recent version of our docs
site.

* Update doc/website/static/_redirects

Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>

---------

Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>
Co-authored-by: Jeff McCune <jeff@openinfrastructure.co>
2024-11-20 15:33:14 -08:00
Jeff McCune
b19022d3ff tests: fill in the buildplan.yaml for better coverage (#348)
Assert against the complete build plan so we know if we change the
output format in the future.

It's easy to update if so:

  HOLOS_UPDATE_SCRIPTS=1 go test github.com/holos-run/holos/cmd/holos
2024-11-20 15:21:41 -08:00
Jeff McCune
3fd06f594b schemas: fix kustomize patch name field (#348)
Without this patch trying to use a Kustomize patch with the optional
name field omitted results in the following error:

  could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
  holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
      $WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2

This patch fixes the problem by providing a default value for the name
field matching the Go zero value for a string.
2024-11-20 15:08:44 -08:00
Jeff McCune
a75338f21c docs: add bug report issue template, disable dev deploy 2024-11-20 14:50:25 -08:00
Jeff McCune
6002040360 version 0.100.0 2024-11-20 11:34:04 -08:00
Jeff McCune
864d7d442b build: pass platform component labels and annotations to BuildPlan
Without this patch the BuildPlan resulting from a Platform that has
components with labels and annotations does not have the labels or
annotations of the source component.

Holos should copy the labels and annotations defined on each of the
Platform.spec.components to the resulting BuildPlan so end users can see
clearly where a BuildPlan originated from, and filter with selectors the
intermediate output BuildPlan the same way we filter with selectors the
original Platform spec components list.

Result:

```
holos init platform v1alpha5 --force
holos show buildplans  | head
```

```yaml
kind: BuildPlan
apiVersion: v1alpha5
metadata:
  name: podinfo
  labels:
    app.holos.run/cluster: local
    app.holos.run/name: podinfo
  annotations:
    app.holos.run/description: podinfo for cluster local
```
2024-11-20 11:34:03 -08:00
Jeff McCune
791c0a9ffd util: fix misleading RunCmd error message source location
The error message misleads the reader to the utility function.  It
should lead to the caller.
2024-11-20 11:23:54 -08:00
Jeff McCune
bfe4a8d7c4 remove unmaintained embedded platforms
v1alpha5 is current and maintained.
2024-11-20 11:23:54 -08:00
Jeff McCune
eaa508a6f8 remove v1alpha1 schemas and support 2024-11-20 11:23:53 -08:00
Jeff McCune
63256a2845 remove embedded k3d platform
No longer used, v1alpha5 is used in the docs and maintained.
2024-11-20 11:23:53 -08:00
Jeff McCune
937c1dc953 show: fix inconsistent ordering of output
Without this patch the holos show buildplans command returns results in
an inconsistent order.  This is a problem because the output should be
idempotent.

This patch fixes the problem by adding an EncodeSeq(idx int, v any) method to
the encoder interface.  idx represents the index position of the
Platform.spec.components list after selector filtering has been applied.

This patch modifies the json and yaml encoders to buffer out of order
results from the concurrent go routines.

Result:

Concurrent execution is preserved. The buffer is kept to a reasonable
size, entries are deleted once they're encoded in the correct order.

Most importantly the output is consistent and idempotent so we can write
effective integration tests.
2024-11-20 11:23:52 -08:00
Jeff McCune
11bd50e2eb show: fix buildplans selector inconsistency
Sometimes, but not always, the holos show buildplans command produces no
output.

```
❯ holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug
finalized config from flags
rendered platform in 13.458µs
```

It only happens when there's a selector.  It doesn't happen without the
selector flag.  It only happens with ==, not with =.

This test fails quickly.

```
while [[ $(holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug | wc -l) -eq 39 ]]; do true; done
```

This test runs until killed.

```
while [[ $(holos show buildplans --log-level=debug | wc -l) -eq 279 ]]; do true; done
```

Solution:

The problem is the use of the map.  Iterating over the keys happens in a
random order.  With the fix we check in an explicit order.
2024-11-20 11:23:52 -08:00
Jeff McCune
7cfcf55565 add yaml struct tags to core v1alpha5 schemas
Without this patch the `holos show buildplans` BuildPlan output has
incorrect yaml with the v3 encoder.  For example apiversion: v1alpha5
instead of apiVersion.
2024-11-20 11:23:52 -08:00
Jeff McCune
8b22ba04e1 cli: add show command and refactor interfaces (#331)
Show subcommand:

This is large change that accomplishes a number of goals.  First, there
was no convenient way to show a build plan without using the debug logs
to indentify the tags to inject, then calling the cue command with the
right incantation to inspect the BuildPlan.

This patch addresses the problem by adding a `holos show buildplans`
command.  The command loads the Platform spec from the platform
directory, then iterates over all Components to produce the BuildPlan.

This patch adds labels and annotations to the platform Components
collection in order to select and filter the output.

Result:

```
❯ holos show components --selector app.holos.run/cluster=local --format=yaml | head
kind: BuildPlan
apiversion: v1alpha5
metadata:
  name: podinfo
spec:
  artifacts:
    - artifact: clusters/local/components/podinfo/podinfo.gen.yaml
      generators:
        - kind: Helm
          output: helm.gen.yaml
```

---

Interface refactor:

This refactors the interface between the `holos` Go CLI layer and the
various core schema data structures.  We now use a proper Go interface.
Concurrent execution over platform components has been improved to
accept a closure function so we can use the same interface method to
process the components.  We use this to show each component and render
each component from different subcommands using the same interface
embedded in the builder.Platform struct.

The embedded interface allows us to easily swap in different versions,
e.g. v1beta1 and eventually v1.  The number of interface methods are
quite small.  14 methods across 4 interfaces in holos/interface.go.

---

Remove old versions:

This patch removes support for versions prior to v1alpha5 in an effort
to clean up cruft.
2024-11-20 11:23:51 -08:00
Nate McCurdy
7d5187873b docs: Remove non-existent CLI completions, add pwsh
Cobra provides completions for bash, zsh, fish, and powershell, not
ksh.
2024-11-18 10:27:33 -08:00
Jeff McCune
03b796312a cli: gate grpc client and auth flags behind feature flag
Previously the holos render platform and component subcommands had flags
for oidc authentication and client access to the gRPC service.  These
flags aren't currently used, they're remnants from the json powered form
prototype.

This patch gates the flags behind a feature flag which is disabled by
default.

Result:

  holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int   number of components to render concurrently (default 8)
  -v, --version           version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")

---

  HOLOS_FEATURE_CLIENT=1 holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int             number of components to render concurrently (default 8)
      --oidc-client-id string       oidc client id. (default "270319630705329162@holos_platform")
      --oidc-extra-scopes strings   optional oidc scopes
      --oidc-force-refresh          force refresh
      --oidc-issuer string          oidc token issuer url. (default "https://login.holos.run")
      --oidc-scopes strings         required oidc scopes (default openid,email,profile,groups,offline_access)
      --server string               server to connect to (default "https://app.holos.run:443")
  -v, --version                     version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")
2024-11-17 16:06:57 -08:00
Jeff McCune
20fb39e49b docs: add clusters topic (#343)
Previously we didn't have good documentation on how to manage multiple
sets of clusters.

This patch adds a clusters topic in the structures category.  Each one
of the environments, projects, owners, etc... structures follow the same
pattern as #Clusters and #ClusterSets, so it makes sense to put them
into a dedicated sidebar category for specific CUE structures.
2024-11-17 14:45:32 -08:00
Jeff McCune
c9c8c13810 docs: replace touch with cat 2024-11-15 09:54:09 -07:00
Jeff McCune
374cd872e9 docs: CUE not cue and typo fix in hello holos 2024-11-15 09:40:02 -07:00
Jeff McCune
8db06dd0e1 releaser: fix brew test command (#327)
holos version isn't a valid command but holos --version is.
2024-11-14 16:14:26 -07:00
Jeff McCune
66acadf86d docs: support brew install (#327) 2024-11-14 16:07:13 -07:00
Jeff McCune
032f72b435 render: log helm pull errors (#332)
Previously errors were not logged, giving no indication what went wrong.
This patch changes the error handler to log errors from helm.
2024-11-14 09:44:27 -07:00
Jeff McCune
2380223794 docs: add argocd application example (#340)
When we moved from v1alpha4 to v1alpha5 we removed ArgoConfig from the
author schema.  There was no longer a clear example of how to configure
an ArgoCD Application for every component in v1alpha5.

This patch adds a topic document with an example of how to add an
Application along side the resources by mixing an additional Artifact
into the BuildPlan.
2024-11-13 16:30:59 -07:00
Jeff McCune
e6892c3b16 v0.99.0 2024-11-13 12:49:28 -07:00
Jeff McCune
847fd2958e helm: add support for helm template --kube-version capabilities (#330)
Previously the Helm generator had no support for the --kube-version
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:43:01 -07:00
Jeff McCune
cf622835db helm: add support for helm template --api-versions capabilities (#330)
Previously the Helm generator had no support for the --api-versions
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:42:50 -07:00
Jeff McCune
1f5dc3a082 docs: add note about tested helm version (#335)
To help users understand what should definitely work.
2024-11-13 09:45:56 -07:00
Jeff McCune
9f4da68dc9 v0.98.2 2024-11-13 09:19:30 -07:00
Jeff McCune
2ee056be9f cue: fix panic with no args (#334)
Fixes:

```
❯ holos
panic: runtime error: slice bounds out of range [2:1]

goroutine 1 [running]:
github.com/holos-run/holos/internal/cli.newCueCmd(...)
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:121
github.com/holos-run/holos/internal/cli.New(0xc0002837c0, {0x3826e00, 0x4f60860})
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:102 +0x772
main.main.MakeMain.func1()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/main.go:22 +0x5b
main.main()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/cmd/holos/main.go:10 +0x3e
```
2024-11-13 09:04:37 -07:00
Jeff McCune
394e2cb0b2 docs: add cue tutorial (#318)
Show how to use the ComponentConfig Resources field to mix in resources.
2024-11-13 08:00:37 -07:00
Jeff McCune
cf95c9664d docs: change hello holos parameters to greeting (#328)
Version doesn't make as much sense since we're doing a hello world
tutorial.

Also consolidate the values information into one step, and consolidate
the breaking it down section to make it shorter and clearer.
2024-11-12 09:46:19 -07:00
Jeff McCune
0192eeeb7e docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest

This time in the correct directory.
2024-11-11 17:25:17 -07:00
Jeff McCune
ed54bcc58f docs: rename cue-generator to cue
The main use case is to manage resources from CUE, but CUE has many uses
in Holos such as validation and driving Kustomize.
2024-11-11 17:16:53 -07:00
Jeff McCune
9ac7f185f9 docs: fix broken validators link in diagram 2024-11-11 17:11:35 -07:00
Jeff McCune
7de72d3dab docs: add component parameters example to hello holos (#328)
The important note was weird because we didn't show an example of how to
use component parameters.  This patch replaces the note with an example.
2024-11-11 16:56:16 -07:00
Jeff McCune
2e3c998454 docs: add directory tree to hello holos doc (#324)
Feedback from Zack, give a tree so people skimming know where to figure
out the lay of the land.
2024-11-11 16:19:48 -07:00
Jeff McCune
580afffa7f 0.98.1 - holos init platform 2024-11-11 14:44:00 -07:00
Jeff McCune
67535e1e1d doc: remove init platform from setup guide
We have it in the hello guide, setup should just be install only and not
how to use the tool yet.
2024-11-11 14:41:58 -07:00
Nate McCurdy
767ea69d2e docs: Add a tree view to Hello Holos
A tree view of the `holos-tutorial/` directory should give readers a
quick, high-level understanding of the folder structure of a typical
Holos platform project.
2024-11-11 14:04:40 -07:00
Jeff McCune
21e1a116e4 cli: hide help flags and command (#325)
They're unnecessary.
2024-11-11 14:02:25 -07:00
Jeff McCune
65fe7779be cli: rename generate to init (#325)
This patch changes the `holos generate` command to `holos init` to match
other tools like `go mod init`.
2024-11-11 14:02:25 -07:00
Jeff McCune
0e7abf0173 docs: consolidate diagrams to @site/src/diagrams/
So we don't have two different copies in two different places.
2024-11-11 13:40:46 -07:00
Jeff McCune
cca022ac99 docs: move architecture diagrams (#323) 2024-11-11 12:03:21 -07:00
Jeff McCune
43e939d06a doc: refactor breaking it down table in hello holos
So it displays nicely on mobile.
2024-11-09 14:52:11 -08:00
Jeff McCune
8096268826 docs: fix diagram urls again 2024-11-09 14:40:08 -08:00
Jeff McCune
631b23091d docs: fix rendering overview diagram on blog 2024-11-09 14:33:54 -08:00
Jeff McCune
09c6476282 docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-09 14:30:44 -08:00
Jeff McCune
a768d16c5f docs: set current version to v1alpha5
Previously the current version would always be unreleased at /docs/next
and we'd have to copy the doc/md/ folder into the
doc/website/versioned_docs/version-v1alpha5/ every time we made a
change.

We're going to be working on v1alpha5 for awhile so it makes sense to
just have the current version published at /docs/v1alpha5/ and we can
start /docs/v1alpha6/ whenever we're ready.

This also has the nice effect of giving us permalinks if we change the
structure again.  /docs/v1alpha5/ will remain over time.
2024-11-09 14:29:27 -08:00
Jeff McCune
3834a7ef85 docs: add missing link to kustomize tutorial 2024-11-08 22:25:34 -08:00
Jeff McCune
606a1aae73 docs: add nav bar title back 2024-11-08 19:29:05 -08:00
Jeff McCune
340d07ee7a docs: fix announcing holos blog (#321) 2024-11-08 19:24:46 -08:00
Jeff McCune
12d2cec4d5 docs: fix rendering overview diagram links (#321) 2024-11-08 19:12:23 -08:00
Jeff McCune
e93feb49b7 docs: add version drop down to nav bar (#321) 2024-11-08 17:05:37 -08:00
Jeff McCune
dcf8602a0b docs: release v1alpha5 (#321) 2024-11-08 17:00:34 -08:00
Jeff McCune
e07c4d11c8 docs: revise helm values and kustomize tutorials (#316)
These are now where I'd like them to be.
2024-11-08 15:12:22 -08:00
Jeff McCune
b7e1c14192 docs: kustomize tutorial (#316)
Add httpbin using kustomize and patch the result, all from CUE.  The is
the second half of the v1alpha4 helm guide split into a dedicated
tutorial.
2024-11-08 14:08:48 -08:00
Jeff McCune
29f44cdac9 docs: helm values (#316)
Add a helm values tutorial which is a cut down version of the v1alpha4
helm guide.  The httpbin kustomize will immediately follow building on
the prometheus and blackbox charts.
2024-11-08 12:03:14 -08:00
Jeff McCune
96be7a4ae3 docs: add generate platform to hello holos (#311) 2024-11-07 17:59:07 -08:00
Jeff McCune
d6bd030a72 docs: fixup overview 2024-11-07 17:59:07 -08:00
Jeff McCune
75047b590f docs: hello holos edits (#311) 2024-11-07 17:19:53 -08:00
Gary Larizza
a05881df0f Add the Hello Holos tutorial 2024-11-07 16:47:36 -08:00
Jeff McCune
5f406fce5c docs: organize docs for new structure (#301) 2024-11-07 15:10:15 -08:00
Jeff McCune
49c945a037 docs: setup tutorial with diagrams (#301) 2024-11-07 14:13:49 -08:00
Gary Larizza
54de20f0b8 docs: setup tutorial (#301) 2024-11-07 10:24:49 -08:00
Gary Larizza
80b4ab9852 Merge pull request #310 from holos-run/jeff/308-holos-field
docs: rewrite technical overview for v1alpha5
2024-11-07 10:15:01 -08:00
Jeff McCune
acd98aa63c docs: rewrite technical overview for v1alpha5
Attribution: following the structure and length of the tokio docs, with
some more diagrams.
2024-11-07 10:07:45 -08:00
Jeff McCune
0afaab8f2b render: nest output under the holos top level field (#308)
Previously the holos command line expected a Platform and BuildPlan
resource at the top level of the exported data from CUE.  This forced us
to use hidden fields for everything else.

This patch modifies the BuildData struct to first look for a holos top
level field and use it if present.  This opens up other top level fields
for use by end users.

Our intent is to reserve any top level field prefixed with holos.

Note this follows how Timoni works as well.
2024-11-07 07:00:26 -08:00
Jeff McCune
7ded38bc3f v1alpha5: strip down the core and author schemas (#306)
This patch strips down the v1alpha4 core and author schemas to only with
is absolutely necessary for all holos users.  Aspects of platform
configuration applicable to some, even most, but not all users will be
moved into documentation topics organized as a recipe book.

The functionality removed from the v1alpha4 author schemas in v1alpha5
will move into self contained examples documented as topics on the docs
site.

The overall purpose is to have a focused, composeable, maintainable
author schema to help people get started and ideally we can support for
years with making breaking changes.

With this patch the v1alpha5 helm guide test passes.  We're not going to
have this guide anymore but it demonstrates we're back to where we were
with v1alpha4.
2024-11-06 15:22:17 -08:00
Jeff McCune
840676709a docs: partially write the overview doc for v1alpha5
Partial, switching gears to v1alpha5 to unblock others working on
discrete topics.
2024-11-05 10:19:59 -08:00
Jeff McCune
ee30c52673 docs: generate version specific api docs (#303)
Without this patch each version of the core and author schemas are
duplicated into each docs version.  This is unnecessary and difficult to
maintain now that we have docusaurus versioned docs enabled.

This patch updates the schema generation script to check if the docs
version has been released, and if so write into a markdown file in the
versioned docs folder.  If not, the version is written into the next
version folder.

This patch also updates some, but not all, document links to the md or
mdx relative file paths.  This is necessary to generate the correct
versioned links.

A nice outcome of this change is that technical docs no longer need to
link to version specific pages.  For example, `[Core Schema]:
./api/core.md` will always refer to the correct auto generated docs
associated with the docs version.
2024-11-05 07:20:53 -08:00
Jeff McCune
117a00334f docs: restructure docs into tutorial and topics (#301)
The docs for v1alpha4 have the right information, but in the wrong
places.  The most important bits are tucked away in the Core API docs.
One of our first users entirely missed the `holos generate platform`
command mentioned in the Helm guide.

We'll fix this by organizing the docs into two distinct categories.
First, a tutorial written as a series progressively building up the
minimum knowledge to use holos effectively and gain the benefits.  Think
of it as a tour of the essential bits.

The second category are focused topics which stand alone.  They're the
things most people using holos will need to know eventually, but aren't
essential for everyone to know.  For example, Clusters and Fleets will
move from the Author API to stand alone examples of how to implement
these features if necessary.

Then there's a Glossary which serves as the place to describe our
concepts and domain specific language.

Finally there's the API documentation which should be cut down to the
specific version.  The next release version will be v1alpha5.

Attribution: We're copying the Tokio docs structure, it's concise and a
similar size and complexity to our own project.

The Go docs are also an inspiration, but the project is much larger so
not directly comparable.  The organization of https://go.dev/doc/ feels
complete at first glance, despite the size and age of the project.  The
site also makes clear who each section is for without needing to come
right out and say it. Getting started, Using and understanding Go,
Writing modules, using databases, etc...
2024-11-04 20:25:04 -08:00
Jeff McCune
1e03debfac tests: add make unity target
For https://cuelabs.dev/unity/
2024-11-04 19:08:44 -08:00
Jeff McCune
72137b2fa9 docs: upgrade docusaurus to 3.6.0
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-04 06:47:48 -08:00
Jeff McCune
5abf967116 docs: npm run docusaurus docs:version v1alpha4 (#299)
Tag version v1alpha4 so we can start working on v1alpha5 as the next
version in main.
2024-11-04 06:43:15 -08:00
Jeff McCune
5d882f465d website: fix resources.yaml tab in helm guide (#293)
We switched from using a kustomize remote base to a local file so the
tests don't need to make a network round trip to github.  It's also
better practice to use local files for this sort of thing.

In doing so I botched the location of the file, putting it in the
platform registration section.  This patch claifies how `resources.yaml`
is linked to `httpbin.cue` through the `KustomizeConfig: Files:
"resources.yaml": _` field.
2024-11-03 10:57:52 -08:00
Jeff McCune
45bdaac833 main: cue v0.10.1 and add e2e test for helm guide (#293)
Previously there was no test coverage of the
https://holos.run/docs/guides/helm/ guide.  This patch uses Roger's
testscript package, which the CUE folks also use to add comprehensive
test coverage of each step in the guide.  Ideally we would execute these
commands directly from the guide itself, but for now we'll duplicate the
commands into the test script.  This could be enhanced by generating the
test script from the document itself in some way.

When updating the script, use the `holos txtar` command to embed entire
helm charts into the test script.  It's not super fast, but it's better
than network access and it's not terribly slow either.  A few seconds to
unpack.

---

txtar: quote files for testscript unquote

For the helm guide test script we want to include the entire helm chart
which may have files that need to be quoted.  This patch changes the
default behavior of the holos txtar command to quote files if necessary
and list them in an unquote script command in the comment of the
archive.

The purpose is for testscript authors to copy and paste the entire thing
into a test script and include the unquote command at the top.

---

This change also updates CUE to v0.10.1
2024-11-03 10:27:46 -08:00
Jeff McCune
7ae1f990ef website: update quickstart diagram to match helm
Avoid confusion, got a question about this in discord.
2024-11-03 08:53:43 -08:00
Jeff McCune
b526fd1669 testdata: clean up old v1alpha1 tests (#292)
No longer necessary now that we're on v1alpha4.  Test coverage for
v1alpha4 and the user facing guides will be added back soon for use both
in the holos repo and in Unity.
2024-11-01 15:22:13 -07:00
Jeff McCune
5e07655f35 website: fix port in helm guide
Should be 9115 not 6115.
2024-11-01 06:39:48 -07:00
Jeff McCune
6fb6afe8d5 v0.97.3 2024-10-31 21:04:22 -07:00
Jeff McCune
d6f89052d9 website: update helm guide to apply patches (#291)
Updated the helm guide to apply patches while still showing the diff in
the documentation markdown.  The only gotcha is it creates orig files.
2024-10-31 20:54:57 -07:00
Jeff McCune
e4aa7f5994 website: update change-a-service to use hidden fields (#291)
Use _Foo instead of #Foo to hold concrete values.
2024-10-31 20:25:40 -07:00
Jeff McCune
6e4c65cb6c website: update deploy-a-service to use hidden fields (#291)
Use _Foo instead of #Foo to hold concrete values.
2024-10-31 20:13:17 -07:00
Jeff McCune
4f091677e2 website: update quickstart for v1alpha4 hidden fields (#291) 2024-10-31 16:35:41 -07:00
Jeff McCune
0c05df1162 website: update technical overview with consistent fields (#291) 2024-10-31 11:30:20 -07:00
Jeff McCune
64a745fd34 v1alpha4: use hidden fields consistently (#291)
Previously it wasn't clear for users if platform wide structs should be
definitions or hidden fields in CUE.  They should be hidden fields when
they contain data and definitions when they define a schema.

This patch updates the generate platform v1alpha4 subcommand to use the
correct field names consistently for clarity.
2024-10-31 10:45:47 -07:00
Jeff McCune
490f91f580 cli: hide unsupported commands (#289)
Use a simple feature flag system that checks env vars if a feature is
enabled.
2024-10-31 10:04:01 -07:00
Jeff McCune
79b065cda8 website: add open graph image for helm guide try 6 2024-10-30 12:23:18 -07:00
Jeff McCune
0fa6047552 website: add open graph image for helm guide try 5 2024-10-30 12:20:19 -07:00
Jeff McCune
11ecc0cc3a website: add open graph image for helm guide try 4 2024-10-30 12:04:39 -07:00
Jeff McCune
a62e4ba117 website: add open graph image for helm guide try 3 2024-10-30 11:56:01 -07:00
Jeff McCune
07fe667f30 website: add open graph image for helm guide try 2 2024-10-30 11:40:39 -07:00
Jeff McCune
3ad994cbb9 website: add open graph image for helm guide 2024-10-30 11:26:54 -07:00
Jeff McCune
b3d9bd32af website: add why cue for configuration blog post
This is going to be one of the first questions we get.
2024-10-28 21:30:11 -07:00
Jeff McCune
d398b49d7f website: fix head title tag try 2
The open graph title was still showing up poorly, docusaurus generates
it with the Holos | Holos repetition, so we need to override it.
2024-10-28 14:37:43 -07:00
Jeff McCune
12179a6991 website: fix head title tag and social card
Generate the social card manually from https://www.opengraph.xyz/
Override the page title tag, otherwise it shows up as "Announcing Holos
| Holos" in social links, which is weird.
2024-10-28 14:07:40 -07:00
Jeff McCune
fee472bb66 website: add stock social card for annoucement 2024-10-28 13:31:10 -07:00
Jeff McCune
c6a13059f3 v0.97.1 2024-10-28 11:12:55 -07:00
Jeff McCune
ff3eb896f3 webite: put ois logo back
Until we get a better logo.
2024-10-28 10:46:59 -07:00
Jeff McCune
70f70ae6b9 website: fix launch announcement 2024-10-28 10:45:43 -07:00
Jeff McCune
2580ec1c5f website: fix order of api references
The api references are in reverse order and don't have good descriptions
in the index listings.  This patch adds front matter to each generated
document to order them correctly and add a nice description.
2024-10-27 20:43:54 -07:00
Jeff McCune
4fa99e0faa website: add helm prometheus blackbox httpbin guide
The purpose of this guide is to demonstrate how holos offers value
above and beyond helm and kustomize alone.
2024-10-27 19:48:42 -07:00
Jeff McCune
7341d25483 website: add at proto did for bsky 2024-10-25 14:26:42 -07:00
Jeff McCune
3074b3a241 website: add discord invite link 2024-10-24 10:27:37 -07:00
Jeff McCune
9a5e7869c6 v1alpha4: omit the platform model if empty
The platform model distracts from getting started:

  cue export --out yaml ./platform

  kind: Platform
  apiVersion: v1alpha4
  metadata:
    name: default
  spec:
    components:
      - name: prometheus
        component: projects/platform/components/prometheus
        cluster: local
        model: {}

With this patch it's absent by default.
2024-10-23 13:46:58 -07:00
Jeff McCune
1064ceba31 v1alpha4: manage a single workload cluster named local by default
Manage a single cluster by default after generating the platform.
Assume it's a local cluster for use with the guides.
2024-10-23 13:42:46 -07:00
Jeff McCune
4bccaa3710 v1alpha4: _Platform not #Platform for component registration
`_Platform` is a hidden field representing the platform components,
`#Platform` defines the schema of the hidden field.
2024-10-23 13:33:28 -07:00
Jeff McCune
95efae1343 docs: update rendered manifest figure in technical overview
It's too wide with KubeAPI, take it out.
2024-10-20 10:11:57 -07:00
Jeff McCune
ba88125877 v1alpha4: enable config map generator for Kustomize
Without this patch it's difficult to mix in a plain file as a config
map.  This is necessary for the use case of using a Job to generate a
secret in-cluster.  We want a plain shell script to be carried through
and transformed into the job.

We already have the KustomizeConfig fields to support this, they just
weren't wired up to the #Kustomization component kind.

I didn't check if it's wired up to Helm and Kustomize for expedience.
They may be missing there as well.
2024-10-19 10:57:37 -07:00
Jeff McCune
d12c1a0c11 Merge pull request #284 from holos-run/gl/deploy-a-service-v4alpha-update
Update deploy-a-service guide for Author API v1alpha4
2024-10-18 20:35:51 -07:00
Jeff McCune
d56d3400a7 docs: replace tabs in deploy-a-service guide 2024-10-18 20:33:54 -07:00
Gary Larizza
4f0f9dced5 Update deploy-a-service guide for Author API v1alpha4
PROBLEM:

Version v1alpha4 of the Author API has been updated with backwards
incompatible changes, and the deploy-a-service guide uses code from
version v1alpha3.

SOLUTION:

Update any code, links, and data that is out of date, and then run
through the guide to make sure it works locally.

OUTCOME:

The instructions in the deploy-a-service guide will work successfully
with version v1alpha4 of the Author API.
2024-10-18 15:21:56 -07:00
Jeff McCune
6bf0cb8d8e v1alpha4: v0.97.0 2024-10-17 07:50:49 -07:00
Jeff McCune
766c8912b7 Merge pull request #281 from holos-run/jeff/280-authorapi-v1alpha4
v1alpha4 Author API
2024-10-17 07:17:14 -07:00
Jeff McCune
be1dee5f1c v1alpha4: update technical overview guide (#280)
Update the guide and the bank of holos repository to use v1alpha4
instead of v1alpha3.
2024-10-17 07:13:51 -07:00
Jeff McCune
6ad56525ac v1alpha4: refactor --tag to --inject and remove environment (#276)
Cue uses --inject, -t as the flags to set variables for fields tagged
using @tag(var,type=string).

We used --tag, which is different and requires a mental mapping.  Let's
use the same flag and also pass it multiple times like they require so
we can copy and paste the command line output from the debug logs into a
cue export command to see what's going on.

This patch deprecates the --cluster-name flag, use --inject
holos_cluster=mycluster instead.

This patch also removes the environment field from the Component core
API, leaving this to the user namespace to define via tags.  We don't
want to be too opinionated on how users manage their platform, baking
environment into the schema is a slippery slope toward those kinds of
opinions.

Closes: #276
2024-10-16 22:07:47 -07:00
Jeff McCune
791ec5ee71 v1alpha4: refactor core.Component Tags to map[string]string (#280)
Previously it was a []string slice that must be formatted as key=value.
This is more difficult to work with than a map[string]string.
2024-10-16 20:10:14 -07:00
Jeff McCune
638ac7473c fixup 2024-10-16 20:05:05 -07:00
Jeff McCune
ee24b5ce13 fixup 2024-10-16 20:04:18 -07:00
Jeff McCune
fa2fdbe4e8 fixup 2024-10-16 20:00:53 -07:00
Jeff McCune
63e1df1d4c v1alpha4: add common labels to projects schema (#280)
Now that we have CommonLabels as part of the ComponentConfig for all
components, it makes sense to also mix in CommonLabels for a Project.

Common labes are key aspect of the Technical Overview document.
2024-10-16 17:40:40 -07:00
Jeff McCune
2ad0c2a93e render: refactor tm to typeMeta
Gary and I noted tm wasn't clear when I was showing him code.
2024-10-16 17:11:09 -07:00
Jeff McCune
3a6a04f318 v1alpha4: add projects to author api (#280)
Projects are a key element of the Technical Overview guide, so we need
the schema for them in the Author API.
2024-10-16 12:29:35 -07:00
Jeff McCune
8afeece890 v1alpha4: embed ComponentConfig in Helm, Kustomize, Kubernetes (#280)
For the Author API, it would be nice to define a schema for the fields
common to all component kinds.  Users could then configure all kinds by
unifying the schema into their own platform tree.

This makes a clear use case to extract the common fields back into an
embedded struct like we did in v1alpha3.  I removed the embedded struct
in v1alpha4 because it wasn't clear why it should be separate, but now
the use case is clear, to configure all component kinds.
2024-10-16 12:16:48 -07:00
Jeff McCune
bc9c43a0b9 fix argocd application project 2024-10-15 20:52:56 -07:00
Jeff McCune
5a98c77e4c add argocd.argoproj.io/instance label to resources
But not the ArgoCD Application resource.
2024-10-15 20:41:39 -07:00
Jeff McCune
b3f7de39ec v1alpha4: feedback in case of chart cache dead lock (#280)
Without this patch holos render platform may hang until the overall
context timeout is reached.  This is a problem because the user has no
idea why it's hung.

This patch adds a warning at the 5 second and another at the 10 second
mark indicating the lock may be deadlocked.  The user can then remove
the directory.
2024-10-15 16:43:36 -07:00
Jeff McCune
ca4ecf1b28 v1alpha4: KustomizeConfig Resources and Files (#280)
The Kustomize build plan kind needs to support both copying files from
the component directory and pulling resources from https URL's.  Without
this patch this support is missing from the Author API

With this patch the Kustomize build plan kind has a KustomizeConfig
field with two structs, Files and Resources.  The kustomization
resources list is built up from both of these.

Two transformers are used so we don't affect the GitOps transfomer which
really only needs CommonLabels.

I decided to keep this field exclusive to the Kustomize kind, but it
could replace the Kustomization field of the other kinds as well.
2024-10-15 16:11:05 -07:00
Jeff McCune
9ce28660ce v1alpha4: intermediate kustomization (#280)
Without this patch the user facing API doesn't have a way to kustomize
the output of all the build plan kinds.  This patch ensures the
Kustomization field is present on all of Helm, Kustomize, and
Kubernetes.

This field is inteded for patches and transforms.  The second
kustomization in the transformer sequence is intended for common labels
and annotations, managed by a corresponding field instead of a full on
Kustomization resource.
2024-10-15 15:02:15 -07:00
Jeff McCune
728e8ba06e v1alpha4: default helm chart release to chart name (#280) 2024-10-15 14:47:42 -07:00
Jeff McCune
e4b07dad6d v1alpha4: helm enable hooks default false (#280)
Fix:

could not run: could not marshal json projects/platform/components/cert-manager: cue: marshal error: spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON at internal/builder/builder.go:63
spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON:
    /Users/jeff/Holos/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha4/types_go_gen.cue:235:16
could not run: could not render component: exit status 1 at builder/v1alpha4/builder.go:94
2024-10-15 14:43:05 -07:00
Jeff McCune
b7c0bba2b9 v1alpha4: add resources to schema.cue (#280)
Without this patch kustomize cannot execute because resources from CUE
are incomplete.
2024-10-15 14:41:49 -07:00
Jeff McCune
847ab8441c v1alpha4: add default chart name (#280)
Defaults to the build plan name.
2024-10-15 14:40:36 -07:00
Jeff McCune
5f72af3d53 v1alpha4: link helm to the chart struct (#280) 2024-10-15 14:32:16 -07:00
Jeff McCune
33eed43fd1 v1alpha4: surface kustomize stderr output (#280)
Without this patch kustomize errors aren't surfaced when executing holos
render platform.

This patch gives a fighting chance to the user to figure out what's
going on.  The stderr is copied, logged, and surfaced up to the parent
holos render platform command.
2024-10-15 14:32:11 -07:00
Jeff McCune
d2fbbdd1cc logger: log the process pid (#280)
Useful when troubleshooting concurrent processes.
2024-10-15 14:31:42 -07:00
Jeff McCune
e42da118dc v1alpha4: add Helm and Kustomize to author api (#280)
Previously the #Helm and #Kustomize build plan helpers were not defined
in the v1alpha4 Author API.  We need this definition to update the
Quickstart guide for v1alpha4 from v1alpha3.

This patch defines the #Helm and #Kustomize helpers in the Author API
similar to how #Kubernetes is defined.
2024-10-15 10:32:16 -07:00
Jeff McCune
7d36567dcf v1alpha4: define Kubernetes in author api (#280)
Previously #Kubernetes was defined in the platform code.  This is a
problem because every platform engineer would need to copy and paste
this code.

This patch moves the #Kubernetes helper into the cue.mod directory so it
can be imported and used ergonomically.
2024-10-14 20:45:04 -07:00
Jeff McCune
bee698bebe v1alpha4: add platform to the author api (#280)
This patch gets the Author API rendering the namespaces component in the
Bank of Holos guide.  It's not the final form of the API yet, we still
need to decide how best to expose the Kubernetes, Helm, and Kustomize
definitions.

I'm thinking we abstract away the transformers and generators within the
Author API Kubernetes definition.
2024-10-14 17:19:49 -07:00
Jeff McCune
58df0626d0 v1alpha4: plumb --write-to flag from platform (#280)
Without this patch the --write-to flag can't be controlled from the
PlatformSpec in the CoreAPI.  We need to surface this for the ArgoConfig
struct in the AuthorAPI.

That is to say, in v1alpha3 the --write-to flag was previously assumed
to be deploy/ in ArgoConfig using the DeployFiles functionality.  We no
longer have DeployFiles in Core API v1alpha4, all artifacts are instead
written relative to the --write-to flag.  Still, we need to expose this
flag in the PlatformSpec so users can use something other than the
deploy directory.
2024-10-14 15:16:36 -07:00
Jeff McCune
c817a24704 v1alpha4: improve core api documentation 2024-10-10 20:37:38 -07:00
Jeff McCune
bd56f3118c v0.96.0 - v1alpha4 Core API
Release v0.96.0 with the new Core API design.
2024-10-10 13:55:06 -07:00
Jeff McCune
d3aa748e92 v1alpha4: add file generator (#277)
Previously the file generator was unimplemented.  This patch implements
it as a simple file read into the ArtifactMap for use by the Kustomize
or Join transformers.

With this patch all v1alpha4 Core API features are implemented.
Resources, Helm, and File generators.  Kustomize and Join transformers.
2024-10-10 13:52:08 -07:00
Jeff McCune
6738248756 v1alpha4: fix blank log lines
Blank lines show up in the output which is confusing. This patch fixes
the only source location identified with the following command.

    export HOLOS_LOG_LEVEL=debug
    export HOLOS_LOG_FORMAT=json
    holos render platform ./platform 2>&1 | jq -r 'select (.msg == "")'
2024-10-10 13:44:08 -07:00
Jeff McCune
011b488775 v1alpha4: cache helm charts by version (#273)
Previously helm charts were cached only by name, which is a problem
because the wrong version would be used when previously cached.

This patch caches charts by name and version to ensure changes in the
version results in pulling the new cached version.  It is the user's
responsibility to remove old versions.

This patch also ensures only a single go routine can run cacheChart() at
a time across processes.  This is necessary when rendering a platform
because multiple processes will run the Helm generator concurrently, for
example when the same chart is used for multiple environments or
customers.

The mkdir system call serves as the locking mechanism, which is robust
and atomic on all commonly used file systems.
2024-10-10 12:15:20 -07:00
Jeff McCune
c8d89f3291 v1alpha4: add helm chart generator
Previously the helm generator was not implemented and returned an error.
This patch is a first pass copying the helm method from
internal/render/helm.go

Basic testing performed with a podinfo chart.  It works as the previous
versions in v1alpha3 and before works.  This patch does not address the
cached version issue in #273
2024-10-10 08:25:13 -07:00
Jeff McCune
a44ebe5171 refactor Artifact to use strings instead of FilePath
holos.FilePath is intended for paths relative to the platform root
directory.  We use the Artifact to store lots of stuff not related to
the platform root directory, for example kustomization.yaml in a temp
dir.  Most entries are not relative to the platform root directory given
the implicit cfg.WriteTo prefix.
2024-10-09 20:16:13 -07:00
Jeff McCune
66a3b6a874 improve error of missing key
Previously:

  could not run: could not build dev-join: could not get foo.yaml: not set at builder/v1alpha4/builder.go:180

This is confusing because set has nothing to do with the missing input
from the cue code the user writes.

Result:

  could not run: could not build test-join: missing foo.yaml at builder/v1alpha4/builder.go:180

This is better because it at doesn't distract the user from the fact
they're missing a foo.yaml generator output to align with the
transformer input.
2024-10-09 19:39:33 -07:00
Jeff McCune
0886788238 Merge pull request #272 from holos-run/jeff/268-v1alpha4
v1alpha4: add join transformer
2024-10-09 17:45:28 -07:00
Jeff McCune
4fd6785a10 v1alpha4: add Join transformer
The Join transformer was not implemented.  This patch completes the
transformers we support, the Join and Kustomize transformers.
2024-10-09 17:42:56 -07:00
Jeff McCune
e4695fa204 artifact: add Save() method to save artifacts
The code was inlined in a number of places, it makes sense to move it to
the interface.  It'll also make it easier to test, we can provide a null
writer concrete value.
2024-10-09 16:53:31 -07:00
Jeff McCune
4cd9395e6c v1alpha4: add concurrent build plan artifacts
Previously the Artifact collection was processed sequentially.  This
patch provides a modest performance improvement, about 16% faster for
our simple 2 artifact use case, by processing each artifact
concurrently.
2024-10-09 16:15:46 -07:00
Jeff McCune
6f4f355ee0 Merge pull request #270 from holos-run/jeff/268-v1alpha4
v1alpha4 - Resources and Kustomize
2024-10-09 15:28:13 -07:00
Jeff McCune
0fde16f477 v1alpha4: fix lint 2024-10-09 14:43:03 -07:00
Jeff McCune
426b4323f7 v1alpha4: inject component build plan name from platform
Platform rendering provides poor user feedback:

```
❯ holos render platform ./platforms/minimal
rendered namespaces for cluster local in 143.068583ms
rendered namespaces for cluster local in 143.861834ms
rendered namespaces for cluster local in 144.072666ms
rendered namespaces for cluster local in 144.219417ms
rendered platform in 144.326625ms
```

We want to see the metadata.name field of each BuildPlan.  This patch
injects the build plan name from the platform spec to make the name
available through the end to end platform rendering process.

Result:

```
❯ holos render platform ./platforms/minimal
rendered stage-namespaces for cluster local in 146.078375ms
rendered prod-namespaces for cluster local in 146.544583ms
rendered test-namespaces for cluster local in 147.0535ms
rendered dev-namespaces for cluster local in 147.499166ms
rendered platform in 147.553875ms
```
2024-10-09 14:06:42 -07:00
Jeff McCune
ee1e4988a6 v1alpha4: write fully rendered build plan artifacts
With this patch the first use case of CUE Resources + Kustomize is fully
working, artifacts are written into the deploy directory.

❯ holos render platform ./platforms/minimal
rendered namespaces for cluster local in 143.068583ms
rendered namespaces for cluster local in 143.861834ms
rendered namespaces for cluster local in 144.072666ms
rendered namespaces for cluster local in 144.219417ms
rendered platform in 144.326625ms

The output indicates we need to plumb the BuildPlan metadata.name from
the PlatfromSpec through to the render component command.  This is
necessary so we can report the correct name instead of just the base
path.
2024-10-09 14:06:42 -07:00
Jeff McCune
5c391e8444 v1alpha4: refactor error handing for resources and kustomize
Consistently use errors.Format("%s: %w", msg, err)
2024-10-09 14:06:41 -07:00
Jeff McCune
aba1b44f4d v1alpha4: fix resources manifest generation
Without this patch holos writes a single yaml document that is a list.
It needs to write a file that contains multiple documents, each document
a map[string]any representing the kubernetes resource.

This patch fixes the problem.  With this patch kustomize fully executes.
2024-10-09 14:06:41 -07:00
Jeff McCune
44f9615a93 v1alpha4: implement kustomize transformer 2024-10-09 14:06:41 -07:00
Jeff McCune
69a6d2acad v1alpha4: implement resources generator 2024-10-09 14:06:40 -07:00
Jeff McCune
d2c94dc8df refactor artifact manifest field to inputs and output
The manifest field isn't clear.

Much more clear to have generators produce one Output.  Transformers
take multiple Inputs and produce one Output.

The final Transformer, or a single Generator, must produce the final
Artifact.

The Inputs and Output naming to produce an Artifact makes clear the
rendering pipeline we're implementing.

This also makes clear that multiple generators must have at least one
transformer to produce the final output artifact.  We model a simple
Join transformer for this case, which is what `holos` was implicitly
doing previously.
2024-10-09 14:06:40 -07:00
Jeff McCune
01720b38fc refactor BuildContext back to Component
Component makes much more sense, that's the domain terminology we use.
BuildContext was meant to be re-used elsewhere, but we never did so the
name serves no purpose.
2024-10-09 14:06:40 -07:00
Jeff McCune
632e3c2725 refactor BuildStep to Artifact
Step doesn't make sense, they're not sequential.  They're meant to be
concurrently produced, and the whole point is to produce one artifact so
we might as well call it an Artifact.

```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
  name: prod-namespaces
spec:
  component: projects/platform/components/namespaces
  artifacts:
    - artifact: clusters/no-cluster/components/prod-namespaces/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: resources.gen.yaml
          resources:
            Namespace:
              prod-jeff:
                metadata:
                  name: prod-jeff
                  labels:
                    kubernetes.io/metadata.name: prod-jeff
                kind: Namespace
                apiVersion: v1
              prod-gary:
                metadata:
                  name: prod-gary
                  labels:
                    kubernetes.io/metadata.name: prod-gary
                kind: Namespace
                apiVersion: v1
              prod-nate:
                metadata:
                  name: prod-nate
                  labels:
                    kubernetes.io/metadata.name: prod-nate
                kind: Namespace
                apiVersion: v1
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
    - artifact: clusters/no-cluster/gitops/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: application.gen.yaml
          resources:
            Application:
              argocd:
                apiVersion: argoproj.io/v1alpha1
                kind: Application
                metadata:
                  name: prod-namespaces
                  namespace: argocd
                spec:
                  destination:
                    server: https://kubernetes.default.svc
                  project: default
                  source:
                    path: examples/v1alpha4/deploy/clusters/no-cluster/components/prod-namespaces
                    repoURL: https://github.com/holos-run/bank-of-holos
                    targetRevision: main
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
```
2024-10-09 14:06:39 -07:00
Jeff McCune
18e0b48012 refactor generators and transformers
The repeated enabled booleans and file fields are awkward.  It's clear
it's three separate things smashed into one.

kustomize isn't really a generator.  It's useless because there is no
way to reference a plain file in a component directory.

This patch replaces the kustomize generator with a file generator which
simply reads one single file.  Multiple of these generators may be used
to read one or more files.

Then, kustomize may transform these generated files, which are generated
by simply reading from the filesystem.

This API is much improved over the previous.

```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
  name: prod-namespaces
spec:
  component: projects/platform/components/namespaces
  steps:
    - artifact: clusters/no-cluster/components/prod-namespaces/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: resources.gen.yaml
          resources:
            Namespace:
              prod-jeff:
                metadata:
                  name: prod-jeff
                  labels:
                    kubernetes.io/metadata.name: prod-jeff
                kind: Namespace
                apiVersion: v1
              prod-gary:
                metadata:
                  name: prod-gary
                  labels:
                    kubernetes.io/metadata.name: prod-gary
                kind: Namespace
                apiVersion: v1
              prod-nate:
                metadata:
                  name: prod-nate
                  labels:
                    kubernetes.io/metadata.name: prod-nate
                kind: Namespace
                apiVersion: v1
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
    - artifact: clusters/no-cluster/gitops/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: application.gen.yaml
          resources:
            Application:
              argocd:
                apiVersion: argoproj.io/v1alpha1
                kind: Application
                metadata:
                  name: prod-namespaces
                  namespace: argocd
                spec:
                  destination:
                    server: https://kubernetes.default.svc
                  project: default
                  source:
                    path: examples/v1alpha4/deploy/clusters/no-cluster/components/prod-namespaces
                    repoURL: https://github.com/holos-run/bank-of-holos
                    targetRevision: main
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
```
2024-10-09 14:06:39 -07:00
Jeff McCune
7dfa9dcb93 remove unnecessary buildstep name
The manifest field is the critical piece of information, not an
arbitrary name.
2024-10-09 14:06:17 -07:00
Jeff McCune
3dedd857ea add manifest field to each build step
A build step either produces kubernetes objects or a gitops manifest.
Both are effectively the same, they're just kubernetes resources.

For the use case of applying common labels to both, we'll have the
Author API pass the same Kustomization to two separate build steps.  One
step to produce the resources, a second to produce the argocd
application or flux kustomization.
2024-10-09 14:06:17 -07:00
Jeff McCune
cd379167cc don't provide a default value for files
Leave this for the author api to avoid always including them in the
exported output.
2024-10-09 14:06:16 -07:00
Jeff McCune
52a5348f82 add build plan build step name
Each step produces a manifest and a gitops file, so we need a unique
name for each step.  The most common case will be a single build step
matching the name of the build plan itself.
2024-10-09 14:06:16 -07:00
Jeff McCune
7c5c8fe692 add fields to store generator output and transformer input
The kustomize transformer needs a filename to store the output from
generators so it has an input for the transformer.  This patch adds
fields for each kind of generator so the kustomize.#Kustomization can be
configured with the files `holos` will write generated output to.
2024-10-09 14:06:15 -07:00
Jeff McCune
ec371ed688 inject missing component tag from render platform to render component 2024-10-09 14:06:15 -07:00
Jeff McCune
1984410577 lift up component path to build plan spec
Hard to use when it's deep inside each build step.
2024-10-09 14:06:15 -07:00
Jeff McCune
438e01fbad replace structpb.Struct with map[string]any
So we can decode using a plain json decoder without needing to marshal
the api objects into yaml from CUE.
2024-10-09 14:06:14 -07:00
Jeff McCune
d122cadae6 render: draft 1 of v1alpha4 core api (#268)
First draft, now to try and wire it up to a platform and namespaces
component.
2024-10-09 14:06:14 -07:00
Jeff McCune
53fcdf307b render: build plan builder (#268)
This patch implements the v1alpha4 component rendering builder for a
component BuildPlan.  We don't yet have the CUE definitions, so this
hasn't been end to end tested yet, the next step is defining the
generators and transforms in the core API BuildPlan.
2024-10-08 06:15:15 -07:00
Jeff McCune
5e6bb96147 render: plumb v1alpha4 component (#268)
This patch plumbs the switch statement to branch on a v1alpha4
BuildPlan.  Tags need to be passed from the render platform subcommand
to the render component subcommand via the --tags argument.
2024-10-07 17:23:21 -07:00
Jeff McCune
94d03f9c59 render: render v1alpha4 platform (#268)
This patch implements minimal rendering of a v1alpha4 platform using the
new render.Builder interface.

Tags aren't wired up yet, but this patch does cleanly separate Builder
interface from the Artifacts.  Platform rendering doesn't have an
artifact itself, all artifacts are produced by rendering each component,
so we'll see how that works when we make the same changes to component
rendering, breaking it down to a render.Builder interface that sets
values in an Artifact.
2024-10-07 16:09:06 -07:00
Jeff McCune
0c01c9177d render: switch platform on api version (#268)
The holos cli does not use an interface to handle different Platform api
versions.  This makes it difficult to evolve the API in a backwards
compatible way.

This patch adds a top level switch statement to the `holos render
platform` command.  The switch discriminates on the Platform API
version.  v1alpha3 and earlier are classified as legacy versions and
will use the existing strict types.  v1alpha4 and later versions will
use an interface to render the platform, allowing for multiple types to
implement the platform rendering interface.
2024-10-07 16:08:59 -07:00
Gary Larizza
36f542da7a Merge pull request #269 from holos-run/gl/landing-page-condense
Make the landing page more concise
2024-10-07 14:17:08 -07:00
Gary Larizza
e41f0aa70c Make the landing page more concise
PROBLEM:

The landing page contains a lot of text, and much of that text was
written before we refined our messaging within the guides and technical
overview pages.

SOLUTION:

* Whittle down landing page text to only the key messages we want to convey.
* Provide messaging bullets for the features.
* Steer folks (via links) to the quickstart guide or technical overview document.

OUTCOME:

Visitors don't need to wade through a lot of text to receive key
messaging talking points or links to the pages they should read.
2024-10-07 11:17:21 -07:00
Jeff McCune
351c8ba74a docs: landing page learn more button to technical overview
Instead of the introduction.
2024-10-04 13:15:28 -07:00
Gary Larizza
a0e0b5bb75 Merge pull request #267 from holos-run/gl/diagram-landing-page
Add Holos diagram to landing page
2024-10-04 12:44:23 -07:00
Gary Larizza
7bc94e314c Add Holos diagram to landing page
PROBLEM:

There's a lot of text to grok on the landing page. A diagram would help
to visually convey what Holos does.

SOLUTION:

* Create a diagram
* Add to landing page

OUTCOME:

A visual aide is present on the landing page that helps explain where
Holos sits.
2024-10-04 12:25:02 -07:00
Jeff McCune
9681ce02e7 docs: clean up tech overview pass 3 (#263) 2024-10-04 08:56:44 -07:00
Jeff McCune
07e8e9f5f3 docs: clean up tech overview pass 2 (#263) 2024-10-03 16:09:13 -07:00
Jeff McCune
437d8a7824 docs: clean up tech overview pass 1 (#263) 2024-10-03 09:56:43 -07:00
Jeff McCune
6cc8214636 docs: fix spelling in technical overview (#263) 2024-10-02 11:29:14 -07:00
Jeff McCune
7d8f324014 blog: add holos technical overview article (#263) 2024-10-02 11:22:08 -07:00
Jeff McCune
8555d56f8c website: enable gtag 2024-09-26 11:55:23 -07:00
Jeff McCune
5884d720f2 website: add mailing lists to footer 2024-09-26 11:38:58 -07:00
Jeff McCune
92d274ced1 website: add support page 2024-09-26 11:30:50 -07:00
Jeff McCune
4dd78bd826 guides: add rendering pipeline diagram to quickstart 2024-09-24 15:24:34 -07:00
Jeff McCune
aeb8fd8e72 blog: add launch article 2024-09-24 15:05:16 -07:00
Gary Larizza
1e9744f748 website: update quickstart narrative (#260)
PROBLEM:

The Quickstart is lacking narrative tying the changes we're asking
people to make to the underlying organizational problems.

SOLUTION:

Improve the narrative to surface the problems we are solving and how
this affects the different teams at the Bank of Holos

OUTCOME:

Clarity on the problems the quickstart is solving.

Closes: #259
2024-09-24 11:37:24 -07:00
Jeff McCune
a95abe65f6 website: refine introduction to be more concise 2024-09-24 07:10:41 -07:00
Jeff McCune
c58510b92c website: refine landing to be more concise 2024-09-24 06:59:14 -07:00
Jeff McCune
302a7bfcf0 guides: fix grammar and tweak the change a service guide 2024-09-23 20:48:35 -07:00
Jeff McCune
decbbaab0f guides: fix typos in change a service guide 2024-09-23 20:04:58 -07:00
Jeff McCune
822f599202 guides: move reset cluster step in change a service
Move to the correct place, after forking the repo.
2024-09-23 20:02:03 -07:00
Jeff McCune
6a47edbc3d guides: Add Change Service Guide (#253)
This patch also adds Organization to the Author API as an example of a
regular expression constraint.
2024-09-23 19:54:46 -07:00
Jeff McCune
67d00f1dd4 website: get rid of What is Holos? on landing page 2024-09-23 14:08:09 -07:00
Jeff McCune
bfa02cd6ed website: introduction (#258) 2024-09-23 13:45:28 -07:00
Jeff McCune
2d8ca474f3 website: Landing Page 3 (#256) 2024-09-23 10:08:25 -07:00
Jeff McCune
c7cd6f5190 website: Landing Page 2 (#256) 2024-09-20 17:14:34 -07:00
Jeff McCune
21e3e6f5e4 website: Landing Page 1 (#256) 2024-09-20 14:57:40 -07:00
Jeff McCune
16a4f89c2f doc: clarify story in the deploy a service guide pass 2
Focus on the migration team and platform team.
2024-09-20 11:12:41 -07:00
Jeff McCune
f15dea5ee7 doc: clarify story in the deploy a service guide
Focus on the migration team and platform team.
2024-09-20 11:02:41 -07:00
Jeff McCune
a3bbadd1f5 doc: apply manifests for deploy a service guide
This gets us through to the end with podinfo deployed.  Need to tell the
story of the migration team a bit better though, working with the
platform team to expose the service.
2024-09-20 09:42:00 -07:00
Jeff McCune
30cbb0d537 doc: add deploy a service guide
Covers wrapping a helm chart with a Holos Component.
2024-09-19 21:31:45 -07:00
Jeff McCune
6041fd4d76 website: fix broken links 2024-09-19 09:51:27 -07:00
Jeff McCune
fec1de0004 website: holistic platform manager social card 2024-09-19 09:49:56 -07:00
Jeff McCune
6ad24a6eec package: rename schema api to author api (#248)
Schema API is unclear, Author API is more clear the API is intended for
component authors.
2024-09-19 08:51:01 -07:00
Jeff McCune
57dedc6450 website: clean up placeholders 2024-09-19 08:30:31 -07:00
Jeff McCune
8d2a9dd659 quickstart: re-focus on core concepts 2024-09-18 17:24:44 -07:00
Jeff McCune
e3c53f5655 website: tweak features on landing page for clarity 2024-09-18 13:44:44 -07:00
Jeff McCune
3b833cdacd website: update landing page to focus on platform management
Instead of package management.
2024-09-18 13:41:36 -07:00
Jeff McCune
31d1086345 render: shorten the component rendered in the output
It's too long for documentation.  Shorten it for clarity.

Result:

```
❯ holos render platform ./platform
rendered bank-accounts-db for cluster workload in 160.7245ms
rendered bank-ledger-db for cluster workload in 162.465625ms
rendered bank-userservice for cluster workload in 166.150417ms
rendered bank-ledger-writer for cluster workload in 168.075459ms
rendered bank-balance-reader for cluster workload in 172.492292ms
rendered bank-backend-config for cluster workload in 198.117916ms
rendered bank-secrets for cluster workload in 223.200042ms
rendered gateway for cluster workload in 124.841917ms
rendered httproutes for cluster workload in 131.86625ms
rendered bank-contacts for cluster workload in 154.463792ms
rendered bank-transaction-history for cluster workload in 159.968208ms
rendered bank-frontend for cluster workload in 325.24425ms
rendered app-projects for cluster workload in 110.577916ms
rendered ztunnel for cluster workload in 137.502792ms
rendered cni for cluster workload in 209.993375ms
rendered cert-manager for cluster workload in 172.933834ms
rendered external-secrets for cluster workload in 135.759792ms
rendered local-ca for cluster workload in 98.026708ms
rendered istiod for cluster workload in 403.050833ms
rendered argocd for cluster workload in 294.663167ms
rendered gateway-api for cluster workload in 228.47875ms
rendered namespaces for cluster workload in 113.586916ms
rendered base for cluster workload in 533.76675ms
rendered external-secrets-crds for cluster workload in 529.053375ms
rendered crds for cluster workload in 931.180458ms
rendered platform in 1.248310167s
```

Previously:

```
❯ holos render platform ./platform
rendered projects/bank-of-holos/backend/components/bank-ledger-db for cluster workload in 158.534875ms
rendered projects/bank-of-holos/backend/components/bank-accounts-db for cluster workload in 159.836166ms
rendered projects/bank-of-holos/backend/components/bank-userservice for cluster workload in 160.360667ms
rendered projects/bank-of-holos/backend/components/bank-balance-reader for cluster workload in 169.478584ms
rendered projects/bank-of-holos/backend/components/bank-ledger-writer for cluster workload in 169.437833ms
rendered projects/bank-of-holos/backend/components/bank-backend-config for cluster workload in 182.089333ms
rendered projects/bank-of-holos/security/components/bank-secrets for cluster workload in 196.502792ms
rendered projects/platform/components/istio/gateway for cluster workload in 122.273083ms
rendered projects/bank-of-holos/frontend/components/bank-frontend for cluster workload in 307.573584ms
rendered projects/platform/components/httproutes for cluster workload in 149.631583ms
rendered projects/bank-of-holos/backend/components/bank-contacts for cluster workload in 153.529708ms
rendered projects/bank-of-holos/backend/components/bank-transaction-history for cluster workload in 165.375667ms
rendered projects/platform/components/app-projects for cluster workload in 107.253958ms
rendered projects/platform/components/istio/ztunnel for cluster workload in 137.22275ms
rendered projects/platform/components/istio/cni for cluster workload in 233.980958ms
rendered projects/platform/components/cert-manager for cluster workload in 171.966958ms
rendered projects/platform/components/external-secrets for cluster workload in 134.207792ms
rendered projects/platform/components/istio/istiod for cluster workload in 403.19ms
rendered projects/platform/components/local-ca for cluster workload in 97.544708ms
rendered projects/platform/components/argocd/argocd for cluster workload in 289.577208ms
rendered projects/platform/components/gateway-api for cluster workload in 218.290458ms
rendered projects/platform/components/namespaces for cluster workload in 109.534125ms
rendered projects/platform/components/istio/base for cluster workload in 526.32525ms
rendered projects/platform/components/external-secrets-crds for cluster workload in 523.7495ms
rendered projects/platform/components/argocd/crds for cluster workload in 1.002546375s
rendered platform in 1.312824333s
```
2024-09-17 11:40:14 -07:00
Jeff McCune
b737543c13 version: 0.95.0 2024-09-17 08:06:38 -07:00
Jeff McCune
8e150ee0d7 generate: fix external-secrets always out of sync in argocd
Without this patch ArgoCD treats the Application as constantly out of
sync.  This is also a good example of how to patch an arbitrary
component, though it patches the core BuildPlan itself now.  If this is
widely used, it would be nice to add this behavior to the schema api
(aka author api).
2024-09-16 21:22:23 -07:00
Jeff McCune
117a2a886d generate: fix istio-base always out of sync in argocd
Without this patch ArgoCD treats the Application as constantly out of
sync.  This is also a good example of how to patch an arbitrary
component, though it patches the core BuildPlan itself now.  If this is
widely used, it would be nice to add this behavior to the schema api
(aka author api).
2024-09-16 20:46:29 -07:00
Jeff McCune
79b41dfbf5 generate: fix istiod always out of sync in argocd
Without this patch ArgoCD treats the Application as constantly out of
sync.  This is also a good example of how to patch an arbitrary
component, though it patches the core BuildPlan itself now.  If this is
widely used, it would be nice to add this behavior to the schema api
(aka author api).
2024-09-16 20:38:35 -07:00
Jeff McCune
55562f9d83 generate: move istio-k3d schematic to projects structure
To match the current layout of the guide platform we're using for the
guides.
2024-09-16 20:02:23 -07:00
Jeff McCune
e0a636f183 generate: move gateway-api schematic to projects structure
To match the current layout of the guide platform we're using for the
guides.
2024-09-16 19:53:28 -07:00
Jeff McCune
1fa74214cf generate: move istio schematic to projects structure
To match the current layout of the guide platform we're using for the
guides.
2024-09-16 19:51:02 -07:00
Jeff McCune
e5851cac57 generate: fix bank of holos connection reset
Without this patch browsing https://bank.holos.localhost frequently gets
connection reset errors.  These errors are caused by the frontend
deployment redirecting the browser to http, which is not enabled on the
Gateway we use in the guides.

This patch sets the scheme to https which corrects the problems.

See https://github.com/GoogleCloudPlatform/bank-of-anthos/issues/478
2024-09-16 19:29:21 -07:00
Jeff McCune
4a26662b92 generate: add bank-contacts
Needed to load the home page of the Bank of Holos demo.
2024-09-16 19:29:02 -07:00
Jeff McCune
6bc6888ffc generate: add bank-transaction-history
Needed to load the home page of the Bank of Holos demo.
2024-09-16 17:00:15 -07:00
Jeff McCune
dab1f305e1 generate: add bank-balance-reader
Needed to load the home page of the Bank of Holos demo.
2024-09-16 16:52:52 -07:00
Jeff McCune
fbe79dd0af generate: add bank-ledger-db and bank-ledger-writer
Needed to load the home page of the Bank of Holos demo.
2024-09-16 16:46:35 -07:00
Jeff McCune
6d6829b149 generate: refactor bank backend config to a component
To fix ArgoCD SharedResourceWarning.
2024-09-16 16:18:31 -07:00
Jeff McCune
971a3fa280 generate: fix accounts-db using wrong service account
Needs to match the bank-of-holos service account name.
2024-09-16 15:58:50 -07:00
Jeff McCune
7632344cd1 generate: add bank-accounts-db needed by userservice
With this patch the frontend, accounts-db, and userservice all start and
become ready.

The user can log in, but on redirecting to home the site can't be
reached.
2024-09-16 15:51:29 -07:00
Jeff McCune
42067748ad generate: add bank userservice to backend 2024-09-16 15:49:25 -07:00
Jeff McCune
340c3484e5 generate: refactor how the bank jwt-key is created
This makes the example more re-usable, reader need only change the
SecretName, the bash script, and replace the #BankOfHolos references.
2024-09-16 15:02:12 -07:00
Jeff McCune
250238c286 generate: add secret store and external secret to bank-of-holos
Rather than commit the jwt private key to version control like upstream
does, we use a SecretStore and ExternalSecret to sync the secret
generated by the security team in the bank-security namespace.

With this patch the SecretStore validates and the ExternalSecret
automatically syncs the secret from the bank-security namespace to the
bank-frontend namespace.

```
❯ k get ss
NAME            AGE   STATUS   CAPABILITIES   READY
bank-security   1s    Valid    ReadWrite      True

❯ k get es
NAME      STORE           REFRESH INTERVAL   STATUS         READY
jwt-key   bank-security   5s                 SecretSynced   True
```

The pod start successfully.

```
❯ k get pods
NAME                        READY   STATUS    RESTARTS   AGE
frontend-646d797d6b-7jhrx   1/1     Running   0          2m39s

❯ k logs frontend-646d797d6b-7jhrx
{"timestamp": "2024-09-16 21:44:47", "message": "info | Starting gunicorn 22.0.0", "severity": "INFO"}
{"timestamp": "2024-09-16 21:44:47", "message": "info | Listening at: http://0.0.0.0:8080 (7)", "severity": "INFO"}
{"timestamp": "2024-09-16 21:44:47", "message": "info | Using worker: gthread", "severity": "INFO"}
{"timestamp": "2024-09-16 21:44:47", "message": "info | Booting worker with pid: 8", "severity": "INFO"}
{"timestamp": "2024-09-16 21:44:57", "message": "create_app | Unable to retrieve cluster name from metadata server metadata.google.internal.", "severity": "WARNING"}
{"timestamp": "2024-09-16 21:44:57", "message": "create_app | Unable to retrieve zone from metadata server metadata.google.internal.", "severity": "WARNING"}
{"timestamp": "2024-09-16 21:44:57", "message": "create_app | Starting frontend service.", "severity": "INFO"}
{"timestamp": "2024-09-16 21:44:57", "message": "create_app | 🚫 Tracing disabled.", "severity": "INFO"}
{"timestamp": "2024-09-16 21:44:57", "message": "create_app | Platform is set to 'local'", "severity": "INFO"}
```
2024-09-16 14:47:33 -07:00
Jeff McCune
a223e2b426 generate: fix duplicate external secrets crds
They're handled outside the helm chart to make upgrades easier.
2024-09-16 13:39:04 -07:00
Jeff McCune
63a7da02e7 generate: add external secrets operator
Need this to sync secrets into the bank-frontend and bank-backend
namespace from the bank-security namespace.
2024-09-16 13:29:03 -07:00
Jeff McCune
569f827e30 speed up argocd crds with raw urls 2024-09-16 13:29:02 -07:00
Jeff McCune
4a656db2ec render: log total render platform time 2024-09-16 13:29:02 -07:00
Jeff McCune
77b0933961 generate: add httproute for bank.holos.localhost
Expose Service frontend in the bank-frontend namespace via httproute
https://bank.holos.localhost

Organize into frontend, backend, security projects to align with three
teams who would each own this work.

remove secret from version control

Google added the secret to version control but we can generate the
secret in-cluster.  Holos makes it easier to manage the ExternalSecret
or RoleBinding necessary to get it in the right place.
2024-09-16 12:46:00 -07:00
Jeff McCune
3b796cfbbd generate: add bank-of-holos frontend
We need a way to demonstrate the value Holos offers in a platform team
managing projects for other teams.  This patch addresses the need by
establishing the bank-of-holos schematic, which is a port of the Bank of
Anthos project to Holos.

This patch adds only the frontend to get the process started.  As of
this patch the frontend pod starts and becomes ready but is not exposed
via HTTPRoute.

Refer to https://github.com/GoogleCloudPlatform/bank-of-anthos/
2024-09-15 22:08:28 -07:00
Jeff McCune
8a7a010b94 version 0.94.0 2024-09-15 15:41:17 -07:00
Jeff McCune
2454f6e9ee generate: app-projects to organize ArgoCD Applications into Projects
Previously all generated ArgoCD Application resources go into the
default project following the Quickstart guide.  The configuration code
is being organized into the concept of projects in the filesystem, so we
want to the GitOps configuration to also reflect this concept of
projects.

This patch extends the ArgoConfig user facing schema to accept a project
string.  The app-projects component automatically manages AppProject
resources in the argocd namespace for each of the defined projects.

This allows CUE configuration in the a project directory to specify the
project name so that all Applications are automatically assigned to the
correct project.
2024-09-15 14:57:13 -07:00
Jeff McCune
63d00bfddf schema: handle ArgoConfig for all component kinds
Providing ArgoConfig only works with the Helm kind without this patch.
This is a problem because we want to produce an Application for every
supported component kind when rendering the platform.

This patch threads the ArgoConfig struct described in the Quickstart
guide through every supported component kind.
2024-09-15 13:09:07 -07:00
Jeff McCune
f34da6c24e generate: add schematic for manage a project guide gitops
This patch configures the platform to generate Application resources for
all of the components in the manage a project guide.
2024-09-15 12:57:51 -07:00
Jeff McCune
1d98069b73 generate: add httproutes schematic
Define a place for components to register HTTPRoute resources the
platform team needs to manage in the Gateway namespace.

The files are organized to delegate to the platform team.

This patch also fixes the naming of the argocd component so that the
Service is argocd-server instead of argo-cd-argocd-server.
2024-09-15 12:32:45 -07:00
Jeff McCune
e956b64d04 schematic: comment how kustomization of argocd crds works
This is the only example we have right now of producing a kustomization
entirely from CUE.
2024-09-15 09:37:42 -07:00
Jeff McCune
054d33b498 builder: add kustomization post-processing to kubernetes build plans (#246)
Holos does not post-process a KubernetesObjects core package build plan
with kustomize.  This is necessary to pass the ArgoCD version through to
Kustomize to fetch the correct crds.

This patch enables the kustomization and provides an example in the
argocd schematic.

Result: The KubernetesObjects component doesn't actually have any
resources defined, so holos creates an empty `build-plan-resources.yaml`
file.  This is fine, the kustomize post-processing adds the actual
resources via https URL passing in the correct ArgoCD version.

```
❯ holos render component --cluster-name=workload ./projects/platform/components/argocd/crds --log-level=debug --log-format=text
9:20AM DBG config.go:166 finalized config from flags version=0.93.4 state=finalized
9:20AM DBG builder.go:234 cue: building instances version=0.93.4
9:20AM DBG builder.go:251 cue: validating instance version=0.93.4 dir=/Users/jeff/Holos/holos-manage-a-project-guide/projects/platform/components/argocd/crds
9:20AM DBG builder.go:256 cue: decoding holos build plan version=0.93.4 dir=/Users/jeff/Holos/holos-manage-a-project-guide/projects/platform/components/argocd/crds
9:20AM DBG builder.go:270 cue: discriminated build kind: BuildPlan version=0.93.4 dir=/Users/jeff/Holos/holos-manage-a-project-guide/projects/platform/components/argocd/crds kind=BuildPlan apiVersion=v1alpha3
9:20AM DBG builder.go:314 allocated results slice version=0.93.4 cap=1
9:20AM DBG result.go:156 wrote: /var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize3526125146/build-plan-resources.yaml version=0.93.4 op=write path=/var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize3526125146/build-plan-resources.yaml bytes=0
9:20AM DBG result.go:169 wrote: /var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize3526125146/kustomization.yaml version=0.93.4 op=write path=/var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize3526125146/kustomization.yaml bytes=174
9:20AM DBG run.go:40 running: kubectl version=0.93.4 name=kubectl args="[kustomize /var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize3526125146]"
9:20AM DBG remove.go:16 tmp: removed version=0.93.4 path=/var/folders/22/zt67pphj6h1fgknqfy23ppl80000gn/T/holos.kustomize3526125146
9:20AM DBG builder.go:350 returning results version=0.93.4 len=1
9:20AM DBG result.go:214 out: wrote deploy/clusters/workload/components/argocd-crds/argocd-crds.gen.yaml version=0.93.4 action=write path=deploy/clusters/workload/components/argocd-crds/argocd-crds.gen.yaml status=ok
9:20AM INF render.go:79 rendered argocd-crds version=0.93.4 cluster=workload name=argocd-crds status=ok action=rendered
```

Closes: #246
2024-09-15 09:25:25 -07:00
Jeff McCune
f2f75a4e00 generate: fix argo-cd component 2024-09-15 09:06:53 -07:00
Jeff McCune
a0cf73faf9 generate: remove argocd kustomization.yaml 2024-09-15 09:00:41 -07:00
Jeff McCune
d74655c632 generate: add argocd schematic
Using the projects layout.

This patch also includes a method to pass a version to a Kustomization.
2024-09-15 08:57:57 -07:00
Jeff McCune
b8019429b8 docs: add manage a project guide draft (#242)
Initial draft of the Manage a Project guide focused on how a development
team can self serve resources provided by a platform team.
2024-09-14 15:35:12 -07:00
Jeff McCune
9c08214118 docs: add the outline of the projects guide (#242) 2024-09-14 13:43:31 -07:00
Jeff McCune
f58d791e03 api: move #Resources to package holos
Previously, the #Resources struct listing valid resources to use with
APIObjects in each of the components types was closed.  This made it
very difficult for users to mix in new resources and use the Kubernetes
component kind.

This patch moves the definition of the valid resources to package holos
from the schema API.  The schema still enforces some light constraints,
but doesn't keep the struct closed.

A new convention is introduced in the form of configuring all components
using _ComponentConfig defined at the root, then unifying this struct
with all of the component kinds.  See schema.gen.cue for how this works.

This approach enables mixing in ArgoCD applications to all component
kinds, not just Helm as was done previously.  Similarly, the
user-constrained #Resources definition unifies with all component kinds.

It's OK to leave the yaml.Marshall in the schema API.  The user
shouldn't ever have to deal with #APIObjects, instead they should pass
Resources through the schema API which will use APIObjects to create
apiObjectMap for each component type and the BuildPlan.

This is still more awkward than I want, but it's a good step in the
right direction.
2024-09-13 16:43:12 -07:00
Jeff McCune
836033e16a docs: move istio-gateway to a separate schematic
Without this patch the istio-gateway component isn't functional, the
HTTPRoute created for httpbin isn't programmed correctly.  There is no
Gateway resource, just a deployment created by the istio helm chart.

This patch replaces the helm chart with a Gateway resource as was done
previously in the k3d platform schematic.

This patch also simplifies the certificate management to issue a single
cert valid for the platform domain and a wildcard.  We intentionally
avoid building a dynamic Gateway.spec.listeners structure to keep the
expose a service guide relatively simple and focused on getting started
with Holos.
2024-09-13 11:13:18 -07:00
Jeff McCune
77279d9baf docs: add httpbin routes section to expose a service guide
This patch adds the httpbin routes component.  It's missing the
Certificate component, the next step is to wire up automatic certificate
management in the gateway configuration, which is a prime use case for
holos.  Similar to how we register components and namespaces, we'll
register certificates.

This patch also adds the #Platform.Domain field to the user facing
schema API.  We previously stored the domain in the Model but it makes
sense to lift it up to the Platform and have a sensible default value
for it.

Another example of #237 needing to be addressed soon.
2024-09-12 17:35:06 -07:00
Jeff McCune
bf19aee1a7 docs: add httpbin workload section to expose a service
This patch manages the httpbin Deployment, Service, and ReferenceGrant.
The remaining final step is to expose the service with an HTTPRoute and
Certificate.

We again needed to add a field to the schema APIObjects to get this to
work.  We need to fix #237 soon.  We'll need to do it again for the
HTTPRoute and Certificate resources.
2024-09-12 16:55:09 -07:00
Jeff McCune
4de88b3155 docs: insert cert-manager after namespaces in expose a service
The progression of namespaces, cert-manager, then gateway api and istio
makes much more sense than the previous progression of gateway api,
namespaces, istio.

cert-manager builds nicely on top of namespaces.  gateway api are only
crds necessary for istio.

This patch also adds the local-ca component which surfaces issue #237
The Kubernetes APIObjects are unnecessarily constrained to resources we
define in the schema.  We need to move the marshal code into package
holos so the user can add their own resource kinds.
2024-09-12 15:20:08 -07:00
Jeff McCune
6f39cc6fdc docs: add istio section to expose-a-service
This patch adds Istio to the Expose a Service documentation and
introduces new concepts.  The Kubernetes build plan schema, the
namespaces component, and an example of how to safely re-use Helm values
from the root to multiple leaf components.

fix: istio cni not ready on k3d
---

The istio-k3d component embedded into holos fixes the cni pod not
becoming ready with our k3d local cluster guide.  The pod log error this
fixes is:

    configuration requires updates, (re)writing CNI config file at "": no networks found in /host/etc/cni/net.d
    Istio CNI is configured as chained plugin, but cannot find existing CNI network config: no networks found in /host/etc/cni/net.d
    Waiting for CNI network config file to be written in /host/etc/cni/net.d...

[Platform k3d]: https://istio.io/latest/docs/ambient/install/platform-prerequisites/#k3d

docs: clarify how to reset the local cluster
---

This is something we do all the time while developing and documenting,
so make it easy and fast to reset the cluster to a known good state.
2024-09-12 10:36:56 -07:00
Jeff McCune
e410563f82 docs: add namespaces to expose a service guide
This patch adds the schema api for the Kubernetes build plan, which
produces plain API resources directly from CUE.  It's needed for the
namespaces component which is foundational to many of our guides.

The first guide that needs this is the expose a service guide, we need
to register the namespaces from the istio component.
2024-09-11 17:22:01 -07:00
Jeff McCune
0a53bef72a docs: apply the gateway-api in the expose a service doc
This patch completes the first draft of the Gateway API section.
2024-09-11 14:31:02 -07:00
Jeff McCune
02a450e597 api: clarify Name field of Helm and Kustomize schema 2024-09-11 14:09:13 -07:00
Jeff McCune
e1222cf367 docs: add the gateway-api to the expose-a-service doc
The Expose a Service doc is meant to be the second step after the
Quickstart doc.  This commit adds the section describing how to install
the Gateway API.

The Kustomize build plan is introduced at this point in a similar way
the Helm build plan was introduced in the quickstart.
2024-09-11 14:03:40 -07:00
Jeff McCune
740a3d21a1 generate: add schematic for a workload-cluster
We need an easy way to help people add a workload cluster to their
workload fleet when working through the guides.  Generated platforms
should not define any clusters so they can be reused with multiple
guides.

This patch adds a simple component schematic that drops a root cue file
to define a workload cluster named workload.

The result is the following sequence renders the Gateway API when run
from an empty directory.

    holos generate platform guide
    holos generate component workload-cluster
    holos generate component gateway-api
    holos render platform ./platform

Without this patch nothing is rendered because there are no workload
clusters in the base guide platform.
2024-09-11 13:23:36 -07:00
Jeff McCune
1114b65a47 schema: remove management cluster from standard fleet
Having the management cluster hard coded into the definition of the
standard fleets is problematic for guides that don't need a management
cluster.

Define the fleets, but leave the set of clusters empty until they're
needed.
2024-09-11 13:12:44 -07:00
Jeff McCune
c9d892eee3 generate: consolidate holos generate component cue/helm
Previously helm and cue components were split into two different
subcommands off the holos generate component command.  This is
unnecessary, I'm not sure why it was there in the first place.  The code
seemed perfectly duplicated.

This patch combines them to focus on the concept of a Component.  It
doesn't matter what kind it is now that it's expected to be run from the
root of the platform repository and drop configuration at the root and
the leaf of the tree.
2024-09-11 11:12:53 -07:00
Jeff McCune
4c77eba72b website: automatically generate sidebars
Previously, each document needed to be manually included in the sidebars
to show up.  In addition, index paths like /docs/ and /docs/guides/ were
not found.

This patch addresses both problems by switching sidebars to
automatically generate from filesystem directories.  Important documents
like the getting started guide and introduction are expected to add a
`slug: /foo` front matter item to create a permalink.

The result is the sidebar reflects the filesystem while the URL bar is
more of a permalink.  Files should be able to be moved around the file
system and the sidebar tree without affecting their URL.

This patch also consolidates the API and Docs sidebars into one.
2024-09-11 10:24:01 -07:00
Jeff McCune
a8ae56b08b website: remove quickstart and localhost index
No need to have these pages in sub-folders.  If we need to add images or
resources we can simply create a quickstart folder and add them there.
2024-09-11 06:50:58 -07:00
Jeff McCune
b04837ede2 website: add a localhost guide to get a k3d cluster (#234)
Our guides should be useful reading them only from a mobile device.  For
those readers who also want to apply the manifests to a real cluster we
need a companion guide that describes how to get one.

This patch adds that guide, adapted from the old try holos locally page.
2024-09-10 15:28:46 -07:00
Jeff McCune
559c8bc79f quickstart: remove side by side comparisons
Accidentally left over from cleaning up typos and grammar.
2024-09-10 14:31:29 -07:00
Jeff McCune
a30335b171 concepts: add fleet and cluster 2024-09-10 14:12:23 -07:00
Jeff McCune
108831747a quickstart: fix broken link 2024-09-10 13:40:19 -07:00
Jeff McCune
c714a2b61e quickstart: top to bottom edit for grammar, typos, and voice 2024-09-10 12:51:43 -07:00
Jeff McCune
1cba383dc1 quickstart: incorporate feedback from review
This patch incorporates the main feedback from Gary and Nate from this
morning.  The note tab in argocd.cue was awkware to Gary and I.  The use
of _ in CUE needs an explicit comment which this patch adds.
2024-09-10 11:14:59 -07:00
Jeff McCune
265d5773b8 quickstart: add day 2 chart upgrade example
This patch focuses on the Day 2 benefits holos offers, specifically
making it easier to visiualize exactly what will change when upgrading
components.

In addition, it's easier to apply changes slowly and deliberately since
they're all just flat files in the local filesystem and Git repository.
2024-09-09 20:31:56 -07:00
Jeff McCune
44f8779136 quickstart: render a platform with workload clusters
Previously the quickstart didn't cover adding workload clusters and
rendering a platform with multiple clusters.  This patch demonstrates
how it's effectively a one line change to clone the configuration of a
workload cluster to another geographic region.
2024-09-09 19:43:32 -07:00
Jeff McCune
4127804092 quickstart: v0.93.2 with schema.#Platform
Make sure go install works from the quickstart documentation by doing a
release.  Otherwise, v0.93.1 is installed which doesn't include the
platform schema.
2024-09-09 17:04:32 -07:00
Jeff McCune
8f424cfabe quickstart: sync docs to this commit
Sync the documentation to the current output of the code at this commit.
2024-09-09 17:02:53 -07:00
Jeff McCune
699148abdd quickstart: define a convenince schema for the Platform
Previously, the quickstart step of generating the pod info component and
generating the platform as a whole left the task of integrating the
Component into the Platform as an exercise for the reader.  This is a
problem because it creates unnecessary friction.

This patch addresses the problem by lifting up the Platform concept
into the user-facing Schema API.  The generated platform includes a top
level #Platform definition which exposes the core Platform specification
on the Output field.

The Platform CUE instance then reduces to a simple `#Platform.Output`
which provides the Platform spec to holos for rendering each component
for each cluster.

The CUE code for the schema.#Platform iterates over each
Component to derive the list of components to manage for the Platform.

The CUE code for the generated quickstart platform links the definition
of StandardFleets, which is a Workload fleet and a Management cluster
fleet to the Platform conveninece wrapper.

Finally, the generated podinfo component drops a CUE file at the
repository root to automatically add the component to every workload
cluster.

The result is the only task left for the end user is to define at least
one workload cluster.  Once defined, the component is automatically
managed because it is managed on all workload clusters.

This approach futher opens the door to allow generated components to
define their namespaces and generated secrets on the management cluster
separate from their workloads on the workload clusters.

This patch includes a behavior change, from now on all generated
components should assume they are writing to the root of the user's Git
repository so that they can generate files through the whole tree.

In the future, we should template output paths for generated components.
A simple approach might be to embed a file with a .target suffix, with
the contents being a simple Go template of the file path to write to.
The holos generate subcommand can then check if any given embedded file
foo has a foo.target companion, then write the target to the rendered
template value.
2024-09-09 16:05:00 -07:00
1587 changed files with 80117 additions and 231190 deletions

13
.claude/settings.json Normal file
View File

@@ -0,0 +1,13 @@
{
"permissions": {
"allow": [
"Bash(cd:*)",
"Bash(holos:*)",
"Bash(cue:*)",
"Bash(git commit:*)",
"Bash(git add:*)",
"Bash(make:*)"
],
"deny": []
}
}

View File

@@ -5,103 +5,358 @@
"mdx"
],
"words": [
"acmesolver",
"acraccesstoken",
"acraccesstokens",
"admissionregistration",
"alertmanager",
"alertmanagers",
"anchore",
"anthos",
"apiextensions",
"apimachinery",
"apiobjects",
"apiservers",
"applicationset",
"applicationsets",
"appproject",
"appprojects",
"argoproj",
"argumentless",
"authcode",
"authorizationpolicies",
"authorizationpolicy",
"authpolicy",
"authproxy",
"authroutes",
"autoload",
"automount",
"automounting",
"autoscaler",
"balancereader",
"blackbox",
"buildplan",
"buildplans",
"Buildx",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
"cainjector",
"CAROOT",
"certificaterequest",
"certificaterequests",
"certificatesigningrequests",
"chartmuseum",
"clientset",
"clsx",
"clusterexternalsecret",
"clusterexternalsecrets",
"clusterissuer",
"clusterissuers",
"clusterrole",
"clusterrolebinding",
"clustersecretstore",
"clustersecretstores",
"clusterwide",
"Cmds",
"CNCF",
"CODEOWNERS",
"compinit",
"componentconfig",
"configdir",
"configmap",
"configmapargs",
"connectrpc",
"cookiesecret",
"coredns",
"corev",
"CRD's",
"crds",
"creds",
"crossplane",
"crunchydata",
"ctxt",
"cuecontext",
"cuelang",
"customresourcedefinition",
"daemonset",
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devel",
"devicecode",
"distroless",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtoken",
"ecrauthorizationtokens",
"edns",
"endpointslices",
"entgo",
"envoyfilter",
"envoyfilters",
"errdetails",
"errgroup",
"etcdsnapshotfiles",
"externalsecret",
"externalsecrets",
"fctr",
"fieldmaskpb",
"fieldspec",
"flushcache",
"fluxcd",
"fsys",
"fullname",
"gatewayclass",
"gatewayclasses",
"gcraccesstoken",
"gcraccesstokens",
"gendoc",
"generationbehavior",
"generatorargs",
"generatoroptions",
"genproto",
"ggnpl",
"ghaction",
"githubaccesstoken",
"githubaccesstokens",
"gitops",
"GOBIN",
"godoc",
"golangci",
"gomarkdoc",
"googleapis",
"goreleaser",
"gotypesalias",
"grpcreflect",
"grpcroute",
"grpcroutes",
"grpcurl",
"hcfg",
"healthchecks",
"healthz",
"helmchartargs",
"helmchartconfigs",
"helmcharts",
"Hiera",
"holos",
"holoslogger",
"horizontalpodautoscaler",
"horizontalpodautoscalers",
"Hostaliases",
"Hostnames",
"htpasswd",
"httpbin",
"httproute",
"httproutes",
"iampolicygenerator",
"incpatch",
"Infima",
"intstr",
"isatty",
"istiod",
"jbrx",
"jeffmccune",
"jetstack",
"jiralert",
"Jsonnet",
"Kargo",
"kfbh",
"killall",
"kubeadm",
"kubeconfig",
"kubelet",
"kubelogin",
"kubernetesobjects",
"kubeversion",
"Kustomization",
"Kustomizations",
"kustomize",
"kustomizebuild",
"kvpairsources",
"labeldrop",
"labelmap",
"ldflags",
"leaderelection",
"ledgerwriter",
"libnss",
"limitranges",
"livez",
"loadbalancer",
"loadrestrictions",
"logfmt",
"lxnl",
"mattn",
"mccutchen",
"metav",
"mindmap",
"mktemp",
"msqbn",
"mtls",
"Multicluster",
"mutatingwebhookconfiguration",
"mutatingwebhookconfigurations",
"mvdan",
"mxcl",
"mychart",
"myhostname",
"myRegistrKeySecretName",
"mysecret",
"nameofclusterrole",
"nameserver",
"namespacedname",
"ndots",
"networkpolicies",
"nodename",
"nolint",
"oauthproxy",
"objectmap",
"objectmeta",
"omitempty",
"organizationconnect",
"orgid",
"otelconnect",
"outfile",
"overriden",
"Parentspanid",
"patchstrategicmerge",
"pcfg",
"pcjc",
"peerauthentication",
"peerauthentications",
"persistentvolumeclaim",
"persistentvolumeclaims",
"persistentvolumes",
"pflag",
"pgadmin",
"pgupgrade",
"pipefail",
"PKCE",
"platformconnect",
"pluginconfig",
"pluginrestrictions",
"podcli",
"poddisruptionbudget",
"poddisruptionbudgets",
"podinfo",
"podmonitor",
"portmapping",
"postgrescluster",
"privs",
"prometheuses",
"promhttp",
"protobuf",
"protojson",
"providerconfig",
"proxyconfig",
"proxyconfigs",
"Pulumi",
"pushgateway",
"pushsecret",
"pushsecrets",
"putenv",
"qjbp",
"quickstart",
"QVRFLS",
"readyz",
"referencegrant",
"referencegrants",
"Registr",
"replacementfield",
"replicasets",
"replicationcontrollers",
"requestauthentication",
"requestauthentications",
"resourcequotas",
"retryable",
"rogpeppe",
"rolebinding",
"rootfs",
"ropc",
"sboms",
"seccomp",
"secretargs",
"SECRETKEY",
"secretstore",
"secretstores",
"serverlb",
"serverside",
"serviceaccount",
"servicebindings",
"serviceentries",
"serviceentry",
"servicemonitor",
"sigstore",
"somevalue",
"SOMEVAR",
"sortoptions",
"spanid",
"spiffe",
"stackdriver",
"startupapicheck",
"statefulset",
"statefulsets",
"stefanprodan",
"storageclasses",
"streamwatcher",
"stretchr",
"struct",
"structpb",
"subcharts",
"subjectaccessreviews",
"svclb",
"sysfs",
"systemconnect",
"tablewriter",
"templatable",
"testscript",
"testutil",
"thanos",
"Tiltfile",
"timestamppb",
"Timoni",
"tlsclientconfig",
"tokencache",
"Tokener",
"tolerations",
"TOPLEVEL",
"Traceid",
"traefik",
"transactionhistory",
"tsdb",
"txtar",
"typemeta",
"udev",
"uibutton",
"Unmarshal",
"unmarshals",
"unshallow",
"unstage",
"untar",
"upbound",
"Upsert",
"urandom",
"usecases",
"userconnect",
"userdata",
"userservice",
"validatingwebhookconfiguration",
"validatingwebhookconfigurations",
"vaultdynamicsecret",
"vaultdynamicsecrets",
"virtualservice",
"virtualservices",
"volumeattachments",
"wasmplugin",
"wasmplugins",
"workdir",
"workloadentries",
"workloadentry",
"workloadgroup",
"workloadgroups",
"yournamespace",
"zerolog",
"zitadel"
"zitadel",
"ztunnel"
]
}

7
.envrc Normal file
View File

@@ -0,0 +1,7 @@
if ! use flake .#default --accept-flake-config --print-build-logs
then
echo "nix flake could not be built; update flake.nix and run direnv allow/reload" >&2
fi
watch_file nix/*.nix
watch_file flake.nix

131
.github/ISSUE_TEMPLATE/bug-report.md vendored Normal file
View File

@@ -0,0 +1,131 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: NeedsInvestigation, Triage
assignees: ''
---
<!--
Please answer these questions before submitting your issue. Thanks!
To ask questions, see https://github.com/holos-run/holos/discussions
-->
### What version of holos are you using (`holos --version`)?
```
0.0.0
```
### Does this issue reproduce with the latest release?
<!--
Get the latest release with:
brew install holos-run/tap/holos
Or see https://holos.run/docs/v1alpha5/tutorial/setup/
-->
### What did you do?
<!--
Please provide a testscript that should pass, but does not because of the bug.
See the below example.
You can create a txtar from a directory with:
holos txtar ./path/to/dir
Refer to: https://github.com/rogpeppe/go-internal/tree/master/cmd/testscript
-->
Steps to reproduce:
```shell
testscript -v -continue <<EOF
```
```txtar
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work.
exec holos --version
exec holos init platform v1alpha5 --force
# remove the fix to trigger the bug
rm cue.mod/pkg/sigs.k8s.io/kustomize/api/types/var.cue
# want a BuildPlan shown
exec holos show buildplans
cmp stdout buildplan.yaml
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- buildplan.yaml --
kind: BuildPlan
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
```
```shell
EOF
```
### What did you expect to see?
The testscript should pass.
### What did you see instead?
The testscript fails because of the bug.
```txt
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work. (0.168s)
> exec holos --version
[stdout]
0.100.1-2-g9b10e23-dirty
> exec holos init platform v1alpha5 --force
# remove the fix to trigger the bug (0.000s)
> rm cue.mod/pkg/sigs.k8s.io/kustomize/api/types/var.cue
# want a BuildPlan shown (0.091s)
> exec holos show buildplans
[stderr]
could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
$WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2
[exit status 1]
FAIL: <stdin>:8: unexpected command failure
> cmp stdout buildplan.yaml
diff stdout buildplan.yaml
--- stdout
+++ buildplan.yaml
@@ -0,0 +1,1 @@
+kind: BuildPlan
FAIL: <stdin>:9: stdout and buildplan.yaml differ
# want this error to go away (0.000s)
> ! stderr 'cannot convert non-concrete value string'
FAIL: <stdin>:11: unexpected match for `cannot convert non-concrete value string` found in stderr: cannot convert non-concrete value string
failed run
```

143
.github/workflows/container.yaml vendored Normal file
View File

@@ -0,0 +1,143 @@
name: Container
# Only allow actors with write permission to the repository to trigger this
# workflow.
permissions:
contents: write
on:
push:
tags:
- 'v*'
workflow_dispatch:
inputs:
git_ref:
description: 'Git ref to build (e.g., refs/tags/v1.2.3, refs/heads/main)'
required: true
type: string
jobs:
buildx:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
attestations: write
id-token: write
steps:
- name: Set tag from trigger event
id: opts
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "ref=${{ inputs.git_ref }}" >> $GITHUB_OUTPUT
else
echo "ref=${GITHUB_REF}" >> $GITHUB_OUTPUT
fi
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ steps.opts.outputs.ref }}
- name: SHA
id: sha
run: echo "sha=$(/usr/bin/git log -1 --format='%H')" >> $GITHUB_OUTPUT
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Fetch tags
run: git fetch --prune --unshallow --tags
- name: Set Tags
id: tags
run: |
echo "detail=$(/usr/bin/git describe --tags HEAD)" >> $GITHUB_OUTPUT
echo "suffix=$(test -n "$(git status --porcelain)" && echo '-dirty' || echo '')" >> $GITHUB_OUTPUT
echo "tag=$(/usr/bin/git describe --tags HEAD)$(test -n "$(git status --porcelain)" && echo '-dirty' || echo '')" >> $GITHUB_OUTPUT
- name: Login to ghcr.io
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push container images
id: build-and-push
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: |
ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}
ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}
- name: Setup Cosign to sign container images
uses: sigstore/cosign-installer@v3.7.0
- name: Sign with GitHub OIDC Token
env:
DIGEST: ${{ steps.build-and-push.outputs.digest }}
run: |
cosign sign --yes ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST}
cosign sign --yes ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST}
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Get GitHub App User ID
id: get-user-id
run: echo "user-id=$(gh api "/users/${{ steps.app-token.outputs.app-slug }}[bot]" --jq .id)" >> "$GITHUB_OUTPUT"
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
- run: |
git config --global user.name '${{ steps.app-token.outputs.app-slug }}[bot]'
git config --global user.email '${{ steps.get-user-id.outputs.user-id }}+${{ steps.app-token.outputs.app-slug }}[bot]@users.noreply.github.com'
- name: Update holos-run/holos-action
env:
IMAGE: ghcr.io/holos-run/holos:v0.102.1
VERSION: ${{ steps.tags.outputs.tag }}
USER_ID: ${{ steps.get-user-id.outputs.user-id }}
TOKEN: ${{ steps.app-token.outputs.token }}
run: |
set -euo pipefail
git clone "https://github.com/holos-run/holos-action"
cd holos-action
git remote set-url origin https://${USER_ID}:${TOKEN}@github.com/holos-run/holos-action
docker pull --quiet "${IMAGE}"
docker run -v $(pwd):/app --workdir /app --rm "${IMAGE}" \
holos cue export --out yaml action.cue -t "version=${VERSION}" > action.yml
git add action.yml
git commit -m "ci: update holos to ${VERSION} - https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" || (echo "No changes to commit"; exit 0)
git push origin HEAD:main HEAD:v0 HEAD:v1
- name: Login to quay.io
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USER }}
password: ${{ secrets.QUAY_TOKEN }}
- name: Push to quay.io
env:
DIGEST: ${{ steps.build-and-push.outputs.digest }}
run: |
# docker push quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}
docker pull --quiet ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST}
docker tag ghcr.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST} \
quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}
docker push quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}
docker pull --quiet ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST}
docker tag ghcr.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST} \
quay.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}
docker push quay.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}
- name: Sign quay.io image
env:
DIGEST: ${{ steps.build-and-push.outputs.digest }}
run: |
cosign sign --yes quay.io/holos-run/holos:${{ steps.tags.outputs.tag }}@${DIGEST}
cosign sign --yes quay.io/holos-run/holos:${{ steps.sha.outputs.sha }}${{ steps.tags.outputs.suffix }}@${DIGEST}
outputs:
tag: ${{ steps.tags.outputs.tag }}
detail: ${{ steps.tags.outputs.detail }}

View File

@@ -2,7 +2,7 @@ name: Dev Deploy
on:
push:
branches: ['main', 'dev-deploy']
branches: ['dev-deploy']
jobs:
deploy:

View File

@@ -25,6 +25,6 @@ jobs:
with:
go-version: stable
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
uses: golangci/golangci-lint-action@v7
with:
version: v1.60
version: v2.1.6

View File

@@ -1,42 +0,0 @@
---
# https://github.com/golangci/golangci-lint-action?tab=readme-ov-file#how-to-use
name: Lint
"on":
push:
branches:
- main
- test
pull_request:
types: [opened, synchronize]
permissions:
contents: read
jobs:
lint:
name: lint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: stable
## Not needed on ubuntu-latest
# - name: Install Packages
# run: sudo apt update && sudo apt -qq -y install git curl zip unzip tar bzip2 make
- name: Install Tools
run: make tools
- name: Lint
# golangci-lint runs in a separate workflow.
run: make lint -o golangci-lint

View File

@@ -35,6 +35,9 @@ jobs:
with:
go-version: stable
- name: Setup Syft
uses: anchore/sbom-action/download-syft@1ca97d9028b51809cf6d3c934c3e160716e1b605 # v0.17.5
# Necessary to run these outside of goreleaser, otherwise
# /home/runner/_work/holos/holos/internal/frontend/node_modules/.bin/protoc-gen-connect-query is not in PATH
- name: Install Tools
@@ -54,11 +57,19 @@ jobs:
- name: Git diff
run: git diff
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
version: '~> v2'
args: release --clean
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

17
.github/workflows/spelling.yaml vendored Normal file
View File

@@ -0,0 +1,17 @@
---
name: Spelling
"on":
push:
branches:
- main
- test
pull_request:
types: [opened, synchronize]
permissions:
contents: read
jobs:
cspell:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./hack/cspell

View File

@@ -13,7 +13,7 @@ permissions:
jobs:
test:
runs-on: gha-rs
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -28,19 +28,16 @@ jobs:
with:
go-version: stable
- name: Install Packages
run: sudo apt update && sudo apt -qq -y install git curl zip unzip tar bzip2 make
- name: Set up Helm
uses: azure/setup-helm@v4
with:
version: '3.17.3'
- name: Set up Kubectl
uses: azure/setup-kubectl@v3
uses: azure/setup-kubectl@v4
- name: Install Tools
run: |
set -x
make tools
- name: Install holos
run: make install
- name: Test
run: ./scripts/test

8
.gitignore vendored
View File

@@ -12,3 +12,11 @@ tmp/
/holos-k3d/
/holos-infra/
node_modules/
.tmp/
# nix
/.direnv/
result
# claude
/.claude/settings.local.json

View File

@@ -1,2 +1,20 @@
run:
timeout: 5m
version: "2"
linters:
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
paths:
- third_party$
- builtin$
- examples$
formatters:
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$

View File

@@ -6,7 +6,7 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
version: 2
before:
hooks:
@@ -50,3 +50,39 @@ changelog:
exclude:
- "^docs:"
- "^test:"
source:
enabled: true
name_template: '{{ .ProjectName }}_{{ .Version }}_source_code'
sboms:
- id: source
artifacts: source
documents:
- "{{ .ProjectName }}_{{ .Version }}_sbom.spdx.json"
brews:
- name: holos
repository:
owner: holos-run
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
homepage: "https://holos.run"
description: "Holos CLI"
dependencies:
- name: helm
type: optional
- name: kubectl
type: optional
install: |
bin.install "holos"
bash_output = Utils.safe_popen_read(bin/"holos", "completion", "bash")
(bash_completion/"holos").write bash_output
zsh_output = Utils.safe_popen_read(bin/"holos", "completion", "zsh")
(zsh_completion/"_holos").write zsh_output
fish_output = Utils.safe_popen_read(bin/"holos", "completion", "fish")
(fish_completion/"holos.fish").write fish_output
test: |
system "#{bin}/holos --version"

View File

@@ -1,13 +0,0 @@
# Refer to https://ko.build/configuration/#overriding-go-build-settings
builds:
- id: holos
dir: .
main: ./cmd/holos
env:
- GOPRIVATE=github.com/holos-run/\*
ldflags:
- -s
- -w
- -X
# Makefile provides GIT_DETAIL and GIT_SUFFIX.
- github.com/holos-run/holos/version.GitDescribe={{.Env.GIT_DETAIL}}{{.Env.GIT_SUFFIX}}

1
.nvmrc Normal file
View File

@@ -0,0 +1 @@
22

115
CLAUDE.md Normal file
View File

@@ -0,0 +1,115 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Holos is a configuration management tool for Kubernetes that implements the rendered manifests pattern using CUE. It unifies Helm charts, Kustomize bases, and raw Kubernetes manifests into a single, declarative pipeline.
### Core Flow
```
Platform → Components → BuildPlan → Generators → Transformers → Validators → Manifests
```
## Key Commands
```bash
# Development
make build # Build the binary
make install # Install binary (REQUIRED before testing holos commands)
make test # Run all tests
make fmt # Format Go code
make lint # Run linters
make coverage # Generate coverage report
# Documentation
make update-docs # Update generated docs
make website # Build the documentation website
# Usage (run 'make install' first to test code changes)
holos render platform # Render entire platform
holos render component # Render single component
holos show buildplans # Show build plans
holos init platform # Initialize new platform
```
## Architecture
### Directory Structure
- `/api/` - API definitions (v1alpha5 stable, v1alpha6 in development)
- `/cmd/` - CLI entry point
- `/internal/cli/` - Command implementations
- `/internal/component/` - Component handling logic
- `/internal/platform/` - Platform handling logic
- `/internal/generate/` - Code generation
### Key Files
- `/internal/cli/render/render.go` - Core render logic
- `/internal/component/component.go` - Component processing
- `/api/core/v1alpha*/types.go` - API type definitions
### Component Types
1. **Helm** - Wraps Helm charts
2. **Kustomize** - Wraps Kustomize bases
3. **Kubernetes** - Raw Kubernetes manifests
## CUE Patterns
Components are defined in CUE:
```cue
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "example"
Chart: {
version: "1.0.0"
repository: {
name: "example"
url: "https://charts.example.com"
}
}
}
```
## Testing
- Unit tests: `*_test.go` files colocated with source
- Integration tests: `/cmd/holos/tests/`
- Example platforms: `/internal/testutil/fixtures/`
- Run single test: `go test -run TestName ./path/to/package`
## Development Patterns
1. Error handling: Prefer `errors.Format()` from `/internal/errors/` over `fmt.Errorf()`
2. Logging: Use structured `slog`, get logger with `logger.FromContext(ctx)`
3. CLI commands: Follow Cobra patterns in `/internal/cli/`
4. CUE formatting: Always run `cue fmt` on CUE files
5. Go formatting: Always run `go fmt` on go files
6. Develop against v1alpha6 packages.
7. Commits: Use the package name as the first word in the commit, lower case. Commit without asking permission. Always run `make lint` and `make test` before committing.
## Version Management
- Version files: `/version/embedded/{major,minor,patch}`
- Bump version: `make bump`
- API versions: v1alpha5 (stable), v1alpha6 (development)
## Key Concepts
- **Platform**: Top-level configuration containing all components
- **Component**: Unit of configuration (DAG of Tasks producing deployment configs for one component)
- **TaskSet**: DAG of Tasks (Similar to how make tasks behave)
- **BuildPlan**: Instructions for building a component. Deprecated in v1alpha6, use TaskSet instead.
- **Generator**: Creates manifests (Helm, Kustomize, etc.) author schema only in v1alpha6
- **Transformer**: Modifies generated manifests, author schema only in v1alpha6
- **Validator**: Validates final manifests, author schema only in v1alpha6
## Resources
- Tutorials: `/doc/md/tutorial/`
- Platform templates: `/internal/generate/platforms/`
- Test fixtures: `/internal/testutil/fixtures/`
- Core schemas: `/api/core/` (Abstraction over low level data pipeline tasks)
- Author schemas: `/api/author/` (User facing abstractions over core Schemas)

View File

@@ -1,8 +1,38 @@
FROM quay.io/holos-run/debian:bullseye AS final
USER root
WORKDIR /app
ADD bin bin
RUN chown -R app: /app
# Kubernetes requires the user to be numeric
USER 8192
ENTRYPOINT bin/holos server
FROM registry.k8s.io/kubectl:v1.33.4 AS kubectl
# https://github.com/GoogleContainerTools/distroless
FROM golang:1.24 AS build
WORKDIR /go/src/app
COPY . .
RUN CGO_ENABLED=0 make install
RUN CGO_ENABLED=0 go install sigs.k8s.io/kustomize/kustomize/v5
# Install helm to /usr/local/bin/helm
# https://helm.sh/docs/intro/install/#from-script
# https://holos.run/docs/v1alpha5/tutorial/setup/#dependencies
RUN curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 \
&& chmod 700 get_helm.sh \
&& DESIRED_VERSION=v3.16.2 ./get_helm.sh \
&& rm -f get_helm.sh
COPY --from=kubectl /bin/kubectl /usr/local/bin/
# Use debian slim instead of distroless to get package management.
FROM public.ecr.aws/docker/library/debian:13-slim AS final
COPY --from=build \
/go/bin/holos \
/go/bin/kustomize \
/usr/local/bin/kubectl \
/usr/local/bin/helm \
/bin/
# Extra packages
# git - https://github.com/holos-run/holos/issues/440
RUN apt update && \
apt install -y --no-install-recommends git ca-certificates && \
apt clean && \
rm -rf /var/lib/apt/lists/*
# Usage: docker run -v $(pwd):/app --workdir /app --rm -it quay.io/holos-run/holos holos render platform
CMD ["/bin/holos"]

View File

@@ -32,17 +32,20 @@ bump: bumppatch
.PHONY: bumppatch
bumppatch: ## Bump the patch version.
scripts/bump patch
HOLOS_UPDATE_SCRIPTS=1 scripts/test
.PHONY: bumpminor
bumpminor: ## Bump the minor version.
scripts/bump minor
scripts/bump patch 0
HOLOS_UPDATE_SCRIPTS=1 scripts/test
.PHONY: bumpmajor
bumpmajor: ## Bump the major version.
scripts/bump major
scripts/bump minor 0
scripts/bump patch 0
HOLOS_UPDATE_SCRIPTS=1 scripts/test
.PHONY: show-version
show-version: ## Print the full version.
@@ -58,8 +61,8 @@ tidy: ## Tidy go module.
.PHONY: fmt
fmt: ## Format code.
cd internal/generate/platforms && cue fmt ./...
go fmt ./...
cue fmt ./...
.PHONY: vet
vet: ## Vet Go code.
@@ -75,10 +78,11 @@ build: ## Build holos executable.
@echo "GOPATH=${GOPATH}"
go build -trimpath -o bin/$(BIN_NAME) -ldflags $(LD_FLAGS) $(REPO_PATH)/cmd/$(BIN_NAME)
linux: ## Build holos executable for tilt.
@echo "building ${BIN_NAME}.linux ${VERSION}"
.PHONY: debug
debug: ## Build debug executable.
@echo "building ${BIN_NAME}-debug ${VERSION}"
@echo "GOPATH=${GOPATH}"
GOOS=linux go build -trimpath -o bin/$(BIN_NAME).linux -ldflags $(LD_FLAGS) $(REPO_PATH)/cmd/$(BIN_NAME)
go build -o bin/$(BIN_NAME)-debug $(REPO_PATH)/cmd/$(BIN_NAME)
.PHONY: install
install: build ## Install holos to GOPATH/bin
@@ -97,9 +101,7 @@ golangci-lint:
golangci-lint run
.PHONY: lint
lint: golangci-lint ## Run linters.
buf lint
cd internal/frontend/holos && ng lint
lint: vet golangci-lint ## Run linters.
./hack/cspell
.PHONY: coverage
@@ -111,45 +113,32 @@ snapshot: ## Go release snapshot
goreleaser release --snapshot --clean
.PHONY: tools
tools: go-deps frontend-deps website-deps ## install tool dependencies
tools: go-deps website-deps ## install tool dependencies
.PHONY: go-deps
go-deps: ## tool versions pinned in tools.go
go install cuelang.org/go/cmd/cue
go install github.com/bufbuild/buf/cmd/buf
go install github.com/fullstorydev/grpcurl/cmd/grpcurl
go install google.golang.org/protobuf/cmd/protoc-gen-go
go install connectrpc.com/connect/cmd/protoc-gen-connect-go
go install honnef.co/go/tools/cmd/staticcheck
go install golang.org/x/tools/cmd/godoc
go install github.com/princjef/gomarkdoc/cmd/gomarkdoc
go install github.com/google/ko
go install github.com/rogpeppe/go-internal/cmd/testscript
# curl https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | bash
.PHONY: frontend-deps
frontend-deps: ## Install Angular deps for go generate
cd internal/frontend/holos && npm install
.PHONY: website-deps
website-deps: ## Install Docusaurus deps for go generate
cd doc/website && npm install
.PHONY: image # refer to .ko.yaml as well
image: ## Container image build for workflows/publish.yaml
KO_DOCKER_REPO=$(DOCKER_REPO) GIT_DETAIL=$(GIT_DETAIL) GIT_SUFFIX=$(GIT_SUFFIX) ko build --platform=all --bare ./cmd/holos --tags $(GIT_DETAIL)$(GIT_SUFFIX) --tags latest
.PHONY: prod-deploy
prod-deploy: install image ## deploy to PROD
GIT_DETAIL=$(GIT_DETAIL) GIT_SUFFIX=$(GIT_SUFFIX) bash ./hack/deploy
.PHONY: dev-deploy
dev-deploy: install image ## deploy to dev
GIT_DETAIL=$(GIT_DETAIL) GIT_SUFFIX=$(GIT_SUFFIX) bash ./hack/deploy-dev
.PHONY: website
website: ## Build website
./hack/build-website
.PHONY: unity
unity: ## https://cuelabs.dev/unity/
./scripts/unity
.PHONY: update-docs
update-docs: ## Update doc examples
HOLOS_UPDATE_SCRIPTS=1 go test -v ./doc/md/...
HOLOS_UPDATE_SCRIPTS=1 go test -v ./doc/website/versioned_docs/...
.PHONY: help
help: ## Display this help menu.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)

129
README.md
View File

@@ -1,35 +1,130 @@
## Holos - A Holistic Development Platform
# Holos
<img width="50%"
align="right"
style="display: block; margin: 40px auto;"
src="https://openinfrastructure.co/blog/2016/02/27/logo/logorectangle.png">
Building and maintaining a software development platform is a complex and time
consuming endeavour. Organizations often dedicate a team of 3-4 who need 6-12
months to build the platform.
[Holos] is a configuration management tool for Kubernetes implementing the
[rendered manifests pattern]. It handles configurations ranging from single
resources to multi-cluster platforms across regions.
Holos is a tool and a reference platform to reduce the complexity and speed up
the process of building a modern, cloud native software development platform.
Key components:
- Platform schemas defining component integration
- Building blocks unifying Helm, Kustomize and Kubernetes configs with CUE
- BuildPlan pipeline for generating, transforming and validating manifests
- **Accelerate new projects** - Reduce time to market and operational complexity by starting your new project on top of the Holos reference platform.
- **Modernize existing projects** - Incrementally incorporate your existing platform services into Holos for simpler integration.
- **Unified configuration model** - Increase safety and reduce the risk of config changes with CUE.
- **First class Helm and Kustomize support** - Leverage and reuse your existing investment in existing configuration tools such as Helm and Kustomize.
- **Modern Authentication and Authorization** - Holos seamlessly integrates platform identity and access management with zero-trust beyond corp style authorization policy.
```mermaid
---
title: Rendering Overview
---
graph LR
Platform[<a href="https://holos.run/docs/v1alpha5/api/author/#Platform">Platform</a>]
Component[<a href="https://holos.run/docs/v1alpha5/api/author/#ComponentConfig">Components</a>]
## Quick Installation
Helm[<a href="https://holos.run/docs/v1alpha5/api/author/#Helm">Helm</a>]
Kustomize[<a href="https://holos.run/docs/v1alpha5/api/author/#Kustomize">Kustomize</a>]
Kubernetes[<a href="https://holos.run/docs/v1alpha5/api/author/#Kubernetes">Kubernetes</a>]
```console
go install github.com/holos-run/holos/cmd/holos@latest
BuildPlan[<a href="https://holos.run/docs/v1alpha5/api/core/#BuildPlan">BuildPlan</a>]
ResourcesArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="https://holos.run/docs/v1alpha5/api/core/#Artifact">GitOps<br/>Artifact</a>]
Generators[<a href="https://holos.run/docs/v1alpha5/api/core/#Generator">Generators</a>]
Transformers[<a href="https://holos.run/docs/v1alpha5/api/core/#Transformer">Transformers</a>]
Validators[<a href="https://holos.run/docs/v1alpha5/api/core/#Validator">Validators</a>]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Validators --> Files
```
## Docs and Support
## Setup
The documentation for developing and using Holos is available at: https://holos.run
```shell
brew install holos-run/tap/holos
```
For discussion and support, [open a discussion](https://github.com/orgs/holos-run/discussions/new/choose).
Refer to [setup] for other installation methods and dependencies.
## Example
See our [tutorial] for a complete hello world example.
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
## Organizational Role
Platform engineers use Holos to generate Kubernetes manifests, both locally and
in CI pipelines. The manifests are committed to version control and deployed via
GitOps tools like ArgoCD or Flux.
Holos integrates seamlessly with existing Helm charts, Kustomize bases, and
other version-controlled configurations.
## Advantages of Holos
### Safe
Holos leverages [CUE] for strong typing and validation of configuration data,
ensuring consistent output from Helm and other tools.
### Consistent
A unified pipeline processes all configurations - whether from CUE, Helm, or
Kustomize - through the same well-defined stages.
### Flexible
Composable building blocks for generation, transformation, validation and
integration let teams assemble workflows that match their needs.
The core is intentionally unopinionated about platform configuration patterns.
Common needs like environments and clusters are provided as customizable
[topics] recipes rather than enforced structures.
## Getting Help
Get support through our [Discord] channel or [GitHub discussions]. Configuration
challenges arise at all experience levels - we welcome your questions and are
here to help.
## License
Holos is licensed under Apache 2.0 as found in the [LICENSE file](LICENSE).
[Holos]: https://holos.run/docs/overview/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[CUE]: https://cuelang.org/
[Discord]: https://discord.gg/JgDVbNpye7
[GitHub discussions]: https://github.com/holos-run/holos/discussions
[Why CUE for Configuration]: https://holos.run/blog/why-cue-for-configuration/
[tutorial]: https://holos.run/docs/overview/
[setup]: https://holos.run/docs/setup/
[topics]: https://holos.run/docs/topics/

110
Tiltfile
View File

@@ -1,110 +0,0 @@
# -*- mode: Python -*-
# This Tiltfile manages a Go project with live reload in Kubernetes
listen_port = 3000
metrics_port = 9090
# Use our wrapper to set the kube namespace
if os.getenv('TILT_WRAPPER') != '1':
fail("could not run, ./hack/tilt/bin/tilt was not used to start tilt")
# Resource ids
holos_backend = 'Holos Server'
compile_id = 'Go Build'
# Default Registry.
# See: https://github.com/tilt-dev/tilt.build/blob/master/docs/choosing_clusters.md#manual-configuration
# Note, Tilt will append the image name to the registry uri path
# default_registry('{account}.dkr.ecr.{region}.amazonaws.com/holos-run/holos'.format(account=aws_account, region=aws_region))
# Set a name prefix specific to the user. Multiple developers share the tilt-holos namespace.
developer = os.getenv('USER')
holos_server = 'holos'
# We always develop against the k3d-workload cluster
os.putenv('KUBECONFIG', os.path.abspath('./hack/tilt/kubeconfig'))
# Extensions are open-source, pre-packaged functions that extend Tilt
#
# More info: https://github.com/tilt-dev/tilt-extensions
# More info: https://docs.tilt.dev/extensions.html
load('ext://restart_process', 'docker_build_with_restart')
load('ext://k8s_attach', 'k8s_attach')
load('ext://git_resource', 'git_checkout')
load('ext://uibutton', 'cmd_button')
# Paths edited by the developer Tilt watches to trigger compilation.
# Generated files should be excluded to avoid an infinite build loop.
developer_paths = [
'./cmd',
'./internal/server',
'./internal/ent/schema',
'./frontend/package-lock.json',
'./frontend/src',
'./go.mod',
'./pkg',
'./service/holos',
]
# Builds the holos executable GOOS=linux
local_resource(compile_id, 'make linux', deps=developer_paths)
# Build Docker image
# Tilt will automatically associate image builds with the resource(s)
# that reference them (e.g. via Kubernetes or Docker Compose YAML).
#
# More info: https://docs.tilt.dev/api.html#api.docker_build
#
docker_build_with_restart(
'k3d-registry.holos.localhost:5100/holos',
context='.',
entrypoint=[
'/app/bin/holos.linux',
'server',
'--log-format=text',
'--oidc-issuer=https://login.holos.run',
'--oidc-audience=275804490387516853@holos_quickstart', # auth proxy
'--oidc-audience=270319630705329162@holos_platform', # holos cli
],
dockerfile='./Dockerfile',
only=['./bin'],
# (Recommended) Updating a running container in-place
# https://docs.tilt.dev/live_update_reference.html
live_update=[
# Sync files from host to container
sync('./bin/', '/app/bin/'),
],
)
# Troubleshooting
def resource_name(id):
print('resource: {}'.format(id))
return id.name
workload_to_resource_function(resource_name)
# Customize a Kubernetes resource
# By default, Kubernetes resource names are automatically assigned
# based on objects in the YAML manifests, e.g. Deployment name.
#
# Tilt strives for sane defaults, so calling k8s_resource is
# optional, and you only need to pass the arguments you want to
# override.
#
# More info: https://docs.tilt.dev/api.html#api.k8s_resource
#
k8s_yaml(blob(str(read_file('./hack/tilt/k8s/dev-holos-app/deployment.yaml'))))
# Backend server process
k8s_resource(
workload=holos_server,
new_name=holos_backend,
objects=[],
resource_deps=[compile_id],
links=[
link('https://app.holos.localhost/ui/'.format(developer), "Holos Web UI")
],
)
# Database
print("✨ Tiltfile evaluated")

View File

@@ -0,0 +1,155 @@
// Package author contains a standard set of schemas for component authors to
// generate common [core] BuildPlans.
//
// Holos values stability, flexibility, and composition. This package
// intentionally defines only the minimal necessary set of structures.
// Component authors are encouraged to define their own structures building on
// our example [topics].
//
// The Holos Maintainers may add definitions to this package if the community
// identifies nearly all users must define the exact same structure. Otherwise,
// definitions should be added as a customizable example in [topics].
//
// For example, structures representing a cluster and environment almost always
// need to be defined. Their definition varies from one organization to the
// next. Therefore, customizable definitions for a cluster and environment are
// best maintained in [topics], not standardized in this package.
//
// [core]: https://holos.run/docs/api/core/
// [topics]: https://holos.run/docs/topics/
package author
import core "github.com/holos-run/holos/api/core/v1alpha5"
//go:generate ../../../hack/gendoc
// Platform assembles a core Platform in the Resource field for the holos render
// platform command. Use the Components field to register components with the
// platform.
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// ComponentConfig represents the configuration common to all kinds of
// components for use with the holos render component command. All component
// kinds may be transformed with [kustomize] configured with the
// [KustomizeConfig] field.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the kustomize configuration.
KustomizeConfig KustomizeConfig
// Validators represent checks that must pass for output to be written.
Validators map[NameLabel]core.Validator
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
// Helm assembles a BuildPlan rendering a helm chart. Useful to mix in
// additional resources from CUE and transform the helm output with kustomize.
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kubernetes assembles a BuildPlan containing inline resources exported from
// CUE.
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize assembles a BuildPlan rendering manifests from a [kustomize]
// kustomization.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// KustomizeConfig represents the configuration for [kustomize] post processing.
// Use the Files field to mix in plain manifest files located in the component
// directory. Use the Resources field to mix in manifests from network urls.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
// NameLabel represents the common use case of converting a struct to a list
// where the name field of each value unifies with the field name of the outer
// struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,5 @@
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---

View File

@@ -0,0 +1,155 @@
// Package author contains a standard set of schemas for component authors to
// generate common [core] BuildPlans.
//
// Holos values stability, flexibility, and composition. This package
// intentionally defines only the minimal necessary set of structures.
// Component authors are encouraged to define their own structures building on
// our example [topics].
//
// The Holos Maintainers may add definitions to this package if the community
// identifies nearly all users must define the exact same structure. Otherwise,
// definitions should be added as a customizable example in [topics].
//
// For example, structures representing a cluster and environment almost always
// need to be defined. Their definition varies from one organization to the
// next. Therefore, customizable definitions for a cluster and environment are
// best maintained in [topics], not standardized in this package.
//
// [core]: https://holos.run/docs/api/core/
// [topics]: https://holos.run/docs/topics/
package author
import core "github.com/holos-run/holos/api/core/v1alpha6"
//go:generate ../../../hack/gendoc
// Platform assembles a core Platform in the Resource field for the holos render
// platform command. Use the Components field to register components with the
// platform.
type Platform struct {
Name string `json:"name" yaml:"name" cue:"string | *\"default\""`
Components map[NameLabel]core.Component `json:"components" yaml:"components"`
Resource core.Platform `json:"resource" yaml:"resource"`
}
// ComponentConfig represents the configuration common to all kinds of
// components for use with the holos render component command. All component
// kinds may be transformed with [kustomize] configured with the
// [KustomizeConfig] field.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the kustomize configuration.
KustomizeConfig KustomizeConfig
// Validators represent checks that must pass for output to be written.
Validators map[NameLabel]core.Validator
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
// Helm assembles a BuildPlan rendering a helm chart. Useful to mix in
// additional resources from CUE and transform the helm output with kustomize.
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kubernetes assembles a BuildPlan containing inline resources exported from
// CUE.
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize assembles a BuildPlan rendering manifests from a [kustomize]
// kustomization.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// KustomizeConfig represents the configuration for [kustomize] post processing.
// Use the Files field to mix in plain manifest files located in the component
// directory. Use the Resources field to mix in manifests from network urls.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
// NameLabel represents the common use case of converting a struct to a list
// where the name field of each value unifies with the field name of the outer
// struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,5 @@
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---

View File

@@ -1,44 +0,0 @@
package v1alpha2
import "google.golang.org/protobuf/types/known/structpb"
// Label is an arbitrary unique identifier internal to holos itself. The holos
// cli is expected to never write a Label value to rendered output files,
// therefore use a [Label] then the identifier must be unique and internal.
// Defined as a type for clarity and type checking.
//
// A Label is useful to convert a CUE struct to a list, for example producing a list of [APIObject] resources from an [APIObjectMap]. A CUE struct using
// Label keys is guaranteed to not lose data when rendering output because a
// Label is expected to never be written to the final output.
type Label string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
type Kind string
// APIObject represents the most basic generic form of a single kubernetes api
// object. Represented as a JSON object internally for compatibility between
// tools, for example loading from CUE.
type APIObject structpb.Struct
// APIObjectMap represents the marshalled yaml representation of kubernetes api
// objects. Do not produce an APIObjectMap directly, instead use [APIObjects]
// to produce the marshalled yaml representation from CUE data, then provide the
// result to [HolosComponent].
type APIObjectMap map[Kind]map[Label]string
// APIObjects represents Kubernetes API objects defined directly from CUE code.
// Useful to mix in resources to any kind of [HolosComponent], for example
// adding an ExternalSecret resource to a [HelmChart].
//
// [Kind] must be the resource kind, e.g. Deployment or Service.
//
// [Label] is an arbitrary internal identifier to uniquely identify the resource
// within the context of a `holos` command. Holos will never write the
// intermediate label to rendered output.
//
// Refer to [HolosComponent] which accepts an [APIObjectMap] field provided by
// [APIObjects].
type APIObjects struct {
APIObjects map[Kind]map[Label]APIObject `json:"apiObjects"`
APIObjectMap APIObjectMap `json:"apiObjectMap"`
}

View File

@@ -1,96 +0,0 @@
package v1alpha2
// FilePath represents a file path.
type FilePath string
// FileContent represents file contents.
type FileContent string
// FileContentMap represents a mapping of file paths to file contents. Paths
// are relative to the `holos` output "deploy" directory, and may contain
// sub-directories.
type FileContentMap map[FilePath]FileContent
// BuildPlan represents a build plan for the holos cli to execute. The purpose
// of a BuildPlan is to define one or more [HolosComponent] kinds. For example a
// [HelmChart], [KustomizeBuild], or [KubernetesObjects].
//
// A BuildPlan usually has an additional empty [KubernetesObjects] for the
// purpose of using the [HolosComponent] DeployFiles field to deploy an ArgoCD
// or Flux gitops resource for the holos component.
type BuildPlan struct {
Kind string `json:"kind" cue:"\"BuildPlan\""`
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
Spec BuildPlanSpec `json:"spec"`
}
// BuildPlanSpec represents the specification of the build plan.
type BuildPlanSpec struct {
// Disabled causes the holos cli to take no action over the [BuildPlan].
Disabled bool `json:"disabled,omitempty"`
// Components represents multiple [HolosComponent] kinds to manage.
Components BuildPlanComponents `json:"components,omitempty"`
}
type BuildPlanComponents struct {
Resources map[Label]KubernetesObjects `json:"resources,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty"`
HelmChartList []HelmChart `json:"helmChartList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty"`
}
// HolosComponent defines the fields common to all holos component kinds. Every
// holos component kind should embed HolosComponent.
type HolosComponent struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind"`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
// Metadata represents data about the holos component such as the Name.
Metadata Metadata `json:"metadata"`
// APIObjectMap holds the marshalled representation of api objects. Useful to
// mix in resources to each HolosComponent type, for example adding an
// ExternalSecret to a HelmChart HolosComponent. Refer to [APIObjects].
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty"`
// DeployFiles represents file paths relative to the cluster deploy directory
// with the value representing the file content. Intended for defining the
// ArgoCD Application resource or Flux Kustomization resource from within CUE,
// but may be used to render any file related to the build plan from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:"kustomize,omitempty"`
// Skip causes holos to take no action regarding this component.
Skip bool `json:"skip" cue:"bool | *false"`
}
// Metadata represents data about the holos component such as the Name.
type Metadata struct {
// Name represents the name of the holos component.
Name string `json:"name"`
// Namespace is the primary namespace of the holos component. A holos
// component may manage resources in multiple namespaces, in this case
// consider setting the component namespace to default.
//
// This field is optional because not all resources require a namespace,
// particularly CRD's and DeployFiles functionality.
// +optional
Namespace string `json:"namespace,omitempty"`
}
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process a [KustomizeBuild] [HolosComponent] which represents raw yaml
// file resources in a holos component directory.
// 2. Post process a [HelmChart] [HolosComponent] to inject istio, patch jobs,
// add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty"`
}

View File

@@ -1,11 +0,0 @@
package v1alpha2
const (
APIVersion = "v1alpha2"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,44 +0,0 @@
package v1alpha2
import "google.golang.org/protobuf/types/known/structpb"
type PlatformMetadata struct {
// Name represents the Platform name.
Name string `json:"name"`
}
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
// Metadata represents data about the object such as the Name.
Metadata PlatformMetadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
// PlatformSpec represents the specification of a Platform. Think of a platform
// specification as a list of platform components to apply to a list of
// kubernetes clusters combined with the user-specified Platform Model.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `json:"model"`
// Components represents a list of holos components to manage.
Components []PlatformSpecComponent `json:"components"`
}
// PlatformSpecComponent represents a holos component to build or render.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path"`
// Cluster is the cluster name to provide when rendering the component.
Cluster string `json:"cluster"`
}

View File

@@ -1,26 +0,0 @@
// Package v1alpha2 contains the core API contract between the holos cli and CUE
// configuration code. Platform designers, operators, and software developers
// use this API to write configuration in CUE which `holos` loads. The overall
// shape of the API defines imperative actions `holos` should carry out to
// render the complete yaml that represents a Platform.
//
// [Platform] defines the complete configuration of a platform. With the holos
// reference platform this takes the shape of one management cluster and at
// least two workload cluster. Each cluster has multiple [HolosComponent]
// resources applied to it.
//
// Each holos component path, e.g. `components/namespaces` produces exactly one
// [BuildPlan] which in turn contains a set of [HolosComponent] kinds.
//
// The primary kinds of [HolosComponent] are:
//
// 1. [HelmChart] to render config from a helm chart.
// 2. [KustomizeBuild] to render config from [Kustomize]
// 3. [KubernetesObjects] to render [APIObjects] defined directly in CUE
// configuration.
//
// Note that Holos operates as a data pipeline, so the output of a [HelmChart]
// may be provided to [Kustomize] for post-processing.
package v1alpha2
//go:generate ../../../hack/gendoc

View File

@@ -1,38 +0,0 @@
package v1alpha2
// HelmChart represents a holos component which wraps around an upstream helm
// chart. Holos orchestrates helm by providing values obtained from CUE,
// renders the output using `helm template`, then post-processes the helm output
// yaml using the general functionality provided by [HolosComponent], for
// example [Kustomize] post-rendering and mixing in additional kubernetes api
// objects.
type HelmChart struct {
HolosComponent `json:",inline"`
Kind string `json:"kind" cue:"\"HelmChart\""`
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// ValuesContent represents the values.yaml file holos passes to the `helm
// template` command.
ValuesContent string `json:"valuesContent"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks" cue:"bool | *false"`
}
// Chart represents a helm chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
// Repository represents a helm chart repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}

View File

@@ -1,10 +0,0 @@
package v1alpha2
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents a [HolosComponent] composed of Kubernetes API
// objects provided directly from CUE using [APIObjects].
type KubernetesObjects struct {
HolosComponent `json:",inline"`
Kind string `json:"kind" cue:"\"KubernetesObjects\""`
}

View File

@@ -1,8 +0,0 @@
package v1alpha2
// KustomizeBuild represents a [HolosComponent] that renders plain yaml files in
// the holos component directory using `kubectl kustomize build`.
type KustomizeBuild struct {
HolosComponent `json:",inline"`
Kind string `json:"kind" cue:"\"KustomizeBuild\""`
}

View File

@@ -1,47 +0,0 @@
package v1alpha3
import "google.golang.org/protobuf/types/known/structpb"
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a [InternalLabel] when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
//
// A InternalLabel is useful to convert a CUE struct to a list, for example
// producing a list of [APIObject] resources from an [APIObjectMap]. A CUE
// struct using InternalLabel keys is guaranteed to not lose data when rendering
// output because a InternalLabel is expected to never be written to the final
// output.
type InternalLabel string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type
// checking.
type Kind string
// APIObject represents the most basic generic form of a single kubernetes api
// object. Represented as a JSON object internally for compatibility between
// tools, for example loading from CUE.
type APIObject structpb.Struct
// APIObjectMap represents the marshalled yaml representation of kubernetes api
// objects. Do not produce an APIObjectMap directly, instead use [APIObjects]
// to produce the marshalled yaml representation from CUE data, then provide the
// result to [Component].
type APIObjectMap map[Kind]map[InternalLabel]string
// APIObjects represents Kubernetes API objects defined directly from CUE code.
// Useful to mix in resources to any kind of [Component], for example
// adding an ExternalSecret resource to a [HelmChart].
//
// [Kind] must be the resource kind, e.g. Deployment or Service.
//
// [InternalLabel] is an arbitrary internal identifier to uniquely identify the resource
// within the context of a `holos` command. Holos will never write the
// intermediate label to rendered output.
//
// Refer to [Component] which accepts an [APIObjectMap] field provided by
// [APIObjects].
type APIObjects struct {
APIObjects map[Kind]map[InternalLabel]APIObject `json:"apiObjects"`
APIObjectMap APIObjectMap `json:"apiObjectMap"`
}

View File

@@ -1,52 +0,0 @@
package v1alpha3
// FilePath represents a file path.
type FilePath string
// FileContent represents file contents.
type FileContent string
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// BuildPlan represents a build plan for the holos cli to execute. The purpose
// of a BuildPlan is to define one or more [Component] kinds. For example a
// [HelmChart], [KustomizeBuild], or [KubernetesObjects].
//
// A BuildPlan usually has an additional empty [KubernetesObjects] for the
// purpose of using the [Component] DeployFiles field to deploy an ArgoCD
// or Flux gitops resource for the holos component.
type BuildPlan struct {
Kind string `json:"kind" cue:"\"BuildPlan\""`
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha3\""`
Spec BuildPlanSpec `json:"spec"`
}
// BuildPlanSpec represents the specification of the build plan.
type BuildPlanSpec struct {
// Disabled causes the holos cli to take no action over the [BuildPlan].
Disabled bool `json:"disabled,omitempty"`
// Components represents multiple [HolosComponent] kinds to manage.
Components BuildPlanComponents `json:"components,omitempty"`
}
type BuildPlanComponents struct {
Resources map[InternalLabel]KubernetesObjects `json:"resources,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty"`
HelmChartList []HelmChart `json:"helmChartList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty"`
}
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process a [KustomizeBuild] [Component] which represents raw yaml
// file resources in a holos component directory.
// 2. Post process a [HelmChart] [Component] to inject istio, patch jobs,
// add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty"`
}

View File

@@ -1,43 +0,0 @@
package v1alpha3
// Component defines the fields common to all holos component kinds. Every
// holos component kind should embed Component.
type Component struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind"`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"\"v1alpha3\""`
// Metadata represents data about the holos component such as the Name.
Metadata Metadata `json:"metadata"`
// APIObjectMap holds the marshalled representation of api objects. Useful to
// mix in resources to each Component type, for example adding an
// ExternalSecret to a [HelmChart] Component. Refer to [APIObjects].
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty"`
// DeployFiles represents file paths relative to the cluster deploy directory
// with the value representing the file content. Intended for defining the
// ArgoCD Application resource or Flux Kustomization resource from within CUE,
// but may be used to render any file related to the build plan from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:"kustomize,omitempty"`
// Skip causes holos to take no action regarding this component.
Skip bool `json:"skip" cue:"bool | *false"`
}
// Metadata represents data about the object such as the Name.
type Metadata struct {
// Name represents the name of the holos component.
Name string `json:"name"`
// Namespace is the primary namespace of the holos component. A holos
// component may manage resources in multiple namespaces, in this case
// consider setting the component namespace to default.
//
// This field is optional because not all resources require a namespace,
// particularly CRDs and DeployFiles functionality.
// +optional
Namespace string `json:"namespace,omitempty"`
}

View File

@@ -1,11 +0,0 @@
package v1alpha3
const (
APIVersion = "v1alpha3"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,26 +0,0 @@
// Package v1alpha3 contains the core API contract between the holos cli and CUE
// configuration code. Platform designers, operators, and software developers
// use this API to write configuration in CUE which `holos` loads. The overall
// shape of the API defines imperative actions `holos` should carry out to
// render the complete yaml that represents a Platform.
//
// [Platform] defines the complete configuration of a platform. With the holos
// reference platform this takes the shape of one management cluster and at
// least two workload cluster. Each cluster has multiple [Component]
// resources applied to it.
//
// Each holos component path, e.g. `components/namespaces` produces exactly one
// [BuildPlan] which in turn contains a set of [Component] kinds.
//
// The primary kinds of [Component] are:
//
// 1. [HelmChart] to render config from a helm chart.
// 2. [KustomizeBuild] to render config from [Kustomize]
// 3. [KubernetesObjects] to render [APIObjects] defined directly in CUE
// configuration.
//
// Note that Holos operates as a data pipeline, so the output of a [HelmChart]
// may be provided to [Kustomize] for post-processing.
package v1alpha3
//go:generate ../../../hack/gendoc

View File

@@ -1,38 +0,0 @@
package v1alpha3
// HelmChart represents a holos component which wraps around an upstream helm
// chart. Holos orchestrates helm by providing values obtained from CUE,
// renders the output using `helm template`, then post-processes the helm output
// yaml using the general functionality provided by [Component], for
// example [Kustomize] post-rendering and mixing in additional kubernetes api
// objects.
type HelmChart struct {
Component `json:",inline"`
Kind string `json:"kind" cue:"\"HelmChart\""`
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// ValuesContent represents the values.yaml file holos passes to the `helm
// template` command.
ValuesContent string `json:"valuesContent"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks" cue:"bool | *false"`
}
// Chart represents a helm chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
// Repository represents a helm chart repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}

View File

@@ -1,10 +0,0 @@
package v1alpha3
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents a [Component] composed of Kubernetes API
// objects provided directly from CUE using [APIObjects].
type KubernetesObjects struct {
Component `json:",inline"`
Kind string `json:"kind" cue:"\"KubernetesObjects\""`
}

View File

@@ -1,8 +0,0 @@
package v1alpha3
// KustomizeBuild represents a [Component] that renders plain yaml files in
// the holos component directory using `kubectl kustomize build`.
type KustomizeBuild struct {
Component `json:",inline"`
Kind string `json:"kind" cue:"\"KustomizeBuild\""`
}

View File

@@ -1,44 +0,0 @@
package v1alpha3
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha3\""`
// Metadata represents data about the object such as the Name.
Metadata PlatformMetadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
type PlatformMetadata struct {
// Name represents the Platform name.
Name string `json:"name"`
}
// PlatformSpec represents the specification of a Platform. Think of a platform
// specification as a list of platform components to apply to a list of
// kubernetes clusters combined with the user-specified Platform Model.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `json:"model" cue:"{...}"`
// Components represents a list of holos components to manage.
Components []PlatformSpecComponent `json:"components"`
}
// PlatformSpecComponent represents a holos component to build or render.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path"`
// Cluster is the cluster name to provide when rendering the component.
Cluster string `json:"cluster"`
}

View File

@@ -0,0 +1,5 @@
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---

388
api/core/v1alpha5/types.go Normal file
View File

@@ -0,0 +1,388 @@
// Package core contains schemas for a [Platform] and [BuildPlan]. Holos takes
// a [Platform] as input, then iterates over each [Component] to produce a
// [BuildPlan]. Holos evaluates the [BuildPlan] to produce fully rendered
// manifests, each an [Artifact].
package core
// BuildContextTag represents the cue tag holos render component uses to inject
// the json representation of a [BuildContext] for use in a BuildPlan.
const BuildContextTag string = "holos_build_context"
// ComponentNameTag represents the cue tag holos uses to inject a [Component]
// name from the holos render platform command to the holos render component
// command.
const ComponentNameTag string = "holos_component_name"
// ComponentPathTag represents the cue tag holos uses to inject a [Component]
// path relative to the cue module root from the holos render platform command
// to the holos render component command.
const ComponentPathTag string = "holos_component_path"
// ComponentLabelsTag represents the cue tag holos uses to inject the json
// representation of [Component] metadata labels from the holos render platform
// command to the holos render component command.
const ComponentLabelsTag string = "holos_component_labels"
// ComponentAnnotationsTag represents the tag holos uses to inject the json
// representation of [Component] metadata annotations from the holos render
// platform command to the holos render component command.
const ComponentAnnotationsTag = "holos_component_annotations"
//go:generate ../../../hack/gendoc
// BuildPlan represents an implementation of the [rendered manifest pattern].
// Holos processes a BuildPlan to produce one or more [Artifact] output files.
// BuildPlan artifact files usually contain Kubernetes manifests, but they may
// have any content.
//
// A BuildPlan usually produces two artifacts. One artifact contains a manifest
// of resources. A second artifact contains a GitOps resource to manage the
// first, usually an ArgoCD Application resource.
//
// Holos uses CUE to construct a BuildPlan. A future enhancement will support
// user defined executables providing a BuildPlan to Holos in the style of an
// [external credential provider].
//
// [rendered manifest pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Validators []Validator `json:"validators,omitempty" yaml:"validators,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// Each Generator in an [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents Kubernetes resources. Most commonly used to mix
// resources into the [BuildPlan] generated from CUE, but may be generated from
// elsewhere.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source" yaml:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// ValueFile represents one Helm value file produced from CUE.
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
//
// The Auth field is useful to configure http basic authentication to the Helm
// repository. Holos gets the username and password from the environment
// variables represented by the Auth field.
type Repository struct {
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
Auth Auth `json:"auth,omitempty" yaml:"auth,omitempty"`
}
// Auth represents environment variable names containing auth credentials.
type Auth struct {
Username AuthSource `json:"username" yaml:"username"`
Password AuthSource `json:"password" yaml:"password"`
}
// AuthSource represents a source for the value of an [Auth] field.
type AuthSource struct {
Value string `json:"value,omitempty" yaml:"value,omitempty"`
FromEnv string `json:"fromEnv,omitempty" yaml:"fromEnv,omitempty"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
// outputs into one output. [Kustomize] is the most commonly used transformer.
// A simple [Join] is also supported for use with plain manifest files.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// FileContent represents file contents.
type FileContent string
// Validator validates files. Useful to validate an [Artifact] prior to writing
// it out to the final destination. Holos may execute validators concurrently.
// See the [validators] tutorial for an end to end example.
//
// [validators]: https://holos.run/docs/v1alpha5/tutorial/validators/
type Validator struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Command\""`
// Inputs represents the files to validate. Usually the final Artifact.
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Command represents a validation command. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Command represents a command vetting one or more artifacts. Holos appends
// fully qualified input file paths to the end of the args list, then executes
// the command. Inputs are written into a temporary directory prior to
// executing the command and removed afterwards.
type Command struct {
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
}
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `app.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
// collection and integrates the components together into a holistic platform.
// Holos iterates over the [Component] collection producing a [BuildPlan] for
// each, which holos then executes to render manifests.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components" yaml:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
// from a path containing parameterized CUE configuration.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path" yaml:"path"`
// Instances represents additional cue instance paths to unify with Path.
// Useful to unify data files into a component BuildPlan. Added in holos
// 0.101.7.
Instances []Instance `json:"instances,omitempty" yaml:"instances,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `app.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Instance represents a data instance to unify with the configuration.
//
// Useful to unify json and yaml files with cue configuration files for
// integration with other tools. For example, executing holos render platform
// from a pull request workflow after [Kargo] executes the [yaml update] and
// [git wait for pr] promotion steps.
//
// [Kargo]: https://docs.kargo.io/
// [yaml update]: https://docs.kargo.io/references/promotion-steps#yaml-update
// [git wait for pr]: https://docs.kargo.io/references/promotion-steps#git-wait-for-pr
type Instance struct {
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"ExtractYAML\""`
// Ignored unless kind is ExtractYAML.
ExtractYAML ExtractYAML `json:"extractYAML,omitempty" yaml:"extractYAML,omitempty"`
}
// ExtractYAML represents a cue data instance encoded as yaml or json. If Path
// refers to a directory all files in the directory are extracted
// non-recursively. Otherwise, path must refer to a file.
type ExtractYAML struct {
Path string `json:"path" yaml:"path"`
}

View File

@@ -0,0 +1,5 @@
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---

453
api/core/v1alpha6/types.go Normal file
View File

@@ -0,0 +1,453 @@
// Package core contains schemas for a [Platform] and [BuildPlan]. Holos takes
// a [Platform] as input, then iterates over each [Component] to produce a
// [BuildPlan]. Holos processes the [BuildPlan] to produce fully rendered
// manifests, each an [Artifact].
package core
// BuildContextTag represents the cue tag holos render component uses to inject
// the json representation of a [BuildContext] for use in a BuildPlan.
const BuildContextTag string = "holos_build_context"
// ComponentNameTag represents the cue tag holos uses to inject a [Component]
// name from the holos render platform command to the holos render component
// command.
const ComponentNameTag string = "holos_component_name"
// ComponentPathTag represents the cue tag holos uses to inject a [Component]
// path relative to the cue module root from the holos render platform command
// to the holos render component command.
const ComponentPathTag string = "holos_component_path"
// ComponentLabelsTag represents the cue tag holos uses to inject the json
// representation of [Component] metadata labels from the holos render platform
// command to the holos render component command.
const ComponentLabelsTag string = "holos_component_labels"
// ComponentAnnotationsTag represents the tag holos uses to inject the json
// representation of [Component] metadata annotations from the holos render
// platform command to the holos render component command.
const ComponentAnnotationsTag = "holos_component_annotations"
//go:generate ../../../hack/gendoc
// BuildPlan represents an implementation of the [rendered manifest pattern].
// Holos processes a BuildPlan to produce one or more [Artifact] output files.
// BuildPlan artifact files usually contain Kubernetes manifests, but they may
// have any content.
//
// A BuildPlan usually produces two artifacts. One artifact contains a manifest
// of resources. A second artifact contains a GitOps resource to manage the
// first, usually an ArgoCD Application resource.
//
// Holos uses CUE to construct a BuildPlan. Holos injects late binding values
// such as the build temp dir using the [BuildContext].
//
// [rendered manifest pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
//
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"\"v1alpha6\""`
// Kind represents the type of the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
// BuildContext represents values injected by holos just before evaluating a
// BuildPlan, for example the tempDir used for the build.
BuildContext BuildContext `json:"buildContext" yaml:"buildContext"`
}
// BuildContext represents build context values owned by the holos render
// component command. End users should not manage context field values. End
// users may reference fields from within CUE to refer to late binding
// concrete values defined just before holos executes a [BuildPlan].
//
// Holos injects build context values by marshalling this struct to json through
// the holos_build_context cue tag.
//
// Example usage from cue to produce a [BuildPlan]:
//
// package holos
//
// import (
// "encoding/json"
// "github.com/holos-run/holos/api/core/v1alpha6:core"
// )
//
// _BuildContextJSON: string | *"{}" @tag(holos_build_context, type=string)
// BuildContext: core.#BuildContext & json.Unmarshal(_BuildContextJSON)
//
// holos: core.#BuildPlan & {
// buildContext: BuildContext
// "spec": {
// "artifacts": [
// {
// artifact: "components/slice",
// "transformers": [
// {
// "kind": "Command"
// "inputs": ["resources.gen.yaml"]
// "output": artifact
// "command": {
// "args": [
// "kubectl-slice",
// "-f",
// "\(buildContext.tempDir)/resources.gen.yaml",
// "-o",
// "\(buildContext.tempDir)/\(artifact)",
// ]
// }
// }
// ]
// }
// ]
// }
// }
type BuildContext struct {
// TempDir represents the temporary directory managed and owned by the holos
// render component command for the execution of one BuildPlan. Multiple
// tasks in the build plan share this temporary directory and therefore should
// avoid reading and writing into the same sub-directories as one another.
TempDir string `json:"tempDir" yaml:"tempDir" cue:"string | *\"${TEMP_DIR_PLACEHOLDER}\""`
// RootDir represents the fully qualified path to the platform root directory.
// Useful to construct arguments for commands in BuildPlan tasks.
RootDir string `json:"rootDir" yaml:"rootDir" cue:"string | *\"${ROOT_DIR_PLACEHOLDER}\""`
// LeafDir represents the cleaned path to the holos component relative to the
// platform root. Useful to construct arguments for commands in BuildPlan
// tasks.
LeafDir string `json:"leafDir" yaml:"leafDir" cue:"string | *\"${LEAF_DIR_PLACEHOLDER}\""`
// HolosExecutable represents the fully qualified path to the holos
// executable. Useful to execute tools embedded as subcommands such as holos
// cue vet.
HolosExecutable string `json:"holosExecutable" yaml:"holosExecutable" cue:"string | *\"holos\""`
}
// TODO(jjm): Decide if RootDir and LeafDir are useful. We have
// [Component.Path] and [Command] executes with the platform root as the current
// working directory.
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos render platform command to skip the BuildPlan.
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file or directory artifact. Generator
// Output values are used as Transformer Inputs. The Output field of the final
// [Transformer] should have the same value as the Artifact field.
//
// When there is more than one [Generator] there should be at least one
// [Transformer] to combine outputs into one Artifact file, or the final
// artifact should be a directory containing the outputs of the generators. If
// there is a single Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. One Artifact must not use an output of another Artifact as an
// input. Each [Generator] within an artifact also runs concurrently with
// generators of the same artifact. Each [Transformer] is executed sequentially
// starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
//
// When directories are used as inputs or outputs, they behave similar to how
// `git` works with directories. When the output field references a directory,
// all files within the directory are recursively stored using their relative
// path as a key. Similar to git add . When the input field references an
// absent file, a / is appended and the resulting value is used as a prefix
// match against all previous task outputs.
type Artifact struct {
Artifact FileOrDirectoryPath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Validators []Validator `json:"validators,omitempty" yaml:"validators,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// Each Generator in an [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
// 4. [Command] - Generates data by executing an user defined command.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\" | \"Command\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FileOrDirectoryPath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty" yaml:"file,omitempty"`
// Command generator. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents Kubernetes resources. Most commonly used to mix
// resources into the [BuildPlan] generated from CUE, but may be generated from
// elsewhere.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source" yaml:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// ValueFile represents one Helm value file produced from CUE.
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
//
// The Auth field is useful to configure http basic authentication to the Helm
// repository. Holos gets the username and password from the environment
// variables represented by the Auth field.
type Repository struct {
Name string `json:"name,omitempty" yaml:"name,omitempty"`
URL string `json:"url,omitempty" yaml:"url,omitempty"`
Auth Auth `json:"auth,omitempty" yaml:"auth,omitempty"`
}
// Auth represents environment variable names containing auth credentials.
type Auth struct {
Username AuthSource `json:"username" yaml:"username"`
Password AuthSource `json:"password" yaml:"password"`
}
// AuthSource represents a source for the value of an [Auth] field.
type AuthSource struct {
Value string `json:"value,omitempty" yaml:"value,omitempty"`
FromEnv string `json:"fromEnv,omitempty" yaml:"fromEnv,omitempty"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
// outputs into one output. [Kustomize] is the most commonly used transformer.
// A simple [Join] is also supported for use with plain manifest files.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
// 3. [Command] - Transforms data by executing an user defined command.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\" | \"Command\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FileOrDirectoryPath `json:"inputs" yaml:"inputs"`
// Output represents a file or directory for a subsequent Transformer or
// Artifact to consume.
Output FileOrDirectoryPath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
// Command transformer. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// FileOrDirectoryPath represents a file or a directory path.
type FileOrDirectoryPath string
// FileContent represents file contents.
type FileContent string
// Validator validates files. Useful to validate an [Artifact] prior to writing
// it out to the final destination. Holos may execute validators concurrently.
// See the [validators] tutorial for an end to end example.
//
// [validators]: https://holos.run/docs/v1alpha6/tutorial/validators/
type Validator struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Command\""`
// Inputs represents the files to validate. Usually the final Artifact.
Inputs []FileOrDirectoryPath `json:"inputs" yaml:"inputs"`
// Command validator. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
// Command represents a [BuildPlan] task implemented by executing an user
// defined system command. A task is defined as a [Generator], [Transformer],
// or [Validator]. Commands are executed with the working directory set to the
// platform root.
type Command struct {
// DisplayName of the command. The basename of args[0] is used if empty.
DisplayName string `json:"displayName,omitempty" yaml:"displayName,omitempty"`
// Args represents the argument vector passed to the os. to execute the
// command.
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
// IsStdoutOutput captures the command stdout as the task output if true.
IsStdoutOutput bool `json:"isStdoutOutput,omitempty" yaml:"isStdoutOutput,omitempty"`
}
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `app.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
// collection and integrates the components together into a holistic platform.
// Holos iterates over the [Component] collection producing a [BuildPlan] for
// each, which holos then executes to render manifests.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
type Platform struct {
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha6\""`
// Kind is a string value representing the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components" yaml:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
// from a path containing parameterized CUE configuration.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path" yaml:"path"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Holos copies Labels
// from the Component to the resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `app.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,37 +0,0 @@
package v1alpha2
// TypeMeta describes an individual object in an API response or request with
// strings representing the type of the object and its API schema version.
// Structures that are versioned or persisted should inline TypeMeta.
type TypeMeta struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind"`
// APIVersion defines the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
}
func (tm *TypeMeta) GetKind() string {
return tm.Kind
}
func (tm *TypeMeta) GetAPIVersion() string {
return tm.APIVersion
}
// Discriminator discriminates the kind of an api object.
type Discriminator interface {
// GetKind returns Kind.
GetKind() string
// GetAPIVersion returns APIVersion.
GetAPIVersion() string
}
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are holos objects distinct from
// kubernetes api objects.
type ObjectMeta struct {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
Name string `json:"name,omitempty"`
// Namespace confines a holos component to a single namespace via kustomize if set.
Namespace string `json:"namespace,omitempty"`
}

View File

@@ -1,99 +0,0 @@
// Package v1alpha3 contains CUE definitions intended as convenience wrappers
// around the core data types defined in package core. The purpose of these
// wrappers is to make life easier for platform engineers by reducing boiler
// plate code and generating component build plans in a consistent manner.
package v1alpha3
import core "github.com/holos-run/holos/api/core/v1alpha3"
//go:generate ../../../hack/gendoc
// Helm provides a BuildPlan via the Output field which contains one HelmChart
// from package core. Useful as a convenience wrapper to render a HelmChart
// with optional mix-in resources and Kustomization post-processing.
type Helm struct {
// Name represents the chart name.
Name string
// Version represents the chart version.
Version string
// Namespace represents the helm namespace option when rendering the chart.
Namespace string
// Resources are kubernetes api objects to mix into the output.
Resources map[string]any `cue:"{...}"`
// Repo represents the chart repository
Repo struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Values represents data to marshal into a values.yaml for helm.
Values interface{} `cue:"{...}"`
// Chart represents the derived HelmChart for inclusion in the BuildPlan
// Output field value. The default HelmChart field values are derived from
// other Helm field values and should be sufficient for most use cases.
Chart core.HelmChart
// EnableKustomizePostProcessor processes helm output with kustomize if true.
EnableKustomizePostProcessor bool `cue:"true | *false"`
// KustomizeFiles represents additional files to include in a Kustomization
// resources list. Useful to patch helm output. The implementation is a
// struct with filename keys and structs as values. Holos encodes the struct
// value to yaml then writes the result to the filename key. Component
// authors may then reference the filename in the kustomization.yaml resources
// or patches lists.
// Requires EnableKustomizePostProcessor: true.
KustomizeFiles map[string]any `cue:"{[string]: {...}}"`
// KustomizePatches represents patches to apply to the helm output. Requires
// EnableKustomizePostProcessor: true.
KustomizePatches map[core.InternalLabel]any `cue:"{[string]: {...}}"`
// KustomizeResources represents additional resources files to include in the
// kustomize resources list.
KustomizeResources map[string]any `cue:"{[string]: {...}}"`
// ArgoConfig represents the ArgoCD GitOps configuration for this Component.
ArgoConfig ArgoConfig
// Output represents the derived BuildPlan for the Holos cli to render.
Output core.BuildPlan
}
// Resources represents the default schema for a Kubernetes API object resource.
// For example, a Service, Namespace or Deployment. The top level key is the
// kind of resource so default behavior and strict schema enforcement may be
// enforced for the kind. The second level keys are an arbitrary internal
// label, which serves as the default value for the resource metadata name
// field, but may differ for situations where the same resource kind and name
// are managed in different namespaces.
//
// Refer to [definitions.cue] for the CUE schema definition as an example to
// build on when defining your own Components.
//
// [definitions.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/cue.mod/pkg/github.com/holos-run/holos/api/schema/v1alpha3/definitions.cue#L9
// type Resources map[string]map[string]any
// ArgoConfig represents the ArgoCD GitOps configuration for a Component.
// Useful to define once at the root of the Platform configuration and reuse
// across all Components.
type ArgoConfig struct {
// Enabled causes holos to render an ArgoCD Application resource for GitOps if true.
Enabled bool `cue:"true | *false"`
// ClusterName represents the cluster within the platform the Application
// resource is intended for.
ClusterName string
// DeployRoot represents the path from the git repository root to the `deploy`
// rendering output directory. Used as a prefix for the
// Application.spec.source.path field.
DeployRoot string `cue:"string | *\".\""`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
}

View File

@@ -1,56 +0,0 @@
package v1alpha1
import (
"errors"
"fmt"
"strings"
)
// BuildPlan is the primary interface between CUE and the Holos cli.
type BuildPlan struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
Spec BuildPlanSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
}
type BuildPlanSpec struct {
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
Components BuildPlanComponents `json:"components,omitempty" yaml:"components,omitempty"`
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
type BuildPlanComponents struct {
HelmChartList []HelmChart `json:"helmChartList,omitempty" yaml:"helmChartList,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty" yaml:"kubernetesObjectsList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty" yaml:"kustomizeBuildList,omitempty"`
Resources map[string]KubernetesObjects `json:"resources,omitempty" yaml:"resources,omitempty"`
}
func (bp *BuildPlan) Validate() error {
errs := make([]string, 0, 2)
if bp.Kind != BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", BuildPlanKind, bp.Kind))
}
if bp.APIVersion != APIVersion {
errs = append(errs, fmt.Sprintf("apiVersion invalid: want: %s have: %s", APIVersion, bp.APIVersion))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (bp *BuildPlan) ResultCapacity() (count int) {
if bp == nil {
return 0
}
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}

View File

@@ -1,30 +0,0 @@
package v1alpha1
// HolosComponent defines the fields common to all holos component kinds including the Render Result.
type HolosComponent struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
// APIObjectMap holds the marshalled representation of api objects. Think of
// these as resources overlaid at the back of the render pipeline.
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty" yaml:"apiObjectMap,omitempty"`
// Kustomization holds the marshalled representation of the flux kustomization
// which reconciles resources in git with the api server.
Kustomization `json:",inline" yaml:",inline"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:",inline" yaml:",inline"`
// Skip causes holos to take no action regarding the component.
Skip bool
}
func (hc *HolosComponent) NewResult() *Result {
return &Result{HolosComponent: *hc}
}
func (hc *HolosComponent) GetAPIVersion() string {
return hc.APIVersion
}
func (hc *HolosComponent) GetKind() string {
return hc.Kind
}

View File

@@ -1,11 +0,0 @@
package v1alpha1
const (
APIVersion = "holos.run/v1alpha1"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,2 +0,0 @@
// Package v1alpha1 defines the api boundary between CUE and Holos.
package v1alpha1

View File

@@ -1,13 +0,0 @@
package v1alpha1
import object "github.com/holos-run/holos/service/gen/holos/object/v1alpha1"
// Form represents a collection of Formly json powered form.
type Form struct {
TypeMeta `json:",inline" yaml:",inline"`
Spec FormSpec `json:"spec" yaml:"spec"`
}
type FormSpec struct {
Form object.Form `json:"form" yaml:"form"`
}

View File

@@ -1,184 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// A HelmChart represents a helm command to provide chart values in order to render kubernetes api objects.
type HelmChart struct {
HolosComponent `json:",inline" yaml:",inline"`
// Namespace is the namespace to install into. TODO: Use metadata.namespace instead.
Namespace string `json:"namespace"`
Chart Chart `json:"chart"`
ValuesContent string `json:"valuesContent"`
EnableHooks bool `json:"enableHooks"`
}
type Chart struct {
Name string `json:"name"`
Version string `json:"version"`
Release string `json:"release"`
Repository Repository `json:"repository,omitempty"`
}
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
func (hc *HelmChart) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: hc.HolosComponent}
if err := hc.helm(ctx, &result, path); err != nil {
return nil, err
}
result.addObjectMap(ctx, hc.APIObjectMap)
if err := result.kustomize(ctx); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not kustomize: %w", err))
}
return &result, nil
}
// runHelm provides the values produced by CUE to helm template and returns
// the rendered kubernetes api objects in the result.
func (hc *HelmChart) helm(ctx context.Context, r *Result, path holos.InstancePath) error {
log := logger.FromContext(ctx).With("chart", hc.Chart.Name)
if hc.Chart.Name == "" {
log.WarnContext(ctx, "skipping helm: no chart name specified, use a different component type")
return nil
}
cachedChartPath := filepath.Join(string(path), ChartDir, filepath.Base(hc.Chart.Name))
if isNotExist(cachedChartPath) {
// Add repositories
repo := hc.Chart.Repository
if repo.URL != "" {
out, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo add: %w", err))
}
// Update repository
out, err = util.RunCmd(ctx, "helm", "repo", "update", repo.Name)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo update: %w", err))
}
} else {
log.DebugContext(ctx, "no chart repository url proceeding assuming oci chart")
}
// Cache the chart
if err := cacheChart(ctx, path, ChartDir, hc.Chart); err != nil {
return fmt.Errorf("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos")
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, tempDir)
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, []byte(hc.ValuesContent), 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "helm: wrote values", "path", valuesPath, "bytes", len(hc.ValuesContent))
// Run charts
chart := hc.Chart
args := []string{"template"}
if !hc.EnableHooks {
args = append(args, "--no-hooks")
}
namespace := hc.Namespace
args = append(args, "--include-crds", "--values", valuesPath, "--namespace", namespace, "--kubeconfig", "/dev/null", "--version", chart.Version, chart.Release, cachedChartPath)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Wrap(fmt.Errorf("could not run helm template: %w", err))
}
r.accumulatedOutput = helmOut.Stdout.String()
return nil
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// It is assumed that the only method responsible for writing to chartDir is
// cacheChart itself.
//
// This relies on the atomicity of moving temporary directories into place on
// the same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is an
// expected scenario when this function is called concurrently.
func cacheChart(ctx context.Context, path holos.InstancePath, chartDir string, chart Chart) error {
log := logger.FromContext(ctx)
cacheTemp, err := os.MkdirTemp(string(path), chartDir)
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, cacheTemp)
chartName := chart.Name
if chart.Repository.Name != "" {
chartName = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, chartName)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
cachePath := filepath.Join(string(path), chartDir)
if err := os.MkdirAll(cachePath, 0777); err != nil {
return errors.Wrap(fmt.Errorf("could not mkdir: %w", err))
}
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
for _, item := range items {
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cachePath, item.Name())
log.DebugContext(ctx, "rename", "src", src, "dst", dst)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
}
}
log.InfoContext(ctx, "cached", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
return nil
}
func isNotExist(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}

View File

@@ -1,21 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents CUE output which directly provides Kubernetes api objects to holos.
type KubernetesObjects struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces kubernetes api objects from the APIObjectMap
func (o *KubernetesObjects) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: o.HolosComponent}
result.addObjectMap(ctx, o.APIObjectMap)
return &result, nil
}

View File

@@ -1,7 +0,0 @@
package v1alpha1
// Kustomization holds the rendered flux kustomization api object content for git ops.
type Kustomization struct {
// KsContent is the yaml representation of the flux kustomization for gitops.
KsContent string `json:"ksContent,omitempty" yaml:"ksContent,omitempty"`
}

View File

@@ -1,47 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
const KustomizeBuildKind = "KustomizeBuild"
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process raw yaml file resources in a holos component directory.
// 2. Post process a HelmChart to inject istio, add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty" yaml:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty" yaml:"resourcesFile,omitempty"`
}
// KustomizeBuild renders plain yaml files in the holos component directory using kubectl kustomize build.
type KustomizeBuild struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces a Result by executing kubectl kustomize on the holos
// component path. Useful for processing raw yaml files.
func (kb *KustomizeBuild) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
log := logger.FromContext(ctx)
result := Result{HolosComponent: kb.HolosComponent}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", string(path))
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return nil, errors.Wrap(err)
}
// Replace the accumulated output
result.accumulatedOutput = kOut.Stdout.String()
// Add CUE based api objects.
result.addObjectMap(ctx, kb.APIObjectMap)
return &result, nil
}

View File

@@ -1,14 +0,0 @@
package v1alpha1
// Label is an arbitrary unique identifier. Defined as a type for clarity and type checking.
type Label string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
type Kind string
// APIObjectMap is the shape of marshalled api objects returned from cue to the
// holos cli. A map is used to improve the clarity of error messages from cue.
type APIObjectMap map[Kind]map[Label]string
// FileContentMap is a map of file names to file contents.
type FileContentMap map[string]string

View File

@@ -1,15 +0,0 @@
package v1alpha1
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are by holos objects distinct
// from kubernetes api objects.
type ObjectMeta struct {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
Name string `json:"name,omitempty" yaml:"name,omitempty"`
// Namespace confines a holos component to a single namespace via kustomize if set.
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Labels are not used but are copied from api machinery ObjectMeta for completeness.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations are not used but are copied from api machinery ObjectMeta for completeness.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,32 +0,0 @@
package v1alpha1
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
TypeMeta `json:",inline" yaml:",inline"`
Metadata ObjectMeta `json:"metadata" yaml:"metadata"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform build plan specification.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// holos.platform.v1alpha1.PlatformService.GetPlatform method and provides to
// CUE using a tag.
Model structpb.Struct `json:"model" yaml:"model"`
Components []PlatformSpecComponent `json:"components" yaml:"components"`
}
// PlatformSpecComponent represents a component to build or render with flags to
// pass, for example the cluster name.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path" yaml:"path"`
// Cluster is the cluster name to use when building the component.
Cluster string `json:"cluster" yaml:"cluster"`
}

View File

@@ -1,22 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
type Renderer interface {
GetKind() string
Render(ctx context.Context, path holos.InstancePath) (*Result, error)
}
// Render produces a Result representing the kubernetes api objects to
// configure. Each of the various holos component types, e.g. Helm, Kustomize,
// et al, should implement the Renderer interface. This process is best
// conceptualized as a data pipeline, for example a component may render a
// result by first calling helm template, then passing the result through
// kustomize, then mixing in overlay api objects.
func Render(ctx context.Context, r Renderer, path holos.InstancePath) (*Result, error) {
return r.Render(ctx, path)
}

View File

@@ -1,165 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"slices"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// Result is the build result for display or writing. Holos components Render the Result as a data pipeline.
type Result struct {
HolosComponent
// accumulatedOutput accumulates rendered api objects.
accumulatedOutput string
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
// Continue returns true if Skip is true indicating the result is to be skipped over.
func (r *Result) Continue() bool {
if r == nil {
return false
}
return r.Skip
}
func (r *Result) Name() string {
return r.Metadata.Name
}
func (r *Result) Filename(writeTo string, cluster string) string {
name := r.Metadata.Name
return filepath.Join(writeTo, "clusters", cluster, "components", name, name+".gen.yaml")
}
func (r *Result) KustomizationFilename(writeTo string, cluster string) string {
return filepath.Join(writeTo, "clusters", cluster, "holos", "components", r.Metadata.Name+"-kustomization.gen.yaml")
}
// AccumulatedOutput returns the accumulated rendered output.
func (r *Result) AccumulatedOutput() string {
return r.accumulatedOutput
}
// addObjectMap renders the provided APIObjectMap into the accumulated output.
func (r *Result) addObjectMap(ctx context.Context, objectMap APIObjectMap) {
log := logger.FromContext(ctx)
b := []byte(r.AccumulatedOutput())
kinds := make([]Kind, 0, len(objectMap))
// Sort the keys
for kind := range objectMap {
kinds = append(kinds, kind)
}
slices.Sort(kinds)
for _, kind := range kinds {
v := objectMap[kind]
// Sort the keys
names := make([]Label, 0, len(v))
for name := range v {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
yamlString := v[name]
log.Debug(fmt.Sprintf("%s/%s", kind, name), "kind", kind, "name", name)
b = util.EnsureNewline(b)
header := fmt.Sprintf("---\n# Source: CUE apiObjects.%s.%s\n", kind, name)
b = append(b, []byte(header+yamlString)...)
b = util.EnsureNewline(b)
}
}
r.accumulatedOutput = string(b)
}
// kustomize replaces the accumulated output with the output of kustomize build
func (r *Result) kustomize(ctx context.Context) error {
log := logger.FromContext(ctx)
if r.ResourcesFile == "" {
log.DebugContext(ctx, "skipping kustomize: no resourcesFile")
return nil
}
if len(r.KustomizeFiles) < 1 {
log.DebugContext(ctx, "skipping kustomize: no kustomizeFiles")
return nil
}
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
// Write the main api object resources file for kustomize.
target := filepath.Join(tempDir, r.ResourcesFile)
b := []byte(r.AccumulatedOutput())
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write resources: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
// Write the kustomization tree, kustomization.yaml must be in this map for kustomize to work.
for file, content := range r.KustomizeFiles {
target := filepath.Join(tempDir, file)
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return errors.Wrap(err)
}
b := []byte(content)
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return errors.Wrap(err)
}
// Replace the accumulated output
r.accumulatedOutput = kOut.Stdout.String()
return nil
}
func (r *Result) WriteDeployFiles(ctx context.Context, path string) error {
log := logger.FromContext(ctx)
if len(r.DeployFiles) == 0 {
return nil
}
for k, content := range r.DeployFiles {
path := filepath.Join(path, k)
if err := r.Save(ctx, path, content); err != nil {
return errors.Wrap(err)
}
log.InfoContext(ctx, "wrote deploy file", "path", path, "bytes", len(content))
}
return nil
}
// Save writes the content to the filesystem for git ops.
func (r *Result) Save(ctx context.Context, path string, content string) error {
log := logger.FromContext(ctx)
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, os.FileMode(0775)); err != nil {
log.WarnContext(ctx, "could not mkdir", "path", dir, "err", err)
return errors.Wrap(err)
}
// Write the file content
if err := os.WriteFile(path, []byte(content), os.FileMode(0644)); err != nil {
log.WarnContext(ctx, "could not write", "path", path, "err", err)
return errors.Wrap(err)
}
log.DebugContext(ctx, "out: wrote "+path, "action", "write", "path", path, "status", "ok")
return nil
}

View File

@@ -1,20 +0,0 @@
package v1alpha1
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
func (tm *TypeMeta) GetKind() string {
return tm.Kind
}
func (tm *TypeMeta) GetAPIVersion() string {
return tm.APIVersion
}
// Discriminator is an interface to discriminate the kind api object.
type Discriminator interface {
GetKind() string
GetAPIVersion() string
}

View File

@@ -1,20 +0,0 @@
# Generates gRPC and ConnectRPC bindings for Go and TypeScript
#
# Note: protoc-gen-connect-query is the primary method of wiring up the React
# frontend.
version: v1
plugins:
- plugin: go
out: service/gen
opt: paths=source_relative
- plugin: connect-go
out: service/gen
opt: paths=source_relative
- plugin: es
out: internal/frontend/holos/src/app/gen
opt:
- target=ts
- plugin: connect-es
out: internal/frontend/holos/src/app/gen
opt:
- target=ts

View File

@@ -1,8 +0,0 @@
# Generated by buf. DO NOT EDIT.
version: v1
deps:
- remote: buf.build
owner: bufbuild
repository: protovalidate
commit: b983156c5e994cc9892e0ce3e64e17e0
digest: shake256:fb47a62989d38c2529bcc5cd86ded43d800eb84cee82b42b9e8a9e815d4ee8134a0fb9d0ce8299b27c2d2bbb7d6ade0c4ad5a8a4d467e1e2c7ca619ae9f634e2

View File

@@ -1,3 +0,0 @@
version: v1
directories:
- service

34
cmd/cmd.go Normal file
View File

@@ -0,0 +1,34 @@
package cmd
import (
"context"
"fmt"
"log/slog"
"os"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/internal/holos"
"github.com/holos-run/holos/version"
)
// MakeMain makes a main function for the cli or tests.
func MakeMain(options ...holos.Option) func() int {
return func() (exitCode int) {
// TODO(jjm): check HOLOS_CHDIR and chdir if set for tests.
if len(os.Args) >= 2 && os.Args[1] == "version" {
if _, err := fmt.Println(version.GetVersion()); err != nil {
panic(err)
}
return 0
}
cfg := holos.New(options...)
slog.SetDefault(cfg.Logger())
ctx := context.Background()
cmd := cli.New(cfg)
if err := cmd.ExecuteContext(ctx); err != nil {
return cli.HandleError(ctx, err, cfg)
}
return 0
}
}

View File

@@ -3,9 +3,9 @@ package main
import (
"os"
"github.com/holos-run/holos/internal/cli"
"github.com/holos-run/holos/cmd"
)
func main() {
os.Exit(cli.MakeMain()())
os.Exit(cmd.MakeMain()())
}

View File

@@ -2,20 +2,51 @@ package main
import (
"os"
"path/filepath"
"testing"
"github.com/holos-run/holos/internal/cli"
cue "cuelang.org/go/cmd/cue/cmd"
"github.com/holos-run/holos/cmd"
"github.com/rogpeppe/go-internal/testscript"
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cli.MakeMain(),
}))
}
func TestGetSecrets(t *testing.T) {
testscript.Run(t, testscript.Params{
Dir: "testdata",
holosMain := cmd.MakeMain()
testscript.Main(m, map[string]func(){
"holos": func() { os.Exit(holosMain()) },
"cue": func() { os.Exit(cue.Main()) },
})
}
func TestGuides_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "guides")))
}
func TestSchemas_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "schemas")))
}
func TestIssues_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "issues")))
}
func TestCLI(t *testing.T) {
testscript.Run(t, params("cli"))
}
func params(dir string) testscript.Params {
return testscript.Params{
Dir: filepath.Join("tests", dir),
RequireExplicitExec: true,
RequireUniqueNames: os.Getenv("HOLOS_WORKDIR_ROOT") == "",
WorkdirRoot: os.Getenv("HOLOS_WORKDIR_ROOT"),
UpdateScripts: os.Getenv("HOLOS_UPDATE_SCRIPTS") != "",
Setup: func(env *testscript.Env) error {
// Just like cmd/cue/cmd.TestScript, set up separate cache and config dirs per test.
env.Setenv("CUE_CACHE_DIR", filepath.Join(env.WorkDir, "tmp/cachedir"))
configDir := filepath.Join(env.WorkDir, "tmp/configdir")
env.Setenv("CUE_CONFIG_DIR", configDir)
return nil
},
}
}

View File

@@ -1,34 +0,0 @@
# Want support for intermediary constraints
exec holos build ./foo/... --log-level debug
stdout '^bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- foo/constraints.cue --
package holos
metadata: name: "jeff"
-- foo/bar/bar.cue --
package holos
spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
apiObjectMap: foo: bar: "bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b"
}
]
-- schema.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#KubernetesObjects: {
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
apiObjectMap: {...}
}
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"

View File

@@ -1,20 +0,0 @@
# Want cue errors to show files and lines
! exec holos build .
stderr 'apiObjectMap.foo.bar: cannot convert incomplete value'
stderr '/component.cue:\d+:\d+$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: foo: bar: _baz}]
_baz: string

View File

@@ -1,61 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,62 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
stderr 'skipping helm: no chart name specified'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,25 +0,0 @@
# Want api object kind and name in errors
! exec holos build .
stderr 'apiObjects.secretstore.default.foo: field not allowed'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
metadata: name: string
}
apiObjects: {
secretstore: {
default: #SecretStore & { foo: "not allowed" }
}
}

View File

@@ -1,289 +0,0 @@
# Want helm errors to show up
! exec holos build .
stderr 'Error: execution error at \(zitadel/templates/secret_zitadel-masterkey.yaml:2:4\): Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- zitadel.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [_HelmChart]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
_HelmChart: {
apiVersion: "holos.run/v1alpha1"
kind: "HelmChart"
metadata: name: "zitadel"
namespace: "zitadel"
chart: {
name: "zitadel"
version: "7.9.0"
release: name
repository: {
name: "zitadel"
url: "https://charts.zitadel.com"
}
}
}
-- vendor/zitadel/templates/secret_zitadel-masterkey.yaml --
{{- if (or (and .Values.zitadel.masterkey .Values.zitadel.masterkeySecretName) (and (not .Values.zitadel.masterkey) (not .Values.zitadel.masterkeySecretName)) ) }}
{{- fail "Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName" }}
{{- end }}
{{- if .Values.zitadel.masterkey -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: zitadel-masterkey
{{- with .Values.zitadel.masterkeyAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "zitadel.labels" . | nindent 4 }}
stringData:
masterkey: {{ .Values.zitadel.masterkey }}
{{- end -}}
-- vendor/zitadel/Chart.yaml --
apiVersion: v2
appVersion: v2.46.0
description: A Helm chart for ZITADEL
icon: https://zitadel.com/zitadel-logo-dark.svg
kubeVersion: '>= 1.21.0-0'
maintainers:
- email: support@zitadel.com
name: zitadel
url: https://zitadel.com
name: zitadel
type: application
version: 7.9.0
-- vendor/zitadel/values.yaml --
# Default values for zitadel.
zitadel:
# The ZITADEL config under configmapConfig is written to a Kubernetes ConfigMap
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
configmapConfig:
ExternalSecure: true
Machine:
Identification:
Hostname:
Enabled: true
Webhook:
Enabled: false
# The ZITADEL config under secretConfig is written to a Kubernetes Secret
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
secretConfig:
# Annotations set on secretConfig secret
secretConfigAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# Reference the name of a secret that contains ZITADEL configuration.
configSecretName:
# The key under which the ZITADEL configuration is located in the secret.
configSecretKey: config-yaml
# ZITADEL uses the masterkey for symmetric encryption.
# You can generate it for example with tr -dc A-Za-z0-9 </dev/urandom | head -c 32
masterkey: ""
# Reference the name of the secret that contains the masterkey. The key should be named "masterkey".
# Note: Either zitadel.masterkey or zitadel.masterkeySecretName must be set
masterkeySecretName: ""
# Annotations set on masterkey secret
masterkeyAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The CA Certificate needed for establishing secure database connections
dbSslCaCrt: ""
# The Secret containing the CA certificate at key ca.crt needed for establishing secure database connections
dbSslCaCrtSecret: ""
# The db admins secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslAdminCrtSecret: ""
# The db users secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslUserCrtSecret: ""
# Generate a self-signed certificate using an init container
# This will also mount the generated files to /etc/tls/ so that you can reference them in the pod.
# E.G. KeyPath: /etc/tls/tls.key CertPath: /etc/tls/tls.crt
# By default, the SAN DNS names include, localhost, the POD IP address and the POD name. You may include one more by using additionalDnsName like "my.zitadel.fqdn".
selfSignedCert:
enabled: false
additionalDnsName:
replicaCount: 3
image:
repository: ghcr.io/zitadel/zitadel
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
chownImage:
repository: alpine
pullPolicy: IfNotPresent
tag: "3.19"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
# Annotations to add to the deployment
annotations: {}
# Annotations to add to the configMap
configMap:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podAdditionalLabels: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
securityContext: {}
# Additional environment variables
env:
[]
# - name: ZITADEL_DATABASE_POSTGRES_HOST
# valueFrom:
# secretKeyRef:
# name: postgres-pguser-postgres
# key: host
service:
type: ClusterIP
# If service type is "ClusterIP", this can optionally be set to a fixed IP address.
clusterIP: ""
port: 8080
protocol: http2
annotations: {}
scheme: HTTP
ingress:
enabled: false
className: ""
annotations: {}
hosts:
- host: localhost
paths:
- path: /
pathType: Prefix
tls: []
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
topologySpreadConstraints: []
initJob:
# Once ZITADEL is installed, the initJob can be disabled.
enabled: true
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "1"
resources: {}
backoffLimit: 5
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
# Available init commands :
# "": initialize ZITADEL instance (without skip anything)
# database: initialize only the database
# grant: set ALL grant to user
# user: initialize only the database user
# zitadel: initialize ZITADEL internals (skip "create user" and "create database")
command: ""
setupJob:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "2"
resources: {}
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
additionalArgs:
- "--init-projections=true"
machinekeyWriter:
image:
repository: bitnami/kubectl
tag: ""
resources: {}
readinessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
livenessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
startupProbe:
enabled: true
periodSeconds: 1
failureThreshold: 30
metrics:
enabled: false
serviceMonitor:
# If true, the chart creates a ServiceMonitor that is compatible with Prometheus Operator
# https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#monitoring.coreos.com/v1.ServiceMonitor.
# The Prometheus community Helm chart installs this operator
# https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#kube-prometheus-stack
enabled: false
honorLabels: false
honorTimestamps: true
pdb:
enabled: false
# these values are used for the PDB and are mutally exclusive
minAvailable: 1
# maxUnavailable: 1
annotations: {}

View File

@@ -1,39 +0,0 @@
# Kustomize is a supported holos component kind
exec holos render component --cluster-name=mycluster . --log-level=debug
# Want generated output
cmp want.yaml deploy/clusters/mycluster/components/kstest/kstest.gen.yaml
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KustomizeBuildList: [{metadata: name: "kstest"}]
-- kustomization.yaml --
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mynamespace
resources:
- serviceaccount.yaml
-- serviceaccount.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
-- want.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
namespace: mynamespace

View File

@@ -1,17 +0,0 @@
# https://github.com/holos-run/holos/issues/72
# Want holos to fail on unknown fields to catch typos and aid refactors
! exec holos build .
stderr 'unknown field \\"TypoKubernetesObjectsList\\"'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: TypoKubernetesObjectsList: []

View File

@@ -0,0 +1,12 @@
# https://github.com/holos-run/holos/issues/358
# holos cue vet should fail verifications with exit code 1
! exec holos cue vet ./policy --path strings.ToLower(kind) ./data/secret.yaml
# holos cue vet should report validation errors to stderr
stderr 'Forbidden. Use an ExternalSecret instead.'
-- data/secret.yaml --
kind: Secret
-- policy/validators.cue --
package policy
secret: kind: "Forbidden. Use an ExternalSecret instead."

View File

@@ -0,0 +1,7 @@
# https://github.com/holos-run/holos/issues/334
exec holos
stdout Usage
exec holos --version
stdout \d+\.\d+\.\d+
exec holos version
stdout \d+\.\d+\.\d+

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,65 @@
# https://github.com/holos-run/holos/issues/348
# when the optional kustomize patch name field is omitted
exec holos init platform v1alpha5 --force
# want a buildplan shown
exec holos show buildplans
cp stdout buildplan-output.yaml
exec holos compare yaml buildplan-output.yaml buildplan.yaml
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
-- buildplan.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: example
spec:
artifacts:
- artifact: components/example/example.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/example/example.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
patches:
- patch: |
- op: add
path: /metadata/annotations/example
value: example-value
target:
kind: CustomResourceDefinition
name: ""
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,52 @@
# https://github.com/holos-run/holos/issues/366
# Build tags conditionally include CUE files.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cp stdout empty.yaml
exec holos compare yaml empty.yaml want/empty.yaml
exec holos show platform -t foo
cp stdout foo.yaml
exec holos compare yaml foo.yaml want/foo.yaml
-- platform/empty.cue --
@if(foo)
package holos
Platform: Components: foo: _
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": NAME
annotations: "app.holos.run/description": "\(NAME) empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/empty.yaml --
kind: Platform
apiVersion: v1alpha5
metadata:
name: default
spec:
components: []
-- want/foo.yaml --
kind: Platform
apiVersion: v1alpha5
metadata:
name: default
spec:
components:
- name: foo
path: components/empty
labels:
app.holos.run/name: foo
annotations:
app.holos.run/description: foo empty test case

View File

@@ -0,0 +1,147 @@
# https://github.com/holos-run/holos/issues/330
# take care to install helm 3.17.3 otherwise kube versions may not align
exec holos init platform v1alpha5 --force
# Make sure the helm chart works with plain helm
exec helm template ./components/capabilities/vendor/0.1.0/capabilities
stdout 'name: has-foo-v1beta1'
stdout 'kubeVersion: v'
exec holos render platform ./platform
# When no capabilities are specified
exec holos compare yaml deploy/components/capabilities/capabilities.gen.yaml want/when-no-capabilities-specified.yaml
# With APIVersions specified
exec holos compare yaml deploy/components/specified/specified.gen.yaml want/with-capabilities-specified.yaml
# With KubeVersion specified
exec holos compare yaml deploy/components/kubeversion1/kubeversion1.gen.yaml want/with-kubeversion-specified.yaml
# With both APIVersions and KubeVersion specified
exec holos compare yaml deploy/components/kubeversion2/kubeversion2.gen.yaml want/with-both-specified.yaml
-- want/with-both-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-kubeversion-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/when-no-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.99.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.99.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- platform/capabilities.cue --
package holos
import "encoding/json"
Platform: Components: capabilities: {
name: "capabilities"
path: "components/capabilities"
}
Platform: Components: specified: {
name: "specified"
path: "components/capabilities"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
Platform: Components: kubeversion1: {
name: "kubeversion1"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
}
Platform: Components: kubeversion2: {
name: "kubeversion2"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
-- components/capabilities/capabilities.cue --
package holos
import "encoding/json"
holos: Component.BuildPlan
Component: #Helm & {
Name: string @tag(holos_component_name, type=string)
Chart: name: "capabilities"
Chart: version: "0.1.0"
_APIVersions: string | *"[]" @tag(apiVersions, type=string)
APIVersions: json.Unmarshal(_APIVersions)
KubeVersion: string | *"v1.99.0" @tag(kubeVersion, type=string)
}
-- components/capabilities/vendor/0.1.0/capabilities/Chart.yaml --
apiVersion: v2
name: capabilities
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: "1.16.0"
-- components/capabilities/vendor/0.1.0/capabilities/templates/service.yaml --
apiVersion: v1
kind: Service
metadata:
{{- if .Capabilities.APIVersions.Has "foo/v1" }}
name: has-foo-v1
{{- else }}
name: has-foo-v1beta1
{{- end }}
annotations:
kubeVersion: {{ .Capabilities.KubeVersion }}
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
-- want/helm-template.yaml --
---
# Source: capabilities/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: has-foo-v1beta1
annotations:
kubeVersion: v1.99.0
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http

View File

@@ -0,0 +1,46 @@
# author.#Kubernetes
# Start in an empty directory.
cd $WORK
# Generate the directory structure we're going to work in.
exec holos init platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform
stderr -count=1 '^rendered platform'
# When author.#Kubernetes is empty
exec holos cue export --expression holos --out=yaml ./components/empty
cp stdout empty.yaml
exec holos compare yaml empty.yaml want.txt
-- components/empty/empty.cue --
package holos
Kubernetes: #Kubernetes & {}
holos: Kubernetes.BuildPlan
-- want.txt --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.gen.yaml
validators: []

View File

@@ -0,0 +1,37 @@
# https://github.com/holos-run/holos/issues/357
exec holos init platform v1alpha5 --force
! exec holos render platform
stderr 'secret.kind: conflicting values "Forbidden. Use an ExternalSecret instead." and "Secret"'
-- validators.cue --
package holos
import "github.com/holos-run/holos/api/author/v1alpha5:author"
#ComponentConfig: author.#ComponentConfig & {
Validators: cue: {
kind: "Command"
command: args: ["holos", "cue", "vet", "./policy", "--path", "strings.ToLower(kind)"]
}
}
-- policy/validations.cue --
package validations
secret: kind: "Forbidden. Use an ExternalSecret instead."
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/secret.cue --
package holos
holos: Component.BuildPlan
Component: #Kubernetes & {
Resources: Secret: test: {
metadata: name: "test"
}
}

10
doc/md/api.mdx Normal file
View File

@@ -0,0 +1,10 @@
---
slug: api
description: Schema Reference
---
import DocCardList from '@theme/DocCardList';
# Schema Reference
<DocCardList />

189
doc/md/api/author.md Normal file
View File

@@ -0,0 +1,189 @@
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
```go
import "github.com/holos-run/holos/api/author/v1alpha6"
```
Package author contains a standard set of schemas for component authors to generate common [core](<https://holos.run/docs/api/core/>) BuildPlans.
Holos values stability, flexibility, and composition. This package intentionally defines only the minimal necessary set of structures. Component authors are encouraged to define their own structures building on our example [topics](<https://holos.run/docs/topics/>).
The Holos Maintainers may add definitions to this package if the community identifies nearly all users must define the exact same structure. Otherwise, definitions should be added as a customizable example in [topics](<https://holos.run/docs/topics/>).
For example, structures representing a cluster and environment almost always need to be defined. Their definition varies from one organization to the next. Therefore, customizable definitions for a cluster and environment are best maintained in [topics](<https://holos.run/docs/topics/>), not standardized in this package.
## Index
- [type ComponentConfig](<#ComponentConfig>)
- [type Helm](<#Helm>)
- [type Kubernetes](<#Kubernetes>)
- [type Kustomize](<#Kustomize>)
- [type KustomizeConfig](<#KustomizeConfig>)
- [type NameLabel](<#NameLabel>)
- [type Platform](<#Platform>)
<a name="ComponentConfig"></a>
## type ComponentConfig {#ComponentConfig}
ComponentConfig represents the configuration common to all kinds of components for use with the holos render component command. All component kinds may be transformed with [kustomize](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>) configured with the [KustomizeConfig](<#KustomizeConfig>) field.
- [Helm](<#Helm>) charts.
- [Kubernetes](<#Kubernetes>) resources generated from CUE.
- [Kustomize](<#Kustomize>) bases.
```go
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the kustomize configuration.
KustomizeConfig KustomizeConfig
// Validators represent checks that must pass for output to be written.
Validators map[NameLabel]core.Validator
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm assembles a BuildPlan rendering a helm chart. Useful to mix in additional resources from CUE and transform the helm output with kustomize.
```go
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// ValueFiles represents value files for migration from helm value
// hierarchies. Use Values instead.
ValueFiles []core.ValueFile `json:",omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="Kubernetes"></a>
## type Kubernetes {#Kubernetes}
Kubernetes assembles a BuildPlan containing inline resources exported from CUE.
```go
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize assembles a BuildPlan rendering manifests from a [kustomize](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>) kustomization.
```go
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="KustomizeConfig"></a>
## type KustomizeConfig {#KustomizeConfig}
KustomizeConfig represents the configuration for [kustomize](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>) post processing. Use the Files field to mix in plain manifest files located in the component directory. Use the Resources field to mix in manifests from network urls.
```go
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
```
<a name="NameLabel"></a>
## type NameLabel {#NameLabel}
NameLabel represents the common use case of converting a struct to a list where the name field of each value unifies with the field name of the outer struct.
For example:
```
S: [NameLabel=string]: name: NameLabel
S: jeff: _
S: gary: _
S: nate: _
L: [for x in S {x}]
// L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
```
```go
type NameLabel string
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform assembles a core Platform in the Resource field for the holos render platform command. Use the Components field to register components with the platform.
```go
type Platform struct {
Name string `json:"name" yaml:"name" cue:"string | *\"default\""`
Components map[NameLabel]core.Component `json:"components" yaml:"components"`
Resource core.Platform `json:"resource" yaml:"resource"`
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

621
doc/md/api/core.md Normal file
View File

@@ -0,0 +1,621 @@
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
```go
import "github.com/holos-run/holos/api/core/v1alpha6"
```
Package core contains schemas for a [Platform](<#Platform>) and [BuildPlan](<#BuildPlan>). Holos takes a [Platform](<#Platform>) as input, then iterates over each [Component](<#Component>) to produce a [BuildPlan](<#BuildPlan>). Holos processes the [BuildPlan](<#BuildPlan>) to produce fully rendered manifests, each an [Artifact](<#Artifact>).
## Index
- [Constants](<#constants>)
- [type Artifact](<#Artifact>)
- [type Auth](<#Auth>)
- [type AuthSource](<#AuthSource>)
- [type BuildContext](<#BuildContext>)
- [type BuildPlan](<#BuildPlan>)
- [type BuildPlanSpec](<#BuildPlanSpec>)
- [type Chart](<#Chart>)
- [type Command](<#Command>)
- [type Component](<#Component>)
- [type File](<#File>)
- [type FileContent](<#FileContent>)
- [type FileContentMap](<#FileContentMap>)
- [type FileOrDirectoryPath](<#FileOrDirectoryPath>)
- [type FilePath](<#FilePath>)
- [type Generator](<#Generator>)
- [type Helm](<#Helm>)
- [type InternalLabel](<#InternalLabel>)
- [type Join](<#Join>)
- [type Kind](<#Kind>)
- [type Kustomization](<#Kustomization>)
- [type Kustomize](<#Kustomize>)
- [type Metadata](<#Metadata>)
- [type Platform](<#Platform>)
- [type PlatformSpec](<#PlatformSpec>)
- [type Repository](<#Repository>)
- [type Resource](<#Resource>)
- [type Resources](<#Resources>)
- [type Transformer](<#Transformer>)
- [type Validator](<#Validator>)
- [type ValueFile](<#ValueFile>)
- [type Values](<#Values>)
## Constants
<a name="BuildContextTag"></a>BuildContextTag represents the cue tag holos render component uses to inject the json representation of a [BuildContext](<#BuildContext>) for use in a BuildPlan.
```go
const BuildContextTag string = "holos_build_context"
```
<a name="ComponentAnnotationsTag"></a>ComponentAnnotationsTag represents the tag holos uses to inject the json representation of [Component](<#Component>) metadata annotations from the holos render platform command to the holos render component command.
```go
const ComponentAnnotationsTag = "holos_component_annotations"
```
<a name="ComponentLabelsTag"></a>ComponentLabelsTag represents the cue tag holos uses to inject the json representation of [Component](<#Component>) metadata labels from the holos render platform command to the holos render component command.
```go
const ComponentLabelsTag string = "holos_component_labels"
```
<a name="ComponentNameTag"></a>ComponentNameTag represents the cue tag holos uses to inject a [Component](<#Component>) name from the holos render platform command to the holos render component command.
```go
const ComponentNameTag string = "holos_component_name"
```
<a name="ComponentPathTag"></a>ComponentPathTag represents the cue tag holos uses to inject a [Component](<#Component>) path relative to the cue module root from the holos render platform command to the holos render component command.
```go
const ComponentPathTag string = "holos_component_path"
```
<a name="Artifact"></a>
## type Artifact {#Artifact}
Artifact represents one fully rendered manifest produced by a [Transformer](<#Transformer>) sequence, which transforms a [Generator](<#Generator>) collection. A [BuildPlan](<#BuildPlan>) produces an [Artifact](<#Artifact>) collection.
Each Artifact produces one manifest file or directory artifact. Generator Output values are used as Transformer Inputs. The Output field of the final [Transformer](<#Transformer>) should have the same value as the Artifact field.
When there is more than one [Generator](<#Generator>) there should be at least one [Transformer](<#Transformer>) to combine outputs into one Artifact file, or the final artifact should be a directory containing the outputs of the generators. If there is a single Generator, it may directly produce the Artifact output.
An Artifact is processed concurrently with other artifacts in the same [BuildPlan](<#BuildPlan>). One Artifact must not use an output of another Artifact as an input. Each [Generator](<#Generator>) within an artifact also runs concurrently with generators of the same artifact. Each [Transformer](<#Transformer>) is executed sequentially starting after all generators have completed.
Output fields are write\-once. It is an error for multiple Generators or Transformers to produce the same Output value within the context of a [BuildPlan](<#BuildPlan>).
When directories are used as inputs or outputs, they behave similar to how \`git\` works with directories. When the output field references a directory, all files within the directory are recursively stored using their relative path as a key. Similar to git add . When the input field references an absent file, a / is appended and the resulting value is used as a prefix match against all previous task outputs.
```go
type Artifact struct {
Artifact FileOrDirectoryPath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Validators []Validator `json:"validators,omitempty" yaml:"validators,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
```
<a name="Auth"></a>
## type Auth {#Auth}
Auth represents environment variable names containing auth credentials.
```go
type Auth struct {
Username AuthSource `json:"username" yaml:"username"`
Password AuthSource `json:"password" yaml:"password"`
}
```
<a name="AuthSource"></a>
## type AuthSource {#AuthSource}
AuthSource represents a source for the value of an [Auth](<#Auth>) field.
```go
type AuthSource struct {
Value string `json:"value,omitempty" yaml:"value,omitempty"`
FromEnv string `json:"fromEnv,omitempty" yaml:"fromEnv,omitempty"`
}
```
<a name="BuildContext"></a>
## type BuildContext {#BuildContext}
BuildContext represents build context values owned by the holos render component command. End users should not manage context field values. End users may reference fields from within CUE to refer to late binding concrete values defined just before holos executes a [BuildPlan](<#BuildPlan>).
Holos injects build context values by marshalling this struct to json through the holos\_build\_context cue tag.
Example usage from cue to produce a [BuildPlan](<#BuildPlan>):
```
package holos
import (
"encoding/json"
"github.com/holos-run/holos/api/core/v1alpha6:core"
)
_BuildContextJSON: string | *"{}" @tag(holos_build_context, type=string)
BuildContext: core.#BuildContext & json.Unmarshal(_BuildContextJSON)
holos: core.#BuildPlan & {
buildContext: BuildContext
"spec": {
"artifacts": [
{
artifact: "components/slice",
"transformers": [
{
"kind": "Command"
"inputs": ["resources.gen.yaml"]
"output": artifact
"command": {
"args": [
"kubectl-slice",
"-f",
"\(buildContext.tempDir)/resources.gen.yaml",
"-o",
"\(buildContext.tempDir)/\(artifact)",
]
}
}
]
}
]
}
}
```
```go
type BuildContext struct {
// TempDir represents the temporary directory managed and owned by the holos
// render component command for the execution of one BuildPlan. Multiple
// tasks in the build plan share this temporary directory and therefore should
// avoid reading and writing into the same sub-directories as one another.
TempDir string `json:"tempDir" yaml:"tempDir" cue:"string | *\"${TEMP_DIR_PLACEHOLDER}\""`
// RootDir represents the fully qualified path to the platform root directory.
// Useful to construct arguments for commands in BuildPlan tasks.
RootDir string `json:"rootDir" yaml:"rootDir" cue:"string | *\"${ROOT_DIR_PLACEHOLDER}\""`
// LeafDir represents the cleaned path to the holos component relative to the
// platform root. Useful to construct arguments for commands in BuildPlan
// tasks.
LeafDir string `json:"leafDir" yaml:"leafDir" cue:"string | *\"${LEAF_DIR_PLACEHOLDER}\""`
// HolosExecutable represents the fully qualified path to the holos
// executable. Useful to execute tools embedded as subcommands such as holos
// cue vet.
HolosExecutable string `json:"holosExecutable" yaml:"holosExecutable" cue:"string | *\"holos\""`
}
```
<a name="BuildPlan"></a>
## type BuildPlan {#BuildPlan}
BuildPlan represents an implementation of the [rendered manifest pattern](<https://akuity.io/blog/the-rendered-manifests-pattern>). Holos processes a BuildPlan to produce one or more [Artifact](<#Artifact>) output files. BuildPlan artifact files usually contain Kubernetes manifests, but they may have any content.
A BuildPlan usually produces two artifacts. One artifact contains a manifest of resources. A second artifact contains a GitOps resource to manage the first, usually an ArgoCD Application resource.
Holos uses CUE to construct a BuildPlan. Holos injects late binding values such as the build temp dir using the [BuildContext](<#BuildContext>).
```go
type BuildPlan struct {
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"\"v1alpha6\""`
// Kind represents the type of the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
// BuildContext represents values injected by holos just before evaluating a
// BuildPlan, for example the tempDir used for the build.
BuildContext BuildContext `json:"buildContext" yaml:"buildContext"`
}
```
<a name="BuildPlanSpec"></a>
## type BuildPlanSpec {#BuildPlanSpec}
BuildPlanSpec represents the specification of the [BuildPlan](<#BuildPlan>).
```go
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos render platform command to skip the BuildPlan.
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
```
<a name="Chart"></a>
## type Chart {#Chart}
Chart represents a [Helm](<#Helm>) Chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
```
<a name="Command"></a>
## type Command {#Command}
Command represents a [BuildPlan](<#BuildPlan>) task implemented by executing an user defined system command. A task is defined as a [Generator](<#Generator>), [Transformer](<#Transformer>), or [Validator](<#Validator>). Commands are executed with the working directory set to the platform root.
```go
type Command struct {
// DisplayName of the command. The basename of args[0] is used if empty.
DisplayName string `json:"displayName,omitempty" yaml:"displayName,omitempty"`
// Args represents the argument vector passed to the os. to execute the
// command.
Args []string `json:"args,omitempty" yaml:"args,omitempty"`
// IsStdoutOutput captures the command stdout as the task output if true.
IsStdoutOutput bool `json:"isStdoutOutput,omitempty" yaml:"isStdoutOutput,omitempty"`
}
```
<a name="Component"></a>
## type Component {#Component}
Component represents the complete context necessary to produce a [BuildPlan](<#BuildPlan>) from a path containing parameterized CUE configuration.
```go
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path" yaml:"path"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Holos copies Labels
// from the Component to the resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `app.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
<a name="File"></a>
## type File {#File}
File represents a simple single file copy [Generator](<#Generator>). Useful with a [Kustomize](<#Kustomize>) [Transformer](<#Transformer>) to process plain manifest files stored in the component directory. Multiple File generators may be used to transform multiple resources.
```go
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source" yaml:"source"`
}
```
<a name="FileContent"></a>
## type FileContent {#FileContent}
FileContent represents file contents.
```go
type FileContent string
```
<a name="FileContentMap"></a>
## type FileContentMap {#FileContentMap}
FileContentMap represents a mapping of file paths to file contents.
```go
type FileContentMap map[FilePath]FileContent
```
<a name="FileOrDirectoryPath"></a>
## type FileOrDirectoryPath {#FileOrDirectoryPath}
FileOrDirectoryPath represents a file or a directory path.
```go
type FileOrDirectoryPath string
```
<a name="FilePath"></a>
## type FilePath {#FilePath}
FilePath represents a file path.
```go
type FilePath string
```
<a name="Generator"></a>
## type Generator {#Generator}
Generator generates Kubernetes resources. [Helm](<#Helm>) and [Resources](<#Resources>) are the most commonly used, often paired together to mix\-in resources to an unmodified Helm chart. A simple [File](<#File>) generator is also available for use with the [Kustomize](<#Kustomize>) transformer.
Each Generator in an [Artifact](<#Artifact>) must have a distinct Output value for a [Transformer](<#Transformer>) to reference.
1. [Resources](<#Resources>) \- Generates resources from CUE code.
2. [Helm](<#Helm>) \- Generates rendered yaml from a [Chart](<#Chart>).
3. [File](<#File>) \- Generates data by reading a file from the component directory.
4. [Command](<#Command>) \- Generates data by executing an user defined command.
```go
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\" | \"Command\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FileOrDirectoryPath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty" yaml:"file,omitempty"`
// Command generator. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm represents a [Chart](<#Chart>) manifest [Generator](<#Generator>).
```go
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart. Values follow ValueFiles when both are provided.
Values Values `json:"values" yaml:"values"`
// ValueFiles represents hierarchial value files passed in order to the helm
// template -f flag. Useful for migration from an ApplicationSet. Use Values
// instead. ValueFiles precede Values when both are provided.
ValueFiles []ValueFile `json:"valueFiles,omitempty" yaml:"valueFiles,omitempty"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
```
<a name="InternalLabel"></a>
## type InternalLabel {#InternalLabel}
InternalLabel is an arbitrary unique identifier internal to holos itself. The holos cli is expected to never write a InternalLabel value to rendered output files, therefore use a InternalLabel when the identifier must be unique and internal. Defined as a type for clarity and type checking.
```go
type InternalLabel string
```
<a name="Join"></a>
## type Join {#Join}
Join represents a [Transformer](<#Transformer>) using [bytes.Join](<https://pkg.go.dev/bytes#Join>) to concatenate multiple inputs into one output with a separator. Useful for combining output from [Helm](<#Helm>) and [Resources](<#Resources>) together into one [Artifact](<#Artifact>) when [Kustomize](<#Kustomize>) is otherwise unnecessary.
```go
type Join struct {
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
```
<a name="Kind"></a>
## type Kind {#Kind}
Kind is a discriminator. Defined as a type for clarity and type checking.
```go
type Kind string
```
<a name="Kustomization"></a>
## type Kustomization {#Kustomization}
Kustomization represents a kustomization.yaml file for use with the [Kustomize](<#Kustomize>) [Transformer](<#Transformer>). Untyped to avoid tightly coupling holos to kubectl versions which was a problem for the Flux maintainers. Type checking is expected to happen in CUE against the kubectl version the user prefers.
```go
type Kustomization map[string]any
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize represents a kustomization [Transformer](<#Transformer>).
```go
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
```
<a name="Metadata"></a>
## type Metadata {#Metadata}
Metadata represents data about the resource such as the Name.
```go
type Metadata struct {
// Name represents the resource name.
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `app.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform represents a platform to manage. A Platform specifies a [Component](<#Component>) collection and integrates the components together into a holistic platform. Holos iterates over the [Component](<#Component>) collection producing a [BuildPlan](<#BuildPlan>) for each, which holos then executes to render manifests.
Inspect a Platform resource holos would process by executing:
```
cue export --out yaml ./platform
```
```go
type Platform struct {
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha6\""`
// Kind is a string value representing the resource.
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
```
<a name="PlatformSpec"></a>
## type PlatformSpec {#PlatformSpec}
PlatformSpec represents the platform specification.
```go
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components" yaml:"components"`
}
```
<a name="Repository"></a>
## type Repository {#Repository}
Repository represents a [Helm](<#Helm>) [Chart](<#Chart>) repository.
The Auth field is useful to configure http basic authentication to the Helm repository. Holos gets the username and password from the environment variables represented by the Auth field.
```go
type Repository struct {
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
Auth Auth `json:"auth,omitempty" yaml:"auth,omitempty"`
}
```
<a name="Resource"></a>
## type Resource {#Resource}
Resource represents one kubernetes api object.
```go
type Resource map[string]any
```
<a name="Resources"></a>
## type Resources {#Resources}
Resources represents Kubernetes resources. Most commonly used to mix resources into the [BuildPlan](<#BuildPlan>) generated from CUE, but may be generated from elsewhere.
```go
type Resources map[Kind]map[InternalLabel]Resource
```
<a name="Transformer"></a>
## type Transformer {#Transformer}
Transformer combines multiple inputs from prior [Generator](<#Generator>) or [Transformer](<#Transformer>) outputs into one output. [Kustomize](<#Kustomize>) is the most commonly used transformer. A simple [Join](<#Join>) is also supported for use with plain manifest files.
1. [Kustomize](<#Kustomize>) \- Patch and transform the output from prior generators or transformers. See [Introduction to Kustomize](<https://kubectl.docs.kubernetes.io/guides/config_management/introduction/>).
2. [Join](<#Join>) \- Concatenate multiple prior outputs into one output.
3. [Command](<#Command>) \- Transforms data by executing an user defined command.
```go
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\" | \"Command\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FileOrDirectoryPath `json:"inputs" yaml:"inputs"`
// Output represents a file or directory for a subsequent Transformer or
// Artifact to consume.
Output FileOrDirectoryPath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
// Command transformer. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
```
<a name="Validator"></a>
## type Validator {#Validator}
Validator validates files. Useful to validate an [Artifact](<#Artifact>) prior to writing it out to the final destination. Holos may execute validators concurrently. See the [validators](<https://holos.run/docs/v1alpha6/tutorial/validators/>) tutorial for an end to end example.
```go
type Validator struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" yaml:"kind" cue:"\"Command\""`
// Inputs represents the files to validate. Usually the final Artifact.
Inputs []FileOrDirectoryPath `json:"inputs" yaml:"inputs"`
// Command validator. Ignored unless kind is Command.
Command Command `json:"command,omitempty" yaml:"command,omitempty"`
}
```
<a name="ValueFile"></a>
## type ValueFile {#ValueFile}
ValueFile represents one Helm value file produced from CUE.
```go
type ValueFile struct {
// Name represents the file name, e.g. "region-values.yaml"
Name string `json:"name" yaml:"name"`
// Kind is a discriminator.
Kind string `json:"kind" yaml:"kind" cue:"\"Values\""`
// Values represents values for holos to marshal into the file name specified
// by Name when rendering the chart.
Values Values `json:"values,omitempty" yaml:"values,omitempty"`
}
```
<a name="Values"></a>
## type Values {#Values}
Values represents [Helm](<#Helm>) Chart values generated from CUE.
```go
type Values map[string]any
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -1,3 +0,0 @@
# Core API
- [v1alpha2](v1alpha2)

View File

@@ -1,403 +0,0 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# v1alpha2
```go
import "github.com/holos-run/holos/api/core/v1alpha2"
```
Package v1alpha2 contains the core API contract between the holos cli and CUE configuration code. Platform designers, operators, and software developers use this API to write configuration in CUE which \`holos\` loads. The overall shape of the API defines imperative actions \`holos\` should carry out to render the complete yaml that represents a Platform.
[Platform](<#Platform>) defines the complete configuration of a platform. With the holos reference platform this takes the shape of one management cluster and at least two workload cluster. Each cluster has multiple [HolosComponent](<#HolosComponent>) resources applied to it.
Each holos component path, e.g. \`components/namespaces\` produces exactly one [BuildPlan](<#BuildPlan>) which in turn contains a set of [HolosComponent](<#HolosComponent>) kinds.
The primary kinds of [HolosComponent](<#HolosComponent>) are:
1. [HelmChart](<#HelmChart>) to render config from a helm chart.
2. [KustomizeBuild](<#KustomizeBuild>) to render config from [Kustomize](<#Kustomize>)
3. [KubernetesObjects](<#KubernetesObjects>) to render [APIObjects](<#APIObjects>) defined directly in CUE configuration.
Note that Holos operates as a data pipeline, so the output of a [HelmChart](<#HelmChart>) may be provided to [Kustomize](<#Kustomize>) for post\-processing.
## Index
- [Constants](<#constants>)
- [type APIObject](<#APIObject>)
- [type APIObjectMap](<#APIObjectMap>)
- [type APIObjects](<#APIObjects>)
- [type BuildPlan](<#BuildPlan>)
- [type BuildPlanComponents](<#BuildPlanComponents>)
- [type BuildPlanSpec](<#BuildPlanSpec>)
- [type Chart](<#Chart>)
- [type FileContent](<#FileContent>)
- [type FileContentMap](<#FileContentMap>)
- [type FilePath](<#FilePath>)
- [type HelmChart](<#HelmChart>)
- [type HolosComponent](<#HolosComponent>)
- [type Kind](<#Kind>)
- [type KubernetesObjects](<#KubernetesObjects>)
- [type Kustomize](<#Kustomize>)
- [type KustomizeBuild](<#KustomizeBuild>)
- [type Label](<#Label>)
- [type Metadata](<#Metadata>)
- [type Platform](<#Platform>)
- [type PlatformMetadata](<#PlatformMetadata>)
- [type PlatformSpec](<#PlatformSpec>)
- [type PlatformSpecComponent](<#PlatformSpecComponent>)
- [type Repository](<#Repository>)
## Constants
<a name="APIVersion"></a>
```go
const (
APIVersion = "v1alpha2"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)
```
<a name="KubernetesObjectsKind"></a>
```go
const KubernetesObjectsKind = "KubernetesObjects"
```
<a name="APIObject"></a>
## type APIObject {#APIObject}
APIObject represents the most basic generic form of a single kubernetes api object. Represented as a JSON object internally for compatibility between tools, for example loading from CUE.
```go
type APIObject structpb.Struct
```
<a name="APIObjectMap"></a>
## type APIObjectMap {#APIObjectMap}
APIObjectMap represents the marshalled yaml representation of kubernetes api objects. Do not produce an APIObjectMap directly, instead use [APIObjects](<#APIObjects>) to produce the marshalled yaml representation from CUE data, then provide the result to [HolosComponent](<#HolosComponent>).
```go
type APIObjectMap map[Kind]map[Label]string
```
<a name="APIObjects"></a>
## type APIObjects {#APIObjects}
APIObjects represents Kubernetes API objects defined directly from CUE code. Useful to mix in resources to any kind of [HolosComponent](<#HolosComponent>), for example adding an ExternalSecret resource to a [HelmChart](<#HelmChart>).
[Kind](<#Kind>) must be the resource kind, e.g. Deployment or Service.
[Label](<#Label>) is an arbitrary internal identifier to uniquely identify the resource within the context of a \`holos\` command. Holos will never write the intermediate label to rendered output.
Refer to [HolosComponent](<#HolosComponent>) which accepts an [APIObjectMap](<#APIObjectMap>) field provided by [APIObjects](<#APIObjects>).
```go
type APIObjects struct {
APIObjects map[Kind]map[Label]APIObject `json:"apiObjects"`
APIObjectMap APIObjectMap `json:"apiObjectMap"`
}
```
<a name="BuildPlan"></a>
## type BuildPlan {#BuildPlan}
BuildPlan represents a build plan for the holos cli to execute. The purpose of a BuildPlan is to define one or more [HolosComponent](<#HolosComponent>) kinds. For example a [HelmChart](<#HelmChart>), [KustomizeBuild](<#KustomizeBuild>), or [KubernetesObjects](<#KubernetesObjects>).
A BuildPlan usually has an additional empty [KubernetesObjects](<#KubernetesObjects>) for the purpose of using the [HolosComponent](<#HolosComponent>) DeployFiles field to deploy an ArgoCD or Flux gitops resource for the holos component.
```go
type BuildPlan struct {
Kind string `json:"kind" cue:"\"BuildPlan\""`
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
Spec BuildPlanSpec `json:"spec"`
}
```
<a name="BuildPlanComponents"></a>
## type BuildPlanComponents {#BuildPlanComponents}
```go
type BuildPlanComponents struct {
Resources map[Label]KubernetesObjects `json:"resources,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty"`
HelmChartList []HelmChart `json:"helmChartList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty"`
}
```
<a name="BuildPlanSpec"></a>
## type BuildPlanSpec {#BuildPlanSpec}
BuildPlanSpec represents the specification of the build plan.
```go
type BuildPlanSpec struct {
// Disabled causes the holos cli to take no action over the [BuildPlan].
Disabled bool `json:"disabled,omitempty"`
// Components represents multiple [HolosComponent] kinds to manage.
Components BuildPlanComponents `json:"components,omitempty"`
}
```
<a name="Chart"></a>
## type Chart {#Chart}
Chart represents a helm chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
```
<a name="FileContent"></a>
## type FileContent {#FileContent}
FileContent represents file contents.
```go
type FileContent string
```
<a name="FileContentMap"></a>
## type FileContentMap {#FileContentMap}
FileContentMap represents a mapping of file paths to file contents. Paths are relative to the \`holos\` output "deploy" directory, and may contain sub\-directories.
```go
type FileContentMap map[FilePath]FileContent
```
<a name="FilePath"></a>
## type FilePath {#FilePath}
FilePath represents a file path.
```go
type FilePath string
```
<a name="HelmChart"></a>
## type HelmChart {#HelmChart}
HelmChart represents a holos component which wraps around an upstream helm chart. Holos orchestrates helm by providing values obtained from CUE, renders the output using \`helm template\`, then post\-processes the helm output yaml using the general functionality provided by [HolosComponent](<#HolosComponent>), for example [Kustomize](<#Kustomize>) post\-rendering and mixing in additional kubernetes api objects.
```go
type HelmChart struct {
HolosComponent `json:",inline"`
Kind string `json:"kind" cue:"\"HelmChart\""`
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// ValuesContent represents the values.yaml file holos passes to the `helm
// template` command.
ValuesContent string `json:"valuesContent"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks" cue:"bool | *false"`
}
```
<a name="HolosComponent"></a>
## type HolosComponent {#HolosComponent}
HolosComponent defines the fields common to all holos component kinds. Every holos component kind should embed HolosComponent.
```go
type HolosComponent struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind"`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
// Metadata represents data about the holos component such as the Name.
Metadata Metadata `json:"metadata"`
// APIObjectMap holds the marshalled representation of api objects. Useful to
// mix in resources to each HolosComponent type, for example adding an
// ExternalSecret to a HelmChart HolosComponent. Refer to [APIObjects].
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty"`
// DeployFiles represents file paths relative to the cluster deploy directory
// with the value representing the file content. Intended for defining the
// ArgoCD Application resource or Flux Kustomization resource from within CUE,
// but may be used to render any file related to the build plan from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:"kustomize,omitempty"`
// Skip causes holos to take no action regarding this component.
Skip bool `json:"skip" cue:"bool | *false"`
}
```
<a name="Kind"></a>
## type Kind {#Kind}
Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
```go
type Kind string
```
<a name="KubernetesObjects"></a>
## type KubernetesObjects {#KubernetesObjects}
KubernetesObjects represents a [HolosComponent](<#HolosComponent>) composed of Kubernetes API objects provided directly from CUE using [APIObjects](<#APIObjects>).
```go
type KubernetesObjects struct {
HolosComponent `json:",inline"`
Kind string `json:"kind" cue:"\"KubernetesObjects\""`
}
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize represents resources necessary to execute a kustomize build. Intended for at least two use cases:
1. Process a [KustomizeBuild](<#KustomizeBuild>) [HolosComponent](<#HolosComponent>) which represents raw yaml file resources in a holos component directory.
2. Post process a [HelmChart](<#HelmChart>) [HolosComponent](<#HolosComponent>) to inject istio, patch jobs, add custom labels, etc...
```go
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty"`
}
```
<a name="KustomizeBuild"></a>
## type KustomizeBuild {#KustomizeBuild}
KustomizeBuild represents a [HolosComponent](<#HolosComponent>) that renders plain yaml files in the holos component directory using \`kubectl kustomize build\`.
```go
type KustomizeBuild struct {
HolosComponent `json:",inline"`
Kind string `json:"kind" cue:"\"KustomizeBuild\""`
}
```
<a name="Label"></a>
## type Label {#Label}
Label is an arbitrary unique identifier internal to holos itself. The holos cli is expected to never write a Label value to rendered output files, therefore use a [Label](<#Label>) then the identifier must be unique and internal. Defined as a type for clarity and type checking.
A Label is useful to convert a CUE struct to a list, for example producing a list of [APIObject](<#APIObject>) resources from an [APIObjectMap](<#APIObjectMap>). A CUE struct using Label keys is guaranteed to not lose data when rendering output because a Label is expected to never be written to the final output.
```go
type Label string
```
<a name="Metadata"></a>
## type Metadata {#Metadata}
Metadata represents data about the holos component such as the Name.
```go
type Metadata struct {
// Name represents the name of the holos component.
Name string `json:"name"`
// Namespace is the primary namespace of the holos component. A holos
// component may manage resources in multiple namespaces, in this case
// consider setting the component namespace to default.
//
// This field is optional because not all resources require a namespace,
// particularly CRD's and DeployFiles functionality.
// +optional
Namespace string `json:"namespace,omitempty"`
}
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform represents a platform to manage. A Platform resource informs holos which components to build. The platform resource also acts as a container for the platform model form values provided by the PlatformService. The primary use case is to collect the cluster names, cluster types, platform model, and holos components to build into one resource.
```go
type Platform struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha2\""`
// Metadata represents data about the object such as the Name.
Metadata PlatformMetadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
```
<a name="PlatformMetadata"></a>
## type PlatformMetadata {#PlatformMetadata}
```go
type PlatformMetadata struct {
// Name represents the Platform name.
Name string `json:"name"`
}
```
<a name="PlatformSpec"></a>
## type PlatformSpec {#PlatformSpec}
PlatformSpec represents the specification of a Platform. Think of a platform specification as a list of platform components to apply to a list of kubernetes clusters combined with the user\-specified Platform Model.
```go
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `json:"model"`
// Components represents a list of holos components to manage.
Components []PlatformSpecComponent `json:"components"`
}
```
<a name="PlatformSpecComponent"></a>
## type PlatformSpecComponent {#PlatformSpecComponent}
PlatformSpecComponent represents a holos component to build or render.
```go
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path"`
// Cluster is the cluster name to provide when rendering the component.
Cluster string `json:"cluster"`
}
```
<a name="Repository"></a>
## type Repository {#Repository}
Repository represents a helm chart repository.
```go
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -1,403 +0,0 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# v1alpha3
```go
import "github.com/holos-run/holos/api/core/v1alpha3"
```
Package v1alpha3 contains the core API contract between the holos cli and CUE configuration code. Platform designers, operators, and software developers use this API to write configuration in CUE which \`holos\` loads. The overall shape of the API defines imperative actions \`holos\` should carry out to render the complete yaml that represents a Platform.
[Platform](<#Platform>) defines the complete configuration of a platform. With the holos reference platform this takes the shape of one management cluster and at least two workload cluster. Each cluster has multiple [Component](<#Component>) resources applied to it.
Each holos component path, e.g. \`components/namespaces\` produces exactly one [BuildPlan](<#BuildPlan>) which in turn contains a set of [Component](<#Component>) kinds.
The primary kinds of [Component](<#Component>) are:
1. [HelmChart](<#HelmChart>) to render config from a helm chart.
2. [KustomizeBuild](<#KustomizeBuild>) to render config from [Kustomize](<#Kustomize>)
3. [KubernetesObjects](<#KubernetesObjects>) to render [APIObjects](<#APIObjects>) defined directly in CUE configuration.
Note that Holos operates as a data pipeline, so the output of a [HelmChart](<#HelmChart>) may be provided to [Kustomize](<#Kustomize>) for post\-processing.
## Index
- [Constants](<#constants>)
- [type APIObject](<#APIObject>)
- [type APIObjectMap](<#APIObjectMap>)
- [type APIObjects](<#APIObjects>)
- [type BuildPlan](<#BuildPlan>)
- [type BuildPlanComponents](<#BuildPlanComponents>)
- [type BuildPlanSpec](<#BuildPlanSpec>)
- [type Chart](<#Chart>)
- [type Component](<#Component>)
- [type FileContent](<#FileContent>)
- [type FileContentMap](<#FileContentMap>)
- [type FilePath](<#FilePath>)
- [type HelmChart](<#HelmChart>)
- [type InternalLabel](<#InternalLabel>)
- [type Kind](<#Kind>)
- [type KubernetesObjects](<#KubernetesObjects>)
- [type Kustomize](<#Kustomize>)
- [type KustomizeBuild](<#KustomizeBuild>)
- [type Metadata](<#Metadata>)
- [type Platform](<#Platform>)
- [type PlatformMetadata](<#PlatformMetadata>)
- [type PlatformSpec](<#PlatformSpec>)
- [type PlatformSpecComponent](<#PlatformSpecComponent>)
- [type Repository](<#Repository>)
## Constants
<a name="APIVersion"></a>
```go
const (
APIVersion = "v1alpha3"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)
```
<a name="KubernetesObjectsKind"></a>
```go
const KubernetesObjectsKind = "KubernetesObjects"
```
<a name="APIObject"></a>
## type APIObject {#APIObject}
APIObject represents the most basic generic form of a single kubernetes api object. Represented as a JSON object internally for compatibility between tools, for example loading from CUE.
```go
type APIObject structpb.Struct
```
<a name="APIObjectMap"></a>
## type APIObjectMap {#APIObjectMap}
APIObjectMap represents the marshalled yaml representation of kubernetes api objects. Do not produce an APIObjectMap directly, instead use [APIObjects](<#APIObjects>) to produce the marshalled yaml representation from CUE data, then provide the result to [Component](<#Component>).
```go
type APIObjectMap map[Kind]map[InternalLabel]string
```
<a name="APIObjects"></a>
## type APIObjects {#APIObjects}
APIObjects represents Kubernetes API objects defined directly from CUE code. Useful to mix in resources to any kind of [Component](<#Component>), for example adding an ExternalSecret resource to a [HelmChart](<#HelmChart>).
[Kind](<#Kind>) must be the resource kind, e.g. Deployment or Service.
[InternalLabel](<#InternalLabel>) is an arbitrary internal identifier to uniquely identify the resource within the context of a \`holos\` command. Holos will never write the intermediate label to rendered output.
Refer to [Component](<#Component>) which accepts an [APIObjectMap](<#APIObjectMap>) field provided by [APIObjects](<#APIObjects>).
```go
type APIObjects struct {
APIObjects map[Kind]map[InternalLabel]APIObject `json:"apiObjects"`
APIObjectMap APIObjectMap `json:"apiObjectMap"`
}
```
<a name="BuildPlan"></a>
## type BuildPlan {#BuildPlan}
BuildPlan represents a build plan for the holos cli to execute. The purpose of a BuildPlan is to define one or more [Component](<#Component>) kinds. For example a [HelmChart](<#HelmChart>), [KustomizeBuild](<#KustomizeBuild>), or [KubernetesObjects](<#KubernetesObjects>).
A BuildPlan usually has an additional empty [KubernetesObjects](<#KubernetesObjects>) for the purpose of using the [Component](<#Component>) DeployFiles field to deploy an ArgoCD or Flux gitops resource for the holos component.
```go
type BuildPlan struct {
Kind string `json:"kind" cue:"\"BuildPlan\""`
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha3\""`
Spec BuildPlanSpec `json:"spec"`
}
```
<a name="BuildPlanComponents"></a>
## type BuildPlanComponents {#BuildPlanComponents}
```go
type BuildPlanComponents struct {
Resources map[InternalLabel]KubernetesObjects `json:"resources,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty"`
HelmChartList []HelmChart `json:"helmChartList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty"`
}
```
<a name="BuildPlanSpec"></a>
## type BuildPlanSpec {#BuildPlanSpec}
BuildPlanSpec represents the specification of the build plan.
```go
type BuildPlanSpec struct {
// Disabled causes the holos cli to take no action over the [BuildPlan].
Disabled bool `json:"disabled,omitempty"`
// Components represents multiple [HolosComponent] kinds to manage.
Components BuildPlanComponents `json:"components,omitempty"`
}
```
<a name="Chart"></a>
## type Chart {#Chart}
Chart represents a helm chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
```
<a name="Component"></a>
## type Component {#Component}
Component defines the fields common to all holos component kinds. Every holos component kind should embed Component.
```go
type Component struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind"`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"\"v1alpha3\""`
// Metadata represents data about the holos component such as the Name.
Metadata Metadata `json:"metadata"`
// APIObjectMap holds the marshalled representation of api objects. Useful to
// mix in resources to each Component type, for example adding an
// ExternalSecret to a [HelmChart] Component. Refer to [APIObjects].
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty"`
// DeployFiles represents file paths relative to the cluster deploy directory
// with the value representing the file content. Intended for defining the
// ArgoCD Application resource or Flux Kustomization resource from within CUE,
// but may be used to render any file related to the build plan from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:"kustomize,omitempty"`
// Skip causes holos to take no action regarding this component.
Skip bool `json:"skip" cue:"bool | *false"`
}
```
<a name="FileContent"></a>
## type FileContent {#FileContent}
FileContent represents file contents.
```go
type FileContent string
```
<a name="FileContentMap"></a>
## type FileContentMap {#FileContentMap}
FileContentMap represents a mapping of file paths to file contents.
```go
type FileContentMap map[FilePath]FileContent
```
<a name="FilePath"></a>
## type FilePath {#FilePath}
FilePath represents a file path.
```go
type FilePath string
```
<a name="HelmChart"></a>
## type HelmChart {#HelmChart}
HelmChart represents a holos component which wraps around an upstream helm chart. Holos orchestrates helm by providing values obtained from CUE, renders the output using \`helm template\`, then post\-processes the helm output yaml using the general functionality provided by [Component](<#Component>), for example [Kustomize](<#Kustomize>) post\-rendering and mixing in additional kubernetes api objects.
```go
type HelmChart struct {
Component `json:",inline"`
Kind string `json:"kind" cue:"\"HelmChart\""`
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// ValuesContent represents the values.yaml file holos passes to the `helm
// template` command.
ValuesContent string `json:"valuesContent"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks" cue:"bool | *false"`
}
```
<a name="InternalLabel"></a>
## type InternalLabel {#InternalLabel}
InternalLabel is an arbitrary unique identifier internal to holos itself. The holos cli is expected to never write a InternalLabel value to rendered output files, therefore use a [InternalLabel](<#InternalLabel>) when the identifier must be unique and internal. Defined as a type for clarity and type checking.
A InternalLabel is useful to convert a CUE struct to a list, for example producing a list of [APIObject](<#APIObject>) resources from an [APIObjectMap](<#APIObjectMap>). A CUE struct using InternalLabel keys is guaranteed to not lose data when rendering output because a InternalLabel is expected to never be written to the final output.
```go
type InternalLabel string
```
<a name="Kind"></a>
## type Kind {#Kind}
Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
```go
type Kind string
```
<a name="KubernetesObjects"></a>
## type KubernetesObjects {#KubernetesObjects}
KubernetesObjects represents a [Component](<#Component>) composed of Kubernetes API objects provided directly from CUE using [APIObjects](<#APIObjects>).
```go
type KubernetesObjects struct {
Component `json:",inline"`
Kind string `json:"kind" cue:"\"KubernetesObjects\""`
}
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize represents resources necessary to execute a kustomize build. Intended for at least two use cases:
1. Process a [KustomizeBuild](<#KustomizeBuild>) [Component](<#Component>) which represents raw yaml file resources in a holos component directory.
2. Post process a [HelmChart](<#HelmChart>) [Component](<#Component>) to inject istio, patch jobs, add custom labels, etc...
```go
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty"`
}
```
<a name="KustomizeBuild"></a>
## type KustomizeBuild {#KustomizeBuild}
KustomizeBuild represents a [Component](<#Component>) that renders plain yaml files in the holos component directory using \`kubectl kustomize build\`.
```go
type KustomizeBuild struct {
Component `json:",inline"`
Kind string `json:"kind" cue:"\"KustomizeBuild\""`
}
```
<a name="Metadata"></a>
## type Metadata {#Metadata}
Metadata represents data about the object such as the Name.
```go
type Metadata struct {
// Name represents the name of the holos component.
Name string `json:"name"`
// Namespace is the primary namespace of the holos component. A holos
// component may manage resources in multiple namespaces, in this case
// consider setting the component namespace to default.
//
// This field is optional because not all resources require a namespace,
// particularly CRDs and DeployFiles functionality.
// +optional
Namespace string `json:"namespace,omitempty"`
}
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform represents a platform to manage. A Platform resource informs holos which components to build. The platform resource also acts as a container for the platform model form values provided by the PlatformService. The primary use case is to collect the cluster names, cluster types, platform model, and holos components to build into one resource.
```go
type Platform struct {
// Kind is a string value representing the resource this object represents.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this representation of an object.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha3\""`
// Metadata represents data about the object such as the Name.
Metadata PlatformMetadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
```
<a name="PlatformMetadata"></a>
## type PlatformMetadata {#PlatformMetadata}
```go
type PlatformMetadata struct {
// Name represents the Platform name.
Name string `json:"name"`
}
```
<a name="PlatformSpec"></a>
## type PlatformSpec {#PlatformSpec}
PlatformSpec represents the specification of a Platform. Think of a platform specification as a list of platform components to apply to a list of kubernetes clusters combined with the user\-specified Platform Model.
```go
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `json:"model" cue:"{...}"`
// Components represents a list of holos components to manage.
Components []PlatformSpecComponent `json:"components"`
}
```
<a name="PlatformSpecComponent"></a>
## type PlatformSpecComponent {#PlatformSpecComponent}
PlatformSpecComponent represents a holos component to build or render.
```go
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path"`
// Cluster is the cluster name to provide when rendering the component.
Cluster string `json:"cluster"`
}
```
<a name="Repository"></a>
## type Repository {#Repository}
Repository represents a helm chart repository.
```go
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -1,101 +0,0 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# v1alpha3
```go
import "github.com/holos-run/holos/api/schema/v1alpha3"
```
Package v1alpha3 contains CUE definitions intended as convenience wrappers around the core data types defined in package core. The purpose of these wrappers is to make life easier for platform engineers by reducing boiler plate code and generating component build plans in a consistent manner.
## Index
- [type ArgoConfig](<#ArgoConfig>)
- [type Helm](<#Helm>)
<a name="ArgoConfig"></a>
## type ArgoConfig {#ArgoConfig}
ArgoConfig represents the ArgoCD GitOps configuration for a Component. Useful to define once at the root of the Platform configuration and reuse across all Components.
```go
type ArgoConfig struct {
// Enabled causes holos to render an ArgoCD Application resource for GitOps if true.
Enabled bool `cue:"true | *false"`
// ClusterName represents the cluster within the platform the Application
// resource is intended for.
ClusterName string
// DeployRoot represents the path from the git repository root to the `deploy`
// rendering output directory. Used as a prefix for the
// Application.spec.source.path field.
DeployRoot string `cue:"string | *\".\""`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm provides a BuildPlan via the Output field which contains one HelmChart from package core. Useful as a convenience wrapper to render a HelmChart with optional mix\-in resources and Kustomization post\-processing.
```go
type Helm struct {
// Name represents the chart name.
Name string
// Version represents the chart version.
Version string
// Namespace represents the helm namespace option when rendering the chart.
Namespace string
// Resources are kubernetes api objects to mix into the output.
Resources map[string]any `cue:"{...}"`
// Repo represents the chart repository
Repo struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Values represents data to marshal into a values.yaml for helm.
Values interface{} `cue:"{...}"`
// Chart represents the derived HelmChart for inclusion in the BuildPlan
// Output field value. The default HelmChart field values are derived from
// other Helm field values and should be sufficient for most use cases.
Chart core.HelmChart
// EnableKustomizePostProcessor processes helm output with kustomize if true.
EnableKustomizePostProcessor bool `cue:"true | *false"`
// KustomizeFiles represents additional files to include in a Kustomization
// resources list. Useful to patch helm output. The implementation is a
// struct with filename keys and structs as values. Holos encodes the struct
// value to yaml then writes the result to the filename key. Component
// authors may then reference the filename in the kustomization.yaml resources
// or patches lists.
// Requires EnableKustomizePostProcessor: true.
KustomizeFiles map[string]any `cue:"{[string]: {...}}"`
// KustomizePatches represents patches to apply to the helm output. Requires
// EnableKustomizePostProcessor: true.
KustomizePatches map[core.InternalLabel]any `cue:"{[string]: {...}}"`
// KustomizeResources represents additional resources files to include in the
// kustomize resources list.
KustomizeResources map[string]any `cue:"{[string]: {...}}"`
// ArgoConfig represents the ArgoCD GitOps configuration for this Component.
ArgoConfig ArgoConfig
// Output represents the derived BuildPlan for the Holos cli to render.
Output core.BuildPlan
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -1,3 +0,0 @@
# Backstory
Holos is a tool intended to lighten the burden of managing Kubernetes resources. In 2020 we set out to develop a holistic platform composed from open source cloud native components. We quickly became frustrated with how each of the major components packaged and distributed their software in a different way. Many projects choose to distribute their software with Helm charts, while others provide plain yaml files and Kustomize bases. The popular Kube Prometheus Stack project provides Jsonnet to render and update Kubernetes yaml manifests.

View File

@@ -1,3 +0,0 @@
# CLI
Use the `holos` command line interface (CLI) to render individual platform components, entire platforms, and push/pull the platform model.

View File

@@ -0,0 +1,16 @@
Integrate the `podinfo` component into the platform.
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```

View File

@@ -0,0 +1,34 @@
Create a directory for the example `podinfo` component we'll use to render
platform manifests.
```bash
mkdir -p components/podinfo
```
Create the CUE configuration for the example `podinfo` component.
```bash
cat <<EOF >components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
```bash
EOF
```

View File

@@ -1,65 +0,0 @@
# Comparison with other tools
:::tip
Holos is designed to complement and improve, not replace, existing tools in the cloud native ecosystem.
:::
## Helm
### Chart Users
Describe how things are different when using an upstream helm chart.
### Chart Authors
Describe how things are different when writing a new helm chart.
## Kustomize
TODO
## ArgoCD
TODO
## Flux
TODO
## Timoni
| Aspect | Timoni | Holos | Comment |
| -- | -- | -- | -- |
| Language | CUE | CUE | Like Holos, Timoni is also built on CUE. |
| Artifact | OCI Image | Plain YAML Files | The Holos Authors find plain files easier to work with and reason about than OCI images. |
| Outputs to | OCI Image Repository | Local Git repository | Holos is designed for use with existing GitOps tools. |
| Concept | Module | Component | A Timoni Module is analogous to a Holos Component. |
| Concept | Bundle | Platform | A Timoni Bundle is somewhat similar, but smaller in scope to a Holos Platform. |
:::important
The Holos Authors are deeply grateful to Stefan and Timoni for the capability of
importing Kubernetes custom resource definitions into CUE. Without this
functionality, much of the Kubernetes ecosystem would be more difficult to
manage in CUE and therefore in Holos.
:::
## KubeVela
1. Also built on CUE.
2. Intended to create an Application abstraction.
3. Holos prioritizes composition over abstraction.
4. An abstraction of an Application acts as a filter that removes all but the lowest common denominator functionality. The Holos Authors have found this filtering effect to create excessive friction for software developers.
5. Holos focuses instead on composition to empower developers and platform engineers to leverage the unique features and functionality of their software and platform.
## Pulumi
TODO
## Jsonnet
TODO

View File

@@ -1,341 +0,0 @@
# Concepts
## Introduction
This page is intended as a high level conceptual overview of the key concepts in
Holos. Refer to the [Core API](/docs/api/core/) for low level reference
documentation.
Holos is a tool built for platform engineers. The Holos authors share three
core values which guide our design decisions for the tool.
1. Safety
2. Ease of use
3. Consistency
Each of the following concepts are intended to support and strengthen one or
more of these core values. In this way we hope to lighten the burden carried by
platform engineers.
## Concepts
- [Component](<#component>) - The primary building block in Holos, wraps a Helm chart, Kustomize base, or plain resources defined in CUE.
- [Platform](<#platform>) - A collection of Components integrated into a software development platform.
- [Model](<#model>) - Structured data included in the Platform specification, available to all Components. For example, your organization's domain name.
- [Rendering](<#rendering>) - Holos is a tool that makes the process of rendering Kubernetes manifests safer, easier, and consistent.
```mermaid
graph BT
Platform[<a href="#platform">Platform</a>]
Component[<a href="#component">Components</a>]
Helm[<a href="#component">Helm</a>]
Kustomize[<a href="#component">Kustomize</a>]
CUE[<a href="#component">CUE</a>]
Component --> Platform
Helm --> Component
Kustomize --> Component
CUE --> Component
```
<!--
```mermaid
---
title: Figure 1 - Holos Concepts
---
mindmap
root((Holos))
Platform
Components
HelmChart
KustomizeBuild
KubernetesObjects
Model
name: Example Org
domain: example.com
Renders
YAML Files
Kubernetes Manifests
ArgoCD Application
FluxCD Kustomization
```
-->
## Component
A Component is the primary building block when managing software with Holos. A
software project you wish to integrate into your platform, for example ArgoCD,
is managed using one or more components.
The primary Component kinds are:
1. **HelmChart** to render config provided by Helm.
2. **KustomizeBuild** to render config provided by Kustomize.
3. **KubernetesObjects** to render config provided by CUE.
Components are intended to integrate unmodified upstream software releases into
your Platform. In this way, the focus of a Component is more about the unique
differentiating aspects of your platform than the upstream software contained in
the Component.
#### Example HelmChart Component
The ArgoCD Component is a good example of a HelmChart component because it takes
advantage of most of the key features that empower you to focus on the key
differentiators of your unique platform.
Take note of the following key points in this example ArgoCD Component:
1. The Component wraps the ArgoCD Helm Chart in a way that's easy to upgrade and maintain over time.
2. Newer Gateway API resources are mixed-in replacing the older Ingress resource included in the chart.
3. Helm output is passed through Kustomize to configure secure mutual TLS encryption.
4. Helm values are easier and safer to manipulate with CUE instead of text markup.
5. Kustomize is easier and safer to manipulate with CUE instead of text markup.
6. Platform data Model values are easily accessible, for example the OIDC issuer and the organizations's domain name.
The Component wraps around the unmodified upstream ArgoCD helm chart
providing easier upgrades as new versions are released.
Note how the Component facilitates composition by allowing us to mix-in new
functionality from the ecosystem without modifying the upstream chart. The
Platform this Component integrates with uses the new Gateway API, but the
upstream helm chart does not yet support Gateway API. See how the Resources
field is used to mix-in a ReferenceGrant from the Gateway API without modifying
the upstream helm chart.
The Platform uses Istio to implement service to service encryption with mutual
TLS. The Component passes the Helm output to Kustomize to integrate with Istio.
Kustomize is used to patch the argocd-server Deployment resource to inject the
Istio sidecar for mutual TLS.
Helm values are safer and easier to work with in CUE. Note how you can modify
helm values using well defined data instead of manipulating text yaml files.
Similarly, the yaml files used for Kustomize are produced by CUE, which is again
safer and easier because the Kustomize spec has been imported into CUE and is
validated.
Finally, the domain name used by this Platform is easily accessible from the
PlatformSpec which is defined at the root level and made available to all
components integrated into the platform. Similarly, data values shared by all
of the Components that make up ArgoCD is defined in a structure accessible by
each of these components.
```cue
package holos
import (
"encoding/yaml"
"strings"
)
// Produce a helm chart build plan.
(#Helm & Chart).Output
let Chart = {
Name: "argo-cd"
Namespace: "argocd"
Version: "7.1.1"
Chart: chart: release: "argocd"
// The upstream chart uses a Job to create the argocd-redis Secret. Enable
// hooks to enable the Job.
Chart: enableHooks: true
Repo: name: "argocd"
Repo: url: "https://argoproj.github.io/argo-helm"
// Ensure all of our mix-in resources go into the same namespace as the Chart.
Resources: [_]: [_]: metadata: namespace: Namespace
// Grant the Gateway namespace the ability to refer to the backend service
// from HTTPRoute resources.
Resources: ReferenceGrant: (#IstioGatewaysNamespace): #ReferenceGrant
// Pass the helm output through kustomize.
EnableKustomizePostProcessor: true
// Force all resources into the component namespace, some resources in the
// helm chart do not specify the namespace so they will get mis-applied
// when the kubectl (client-go) context is another namespace.
KustomizeFiles: "kustomization.yaml": namespace: Namespace
// Patch the backend with the service mesh sidecar.
KustomizePatches: {
mesh: {
target: {
group: "apps"
version: "v1"
kind: "Deployment"
name: "argocd-server"
}
patch: yaml.Marshal(IstioInject)
}
}
Values: #Values & {
kubeVersionOverride: "1.29.0"
// handled in the argo-crds component
crds: install: false
global: domain: "argocd.\(_Platform.Model.org.domain)"
dex: enabled: false
// the service mesh handles secure mTLS
configs: params: "server.insecure": true
configs: cm: {
"admin.enabled": false
"oidc.config": yaml.Marshal(OIDCConfig)
"users.anonymous.enabled": "false"
}
// Refer to https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/
let Policy = [
"g, argocd-view, role:readonly",
"g, prod-cluster-view, role:readonly",
"g, prod-cluster-edit, role:readonly",
"g, prod-cluster-admin, role:admin",
]
configs: rbac: "policy.csv": strings.Join(Policy, "\n")
}
}
let IstioInject = [{
op: "add",
path: "/spec/template/metadata/labels/sidecar.istio.io~1inject",
value: "true",
}]
let OIDCConfig = {
name: "Holos Platform"
issuer: _ArgoCD.issuerURL
clientID: _ArgoCD.clientID
requestedScopes: _ArgoCD.scopesList
// Set redirect uri to https://argocd.example.com/pkce/verify
enablePKCEAuthentication: true
// groups is essential for rbac
requestedIDTokenClaims: groups: essential: true
}
```
## Platform
A Platform refers to all of the software and services integrated together to
provide your organization's software development platform. Holos is designed to
manage all of the resources that compose your Platform using the [Kubernetes
Resource Model][krm] (KRM). Nearly all platforms are larger than Kubernetes
itself. For example, your developers likely need a GCS or S3 bucket to store
data. Holos takes advantage of Crossplane to manage resources in a consistent
way.
Holos defines a [Platform][Platform] object which collects multiple Components
together along with organizational data defined by your Model. Consider the
following example, which is a Platform that manages a single Component which
manages namespaces for each cluster in the Platform.
```cue
package holos
import v1 "github.com/holos-run/holos/api/v1alpha2"
v1.#Platform & {
metadata: name: "example"
spec: components: [{
path: "components/namespaces"
cluster: "cluster1"
}]
}
```
This platform is rendered by the command:
```bash
holos render platform ./platform
```
When Holos renders the platform, it iterates over each component, generates and
executes a [BuildPlan][BuildPlan], then writes the fully rendered output of the
component to the filesystem. In this simple example, two files are produced:
1. `deploy/clusters/cluster1/components/namespaces/namespaces.gen.yaml`
2. `deploy/clusters/cluster1/gitops/namespaces.application.gen.yaml`
The first file is a plain kubernetes manifest containing Namespace resources.
The second file is an ArgoCD Application resource to deploy and manage the
resources defined in the first file.
## Model
The Platform Model is where you store top-level data values used throughout
multiple components in your Platform. Your organization's domain name is a
prime example of the kind of data stored in the Model. Many components derive
host names from your organization's domain name. CUE makes this process safe,
easy, and consistent. For example:
```cue
hostname: "argocd.\(_Platform.Model.org.domain)"
```
When Holos renders a Platform, the model is loaded from a JSON file in the local
filesystem. The platform model file is intended to be committed to version
control along with the rest of the Holos Platform and Component code.
Holos additionally provides a web ui and form to make it easy to enter and
validate top level configuration data. You have complete control over the web
form, it's rendered from JSON data defined by CUE. Customizing the web form is
an advanced topic, the key concept to take away is the Model is for top level,
platform-wide data. You control the shape and structure of the Model, and you
have the ability to collect Model values using a simple web form.
## Rendering
Holos uses the Kubernetes resource model to manage configuration. The Holos
command line interface is the primary method you'll use to manage your platform.
Holos uses CUE to provide a unified configuration model of the platform. This
unified configuration is built up from components packaged with Helm, Kustomize,
CUE, or any other tool that can produce Kubernetes resource manifests as output.
This process can be thought of as a data **rendering pipeline**. The key
concept is that Holos will always produce fully rendered output, but delegates
the _application_ of the configuration to other tools like kubectl apply,
ArgoCD, or Flux.
```mermaid
---
title: Figure 2 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/core/v1alpha2#PlatformSpec">PlatformSpec</a>]
BP[<a href="/docs/api/core/v1alpha2#BuildPlan">BuildPlan</a>]
HC[<a href="/docs/api/core/v1alpha2#HolosComponent">Components</a>]
H[<a href="/docs/api/core/v1alpha2#HelmChart">HelmChart</a>]
K[<a href="/docs/api/core/v1alpha2#KustomizeBuild">KustomizeBuild</a>]
O[<a href="/docs/api/core/v1alpha2#KubernetesObjects">KubernetesObjects</a>]
P[<a href="/docs/api/core/v1alpha2#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
C[Kube API Server]
PS --> BP --> HC
HC --> H --> P
HC --> K --> P
HC --> O --> P
P --> Y --> FS
P --> G --> FS
FS --> ArgoCD --> C
FS --> Flux --> C
FS --> kubectl --> C
```
[krm]: https://docs.google.com/document/d/1RmHXdLhNbyOWPW_AtnnowaRfGejw-qlKQIuLKQWlwzs/view#heading=h.sa6p0aye4ide
[Platform]: /docs/api/core/v1alpha2/#Platform
[BuildPlan]: /docs/api/core/v1alpha2/#BuildPlan

View File

@@ -1,47 +0,0 @@
# Rendering
:::tip
This document provides a brief overview of the rendering process, a core design
element in Holos.
:::
Holos uses the Kubernetes resource model to manage configuration. The `holos`
command line interface is the primary method you'll use to manage your platform.
Holos uses CUE to provide a unified configuration model of the platform. This
unified configuration is built up from components packaged with Helm, Kustomize,
CUE, or any other tool that can produce Kubernetes resource manifests as output.
This process can be thought of as a data **rendering pipeline**. The key
concept is that `holos` will always produce fully rendered output, but delegates
the _application_ of the configuration to other tools like `kubectl apply`,
ArgoCD, or Flux.
```mermaid
---
title: Figure 2 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/core/v1alpha2#PlatformSpec">PlatformSpec</a>]
BP[<a href="/docs/api/core/v1alpha2#BuildPlan">BuildPlan</a>]
HC[<a href="/docs/api/core/v1alpha2#HolosComponent">HolosComponent</a>]
H[<a href="/docs/api/core/v1alpha2#HelmChart">HelmChart</a>]
K[<a href="/docs/api/core/v1alpha2#KustomizeBuild">KustomizeBuild</a>]
O[<a href="/docs/api/core/v1alpha2#KubernetesObjects">KubernetesObjects</a>]
P[<a href="/docs/api/core/v1alpha2#Kustomize">Kustomize</a>]
Y[Kubernetes <br>Resources]
G[GitOps <br>Resource]
C[Kube API Server]
PS --> BP --> HC
HC --> H --> P
HC --> K --> P
HC --> O --> P
P --> Y --> C
P --> G --> C
```

Some files were not shown because too many files have changed in this diff Show More