Compare commits

...

53 Commits

Author SHA1 Message Date
Jeff McCune
6002040360 version 0.100.0 2024-11-20 11:34:04 -08:00
Jeff McCune
864d7d442b build: pass platform component labels and annotations to BuildPlan
Without this patch the BuildPlan resulting from a Platform that has
components with labels and annotations does not have the labels or
annotations of the source component.

Holos should copy the labels and annotations defined on each of the
Platform.spec.components to the resulting BuildPlan so end users can see
clearly where a BuildPlan originated from, and filter with selectors the
intermediate output BuildPlan the same way we filter with selectors the
original Platform spec components list.

Result:

```
holos init platform v1alpha5 --force
holos show buildplans  | head
```

```yaml
kind: BuildPlan
apiVersion: v1alpha5
metadata:
  name: podinfo
  labels:
    app.holos.run/cluster: local
    app.holos.run/name: podinfo
  annotations:
    app.holos.run/description: podinfo for cluster local
```
2024-11-20 11:34:03 -08:00
Jeff McCune
791c0a9ffd util: fix misleading RunCmd error message source location
The error message misleads the reader to the utility function.  It
should lead to the caller.
2024-11-20 11:23:54 -08:00
Jeff McCune
bfe4a8d7c4 remove unmaintained embedded platforms
v1alpha5 is current and maintained.
2024-11-20 11:23:54 -08:00
Jeff McCune
eaa508a6f8 remove v1alpha1 schemas and support 2024-11-20 11:23:53 -08:00
Jeff McCune
63256a2845 remove embedded k3d platform
No longer used, v1alpha5 is used in the docs and maintained.
2024-11-20 11:23:53 -08:00
Jeff McCune
937c1dc953 show: fix inconsistent ordering of output
Without this patch the holos show buildplans command returns results in
an inconsistent order.  This is a problem because the output should be
idempotent.

This patch fixes the problem by adding an EncodeSeq(idx int, v any) method to
the encoder interface.  idx represents the index position of the
Platform.spec.components list after selector filtering has been applied.

This patch modifies the json and yaml encoders to buffer out of order
results from the concurrent go routines.

Result:

Concurrent execution is preserved. The buffer is kept to a reasonable
size, entries are deleted once they're encoded in the correct order.

Most importantly the output is consistent and idempotent so we can write
effective integration tests.
2024-11-20 11:23:52 -08:00
Jeff McCune
11bd50e2eb show: fix buildplans selector inconsistency
Sometimes, but not always, the holos show buildplans command produces no
output.

```
❯ holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug
finalized config from flags
rendered platform in 13.458µs
```

It only happens when there's a selector.  It doesn't happen without the
selector flag.  It only happens with ==, not with =.

This test fails quickly.

```
while [[ $(holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug | wc -l) -eq 39 ]]; do true; done
```

This test runs until killed.

```
while [[ $(holos show buildplans --log-level=debug | wc -l) -eq 279 ]]; do true; done
```

Solution:

The problem is the use of the map.  Iterating over the keys happens in a
random order.  With the fix we check in an explicit order.
2024-11-20 11:23:52 -08:00
Jeff McCune
7cfcf55565 add yaml struct tags to core v1alpha5 schemas
Without this patch the `holos show buildplans` BuildPlan output has
incorrect yaml with the v3 encoder.  For example apiversion: v1alpha5
instead of apiVersion.
2024-11-20 11:23:52 -08:00
Jeff McCune
8b22ba04e1 cli: add show command and refactor interfaces (#331)
Show subcommand:

This is large change that accomplishes a number of goals.  First, there
was no convenient way to show a build plan without using the debug logs
to indentify the tags to inject, then calling the cue command with the
right incantation to inspect the BuildPlan.

This patch addresses the problem by adding a `holos show buildplans`
command.  The command loads the Platform spec from the platform
directory, then iterates over all Components to produce the BuildPlan.

This patch adds labels and annotations to the platform Components
collection in order to select and filter the output.

Result:

```
❯ holos show components --selector app.holos.run/cluster=local --format=yaml | head
kind: BuildPlan
apiversion: v1alpha5
metadata:
  name: podinfo
spec:
  artifacts:
    - artifact: clusters/local/components/podinfo/podinfo.gen.yaml
      generators:
        - kind: Helm
          output: helm.gen.yaml
```

---

Interface refactor:

This refactors the interface between the `holos` Go CLI layer and the
various core schema data structures.  We now use a proper Go interface.
Concurrent execution over platform components has been improved to
accept a closure function so we can use the same interface method to
process the components.  We use this to show each component and render
each component from different subcommands using the same interface
embedded in the builder.Platform struct.

The embedded interface allows us to easily swap in different versions,
e.g. v1beta1 and eventually v1.  The number of interface methods are
quite small.  14 methods across 4 interfaces in holos/interface.go.

---

Remove old versions:

This patch removes support for versions prior to v1alpha5 in an effort
to clean up cruft.
2024-11-20 11:23:51 -08:00
Nate McCurdy
7d5187873b docs: Remove non-existent CLI completions, add pwsh
Cobra provides completions for bash, zsh, fish, and powershell, not
ksh.
2024-11-18 10:27:33 -08:00
Jeff McCune
03b796312a cli: gate grpc client and auth flags behind feature flag
Previously the holos render platform and component subcommands had flags
for oidc authentication and client access to the gRPC service.  These
flags aren't currently used, they're remnants from the json powered form
prototype.

This patch gates the flags behind a feature flag which is disabled by
default.

Result:

  holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int   number of components to render concurrently (default 8)
  -v, --version           version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")

---

  HOLOS_FEATURE_CLIENT=1 holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int             number of components to render concurrently (default 8)
      --oidc-client-id string       oidc client id. (default "270319630705329162@holos_platform")
      --oidc-extra-scopes strings   optional oidc scopes
      --oidc-force-refresh          force refresh
      --oidc-issuer string          oidc token issuer url. (default "https://login.holos.run")
      --oidc-scopes strings         required oidc scopes (default openid,email,profile,groups,offline_access)
      --server string               server to connect to (default "https://app.holos.run:443")
  -v, --version                     version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")
2024-11-17 16:06:57 -08:00
Jeff McCune
20fb39e49b docs: add clusters topic (#343)
Previously we didn't have good documentation on how to manage multiple
sets of clusters.

This patch adds a clusters topic in the structures category.  Each one
of the environments, projects, owners, etc... structures follow the same
pattern as #Clusters and #ClusterSets, so it makes sense to put them
into a dedicated sidebar category for specific CUE structures.
2024-11-17 14:45:32 -08:00
Jeff McCune
c9c8c13810 docs: replace touch with cat 2024-11-15 09:54:09 -07:00
Jeff McCune
374cd872e9 docs: CUE not cue and typo fix in hello holos 2024-11-15 09:40:02 -07:00
Jeff McCune
8db06dd0e1 releaser: fix brew test command (#327)
holos version isn't a valid command but holos --version is.
2024-11-14 16:14:26 -07:00
Jeff McCune
66acadf86d docs: support brew install (#327) 2024-11-14 16:07:13 -07:00
Jeff McCune
032f72b435 render: log helm pull errors (#332)
Previously errors were not logged, giving no indication what went wrong.
This patch changes the error handler to log errors from helm.
2024-11-14 09:44:27 -07:00
Jeff McCune
2380223794 docs: add argocd application example (#340)
When we moved from v1alpha4 to v1alpha5 we removed ArgoConfig from the
author schema.  There was no longer a clear example of how to configure
an ArgoCD Application for every component in v1alpha5.

This patch adds a topic document with an example of how to add an
Application along side the resources by mixing an additional Artifact
into the BuildPlan.
2024-11-13 16:30:59 -07:00
Jeff McCune
e6892c3b16 v0.99.0 2024-11-13 12:49:28 -07:00
Jeff McCune
847fd2958e helm: add support for helm template --kube-version capabilities (#330)
Previously the Helm generator had no support for the --kube-version
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:43:01 -07:00
Jeff McCune
cf622835db helm: add support for helm template --api-versions capabilities (#330)
Previously the Helm generator had no support for the --api-versions
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:42:50 -07:00
Jeff McCune
1f5dc3a082 docs: add note about tested helm version (#335)
To help users understand what should definitely work.
2024-11-13 09:45:56 -07:00
Jeff McCune
9f4da68dc9 v0.98.2 2024-11-13 09:19:30 -07:00
Jeff McCune
2ee056be9f cue: fix panic with no args (#334)
Fixes:

```
❯ holos
panic: runtime error: slice bounds out of range [2:1]

goroutine 1 [running]:
github.com/holos-run/holos/internal/cli.newCueCmd(...)
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:121
github.com/holos-run/holos/internal/cli.New(0xc0002837c0, {0x3826e00, 0x4f60860})
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:102 +0x772
main.main.MakeMain.func1()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/main.go:22 +0x5b
main.main()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/cmd/holos/main.go:10 +0x3e
```
2024-11-13 09:04:37 -07:00
Jeff McCune
394e2cb0b2 docs: add cue tutorial (#318)
Show how to use the ComponentConfig Resources field to mix in resources.
2024-11-13 08:00:37 -07:00
Jeff McCune
cf95c9664d docs: change hello holos parameters to greeting (#328)
Version doesn't make as much sense since we're doing a hello world
tutorial.

Also consolidate the values information into one step, and consolidate
the breaking it down section to make it shorter and clearer.
2024-11-12 09:46:19 -07:00
Jeff McCune
0192eeeb7e docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest

This time in the correct directory.
2024-11-11 17:25:17 -07:00
Jeff McCune
ed54bcc58f docs: rename cue-generator to cue
The main use case is to manage resources from CUE, but CUE has many uses
in Holos such as validation and driving Kustomize.
2024-11-11 17:16:53 -07:00
Jeff McCune
9ac7f185f9 docs: fix broken validators link in diagram 2024-11-11 17:11:35 -07:00
Jeff McCune
7de72d3dab docs: add component parameters example to hello holos (#328)
The important note was weird because we didn't show an example of how to
use component parameters.  This patch replaces the note with an example.
2024-11-11 16:56:16 -07:00
Jeff McCune
2e3c998454 docs: add directory tree to hello holos doc (#324)
Feedback from Zack, give a tree so people skimming know where to figure
out the lay of the land.
2024-11-11 16:19:48 -07:00
Jeff McCune
580afffa7f 0.98.1 - holos init platform 2024-11-11 14:44:00 -07:00
Jeff McCune
67535e1e1d doc: remove init platform from setup guide
We have it in the hello guide, setup should just be install only and not
how to use the tool yet.
2024-11-11 14:41:58 -07:00
Nate McCurdy
767ea69d2e docs: Add a tree view to Hello Holos
A tree view of the `holos-tutorial/` directory should give readers a
quick, high-level understanding of the folder structure of a typical
Holos platform project.
2024-11-11 14:04:40 -07:00
Jeff McCune
21e1a116e4 cli: hide help flags and command (#325)
They're unnecessary.
2024-11-11 14:02:25 -07:00
Jeff McCune
65fe7779be cli: rename generate to init (#325)
This patch changes the `holos generate` command to `holos init` to match
other tools like `go mod init`.
2024-11-11 14:02:25 -07:00
Jeff McCune
0e7abf0173 docs: consolidate diagrams to @site/src/diagrams/
So we don't have two different copies in two different places.
2024-11-11 13:40:46 -07:00
Jeff McCune
cca022ac99 docs: move architecture diagrams (#323) 2024-11-11 12:03:21 -07:00
Jeff McCune
43e939d06a doc: refactor breaking it down table in hello holos
So it displays nicely on mobile.
2024-11-09 14:52:11 -08:00
Jeff McCune
8096268826 docs: fix diagram urls again 2024-11-09 14:40:08 -08:00
Jeff McCune
631b23091d docs: fix rendering overview diagram on blog 2024-11-09 14:33:54 -08:00
Jeff McCune
09c6476282 docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-09 14:30:44 -08:00
Jeff McCune
a768d16c5f docs: set current version to v1alpha5
Previously the current version would always be unreleased at /docs/next
and we'd have to copy the doc/md/ folder into the
doc/website/versioned_docs/version-v1alpha5/ every time we made a
change.

We're going to be working on v1alpha5 for awhile so it makes sense to
just have the current version published at /docs/v1alpha5/ and we can
start /docs/v1alpha6/ whenever we're ready.

This also has the nice effect of giving us permalinks if we change the
structure again.  /docs/v1alpha5/ will remain over time.
2024-11-09 14:29:27 -08:00
Jeff McCune
3834a7ef85 docs: add missing link to kustomize tutorial 2024-11-08 22:25:34 -08:00
Jeff McCune
606a1aae73 docs: add nav bar title back 2024-11-08 19:29:05 -08:00
Jeff McCune
340d07ee7a docs: fix announcing holos blog (#321) 2024-11-08 19:24:46 -08:00
Jeff McCune
12d2cec4d5 docs: fix rendering overview diagram links (#321) 2024-11-08 19:12:23 -08:00
Jeff McCune
e93feb49b7 docs: add version drop down to nav bar (#321) 2024-11-08 17:05:37 -08:00
Jeff McCune
dcf8602a0b docs: release v1alpha5 (#321) 2024-11-08 17:00:34 -08:00
Jeff McCune
e07c4d11c8 docs: revise helm values and kustomize tutorials (#316)
These are now where I'd like them to be.
2024-11-08 15:12:22 -08:00
Jeff McCune
b7e1c14192 docs: kustomize tutorial (#316)
Add httpbin using kustomize and patch the result, all from CUE.  The is
the second half of the v1alpha4 helm guide split into a dedicated
tutorial.
2024-11-08 14:08:48 -08:00
Jeff McCune
29f44cdac9 docs: helm values (#316)
Add a helm values tutorial which is a cut down version of the v1alpha4
helm guide.  The httpbin kustomize will immediately follow building on
the prometheus and blackbox charts.
2024-11-08 12:03:14 -08:00
647 changed files with 25220 additions and 207091 deletions

View File

@@ -6,10 +6,12 @@
],
"words": [
"acmesolver",
"acraccesstoken",
"acraccesstokens",
"admissionregistration",
"alertmanager",
"alertmanagers",
"anchore",
"anthos",
"apiextensions",
"apimachinery",
@@ -33,6 +35,7 @@
"balancereader",
"blackbox",
"buildplan",
"buildplans",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
@@ -55,6 +58,7 @@
"Cmds",
"CNCF",
"CODEOWNERS",
"componentconfig",
"configdir",
"configmap",
"configmapargs",
@@ -74,9 +78,11 @@
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devel",
"devicecode",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtoken",
"ecrauthorizationtokens",
"edns",
"endpointslices",
@@ -95,6 +101,7 @@
"fullname",
"gatewayclass",
"gatewayclasses",
"gcraccesstoken",
"gcraccesstokens",
"gendoc",
"generationbehavior",
@@ -103,6 +110,7 @@
"genproto",
"ggnpl",
"ghaction",
"githubaccesstoken",
"githubaccesstokens",
"gitops",
"GOBIN",
@@ -133,6 +141,7 @@
"httproute",
"httproutes",
"iampolicygenerator",
"incpatch",
"Infima",
"intstr",
"isatty",
@@ -149,6 +158,7 @@
"kubelet",
"kubelogin",
"kubernetesobjects",
"kubeversion",
"Kustomization",
"Kustomizations",
"kustomize",
@@ -165,6 +175,7 @@
"loadbalancer",
"loadrestrictions",
"logfmt",
"lxnl",
"mattn",
"mccutchen",
"metav",
@@ -250,6 +261,7 @@
"rolebinding",
"rootfs",
"ropc",
"sboms",
"seccomp",
"secretargs",
"SECRETKEY",
@@ -292,6 +304,7 @@
"tokencache",
"Tokener",
"tolerations",
"TOPLEVEL",
"Traceid",
"traefik",
"transactionhistory",
@@ -299,6 +312,7 @@
"typemeta",
"udev",
"uibutton",
"Unmarshal",
"unstage",
"untar",
"upbound",
@@ -310,6 +324,7 @@
"userservice",
"validatingwebhookconfiguration",
"validatingwebhookconfigurations",
"vaultdynamicsecret",
"vaultdynamicsecrets",
"virtualservice",
"virtualservices",

View File

@@ -35,6 +35,9 @@ jobs:
with:
go-version: stable
- name: Setup Syft
uses: anchore/sbom-action/download-syft@1ca97d9028b51809cf6d3c934c3e160716e1b605 # v0.17.5
# Necessary to run these outside of goreleaser, otherwise
# /home/runner/_work/holos/holos/internal/frontend/node_modules/.bin/protoc-gen-connect-query is not in PATH
- name: Install Tools
@@ -54,11 +57,19 @@ jobs:
- name: Git diff
run: git diff
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
version: '~> v2'
args: release --clean
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -6,7 +6,7 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
version: 2
before:
hooks:
@@ -50,3 +50,39 @@ changelog:
exclude:
- "^docs:"
- "^test:"
source:
enabled: true
name_template: '{{ .ProjectName }}_{{ .Version }}_source_code'
sboms:
- id: source
artifacts: source
documents:
- "{{ .ProjectName }}_{{ .Version }}_sbom.spdx.json"
brews:
- name: holos
repository:
owner: holos-run
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
homepage: "https://holos.run"
description: "Holos CLI"
dependencies:
- name: helm
type: optional
- name: kubectl
type: optional
install: |
bin.install "holos"
bash_output = Utils.safe_popen_read(bin/"holos", "completion", "bash")
(bash_completion/"holos").write bash_output
zsh_output = Utils.safe_popen_read(bin/"holos", "completion", "zsh")
(zsh_completion/"_holos").write zsh_output
fish_output = Utils.safe_popen_read(bin/"holos", "completion", "fish")
(fish_completion/"holos.fish").write fish_output
test: |
system "#{bin}/holos --version"

View File

@@ -46,6 +46,11 @@ type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
@@ -81,6 +86,10 @@ type Helm struct {
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.

View File

@@ -23,23 +23,21 @@ package core
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
// Source reflects the origin of the BuildPlan.
Source BuildPlanSource `json:"source,omitempty"`
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
// BuildPlanSource reflects the origin of a [BuildPlan]. Useful to save a build
@@ -47,7 +45,7 @@ type BuildPlanSpec struct {
// component collection.
type BuildPlanSource struct {
// Component reflects the component that produced the build plan.
Component Component `json:"component,omitempty"`
Component Component `json:"component,omitempty" yaml:"component,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
@@ -71,10 +69,10 @@ type BuildPlanSource struct {
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
@@ -90,19 +88,19 @@ type Artifact struct {
// 3. [File] - Generates data by reading a file from the component directory.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
// Resource represents one kubernetes api object.
@@ -119,20 +117,24 @@ type Resources map[Kind]map[InternalLabel]Resource
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
Source FilePath `json:"source" yaml:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
@@ -141,19 +143,19 @@ type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version"`
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
@@ -167,17 +169,17 @@ type Repository struct {
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
@@ -187,15 +189,15 @@ type Transformer struct {
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
@@ -225,7 +227,13 @@ type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
@@ -238,20 +246,20 @@ type Metadata struct {
// cue export --out yaml ./platform
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components"`
Components []Component `json:"components" yaml:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
@@ -259,17 +267,23 @@ type PlatformSpec struct {
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path"`
Path string `json:"path" yaml:"path"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty"`
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,56 +0,0 @@
package v1alpha1
import (
"errors"
"fmt"
"strings"
)
// BuildPlan is the primary interface between CUE and the Holos cli.
type BuildPlan struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
Spec BuildPlanSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
}
type BuildPlanSpec struct {
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
Components BuildPlanComponents `json:"components,omitempty" yaml:"components,omitempty"`
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
type BuildPlanComponents struct {
HelmChartList []HelmChart `json:"helmChartList,omitempty" yaml:"helmChartList,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty" yaml:"kubernetesObjectsList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty" yaml:"kustomizeBuildList,omitempty"`
Resources map[string]KubernetesObjects `json:"resources,omitempty" yaml:"resources,omitempty"`
}
func (bp *BuildPlan) Validate() error {
errs := make([]string, 0, 2)
if bp.Kind != BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", BuildPlanKind, bp.Kind))
}
if bp.APIVersion != APIVersion {
errs = append(errs, fmt.Sprintf("apiVersion invalid: want: %s have: %s", APIVersion, bp.APIVersion))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (bp *BuildPlan) ResultCapacity() (count int) {
if bp == nil {
return 0
}
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}

View File

@@ -1,30 +0,0 @@
package v1alpha1
// HolosComponent defines the fields common to all holos component kinds including the Render Result.
type HolosComponent struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
// APIObjectMap holds the marshalled representation of api objects. Think of
// these as resources overlaid at the back of the render pipeline.
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty" yaml:"apiObjectMap,omitempty"`
// Kustomization holds the marshalled representation of the flux kustomization
// which reconciles resources in git with the api server.
Kustomization `json:",inline" yaml:",inline"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:",inline" yaml:",inline"`
// Skip causes holos to take no action regarding the component.
Skip bool
}
func (hc *HolosComponent) NewResult() *Result {
return &Result{HolosComponent: *hc}
}
func (hc *HolosComponent) GetAPIVersion() string {
return hc.APIVersion
}
func (hc *HolosComponent) GetKind() string {
return hc.Kind
}

View File

@@ -1,11 +0,0 @@
package v1alpha1
const (
APIVersion = "holos.run/v1alpha1"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,2 +0,0 @@
// Package v1alpha1 defines the api boundary between CUE and Holos.
package v1alpha1

View File

@@ -1,13 +0,0 @@
package v1alpha1
import object "github.com/holos-run/holos/service/gen/holos/object/v1alpha1"
// Form represents a collection of Formly json powered form.
type Form struct {
TypeMeta `json:",inline" yaml:",inline"`
Spec FormSpec `json:"spec" yaml:"spec"`
}
type FormSpec struct {
Form object.Form `json:"form" yaml:"form"`
}

View File

@@ -1,184 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// A HelmChart represents a helm command to provide chart values in order to render kubernetes api objects.
type HelmChart struct {
HolosComponent `json:",inline" yaml:",inline"`
// Namespace is the namespace to install into. TODO: Use metadata.namespace instead.
Namespace string `json:"namespace"`
Chart Chart `json:"chart"`
ValuesContent string `json:"valuesContent"`
EnableHooks bool `json:"enableHooks"`
}
type Chart struct {
Name string `json:"name"`
Version string `json:"version"`
Release string `json:"release"`
Repository Repository `json:"repository,omitempty"`
}
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
func (hc *HelmChart) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: hc.HolosComponent}
if err := hc.helm(ctx, &result, path); err != nil {
return nil, err
}
result.addObjectMap(ctx, hc.APIObjectMap)
if err := result.kustomize(ctx); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not kustomize: %w", err))
}
return &result, nil
}
// runHelm provides the values produced by CUE to helm template and returns
// the rendered kubernetes api objects in the result.
func (hc *HelmChart) helm(ctx context.Context, r *Result, path holos.InstancePath) error {
log := logger.FromContext(ctx).With("chart", hc.Chart.Name)
if hc.Chart.Name == "" {
log.WarnContext(ctx, "skipping helm: no chart name specified, use a different component type")
return nil
}
cachedChartPath := filepath.Join(string(path), ChartDir, filepath.Base(hc.Chart.Name))
if isNotExist(cachedChartPath) {
// Add repositories
repo := hc.Chart.Repository
if repo.URL != "" {
out, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo add: %w", err))
}
// Update repository
out, err = util.RunCmd(ctx, "helm", "repo", "update", repo.Name)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo update: %w", err))
}
} else {
log.DebugContext(ctx, "no chart repository url proceeding assuming oci chart")
}
// Cache the chart
if err := cacheChart(ctx, path, ChartDir, hc.Chart); err != nil {
return fmt.Errorf("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos")
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, tempDir)
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, []byte(hc.ValuesContent), 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "helm: wrote values", "path", valuesPath, "bytes", len(hc.ValuesContent))
// Run charts
chart := hc.Chart
args := []string{"template"}
if !hc.EnableHooks {
args = append(args, "--no-hooks")
}
namespace := hc.Namespace
args = append(args, "--include-crds", "--values", valuesPath, "--namespace", namespace, "--kubeconfig", "/dev/null", "--version", chart.Version, chart.Release, cachedChartPath)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Wrap(fmt.Errorf("could not run helm template: %w", err))
}
r.accumulatedOutput = helmOut.Stdout.String()
return nil
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// It is assumed that the only method responsible for writing to chartDir is
// cacheChart itself.
//
// This relies on the atomicity of moving temporary directories into place on
// the same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is an
// expected scenario when this function is called concurrently.
func cacheChart(ctx context.Context, path holos.InstancePath, chartDir string, chart Chart) error {
log := logger.FromContext(ctx)
cacheTemp, err := os.MkdirTemp(string(path), chartDir)
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, cacheTemp)
chartName := chart.Name
if chart.Repository.Name != "" {
chartName = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, chartName)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
cachePath := filepath.Join(string(path), chartDir)
if err := os.MkdirAll(cachePath, 0777); err != nil {
return errors.Wrap(fmt.Errorf("could not mkdir: %w", err))
}
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
for _, item := range items {
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cachePath, item.Name())
log.DebugContext(ctx, "rename", "src", src, "dst", dst)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
}
}
log.InfoContext(ctx, "cached", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
return nil
}
func isNotExist(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}

View File

@@ -1,21 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents CUE output which directly provides Kubernetes api objects to holos.
type KubernetesObjects struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces kubernetes api objects from the APIObjectMap
func (o *KubernetesObjects) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: o.HolosComponent}
result.addObjectMap(ctx, o.APIObjectMap)
return &result, nil
}

View File

@@ -1,7 +0,0 @@
package v1alpha1
// Kustomization holds the rendered flux kustomization api object content for git ops.
type Kustomization struct {
// KsContent is the yaml representation of the flux kustomization for gitops.
KsContent string `json:"ksContent,omitempty" yaml:"ksContent,omitempty"`
}

View File

@@ -1,47 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
const KustomizeBuildKind = "KustomizeBuild"
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process raw yaml file resources in a holos component directory.
// 2. Post process a HelmChart to inject istio, add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty" yaml:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty" yaml:"resourcesFile,omitempty"`
}
// KustomizeBuild renders plain yaml files in the holos component directory using kubectl kustomize build.
type KustomizeBuild struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces a Result by executing kubectl kustomize on the holos
// component path. Useful for processing raw yaml files.
func (kb *KustomizeBuild) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
log := logger.FromContext(ctx)
result := Result{HolosComponent: kb.HolosComponent}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", string(path))
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return nil, errors.Wrap(err)
}
// Replace the accumulated output
result.accumulatedOutput = kOut.Stdout.String()
// Add CUE based api objects.
result.addObjectMap(ctx, kb.APIObjectMap)
return &result, nil
}

View File

@@ -1,14 +0,0 @@
package v1alpha1
// Label is an arbitrary unique identifier. Defined as a type for clarity and type checking.
type Label string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
type Kind string
// APIObjectMap is the shape of marshalled api objects returned from cue to the
// holos cli. A map is used to improve the clarity of error messages from cue.
type APIObjectMap map[Kind]map[Label]string
// FileContentMap is a map of file names to file contents.
type FileContentMap map[string]string

View File

@@ -1,15 +0,0 @@
package v1alpha1
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are by holos objects distinct
// from kubernetes api objects.
type ObjectMeta struct {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
Name string `json:"name,omitempty" yaml:"name,omitempty"`
// Namespace confines a holos component to a single namespace via kustomize if set.
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Labels are not used but are copied from api machinery ObjectMeta for completeness.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations are not used but are copied from api machinery ObjectMeta for completeness.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,32 +0,0 @@
package v1alpha1
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
TypeMeta `json:",inline" yaml:",inline"`
Metadata ObjectMeta `json:"metadata" yaml:"metadata"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform build plan specification.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// holos.platform.v1alpha1.PlatformService.GetPlatform method and provides to
// CUE using a tag.
Model structpb.Struct `json:"model" yaml:"model"`
Components []PlatformSpecComponent `json:"components" yaml:"components"`
}
// PlatformSpecComponent represents a component to build or render with flags to
// pass, for example the cluster name.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path" yaml:"path"`
// Cluster is the cluster name to use when building the component.
Cluster string `json:"cluster" yaml:"cluster"`
}

View File

@@ -1,22 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
type Renderer interface {
GetKind() string
Render(ctx context.Context, path holos.InstancePath) (*Result, error)
}
// Render produces a Result representing the kubernetes api objects to
// configure. Each of the various holos component types, e.g. Helm, Kustomize,
// et al, should implement the Renderer interface. This process is best
// conceptualized as a data pipeline, for example a component may render a
// result by first calling helm template, then passing the result through
// kustomize, then mixing in overlay api objects.
func Render(ctx context.Context, r Renderer, path holos.InstancePath) (*Result, error) {
return r.Render(ctx, path)
}

View File

@@ -1,165 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"slices"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// Result is the build result for display or writing. Holos components Render the Result as a data pipeline.
type Result struct {
HolosComponent
// accumulatedOutput accumulates rendered api objects.
accumulatedOutput string
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
// Continue returns true if Skip is true indicating the result is to be skipped over.
func (r *Result) Continue() bool {
if r == nil {
return false
}
return r.Skip
}
func (r *Result) Name() string {
return r.Metadata.Name
}
func (r *Result) Filename(writeTo string, cluster string) string {
name := r.Metadata.Name
return filepath.Join(writeTo, "clusters", cluster, "components", name, name+".gen.yaml")
}
func (r *Result) KustomizationFilename(writeTo string, cluster string) string {
return filepath.Join(writeTo, "clusters", cluster, "holos", "components", r.Metadata.Name+"-kustomization.gen.yaml")
}
// AccumulatedOutput returns the accumulated rendered output.
func (r *Result) AccumulatedOutput() string {
return r.accumulatedOutput
}
// addObjectMap renders the provided APIObjectMap into the accumulated output.
func (r *Result) addObjectMap(ctx context.Context, objectMap APIObjectMap) {
log := logger.FromContext(ctx)
b := []byte(r.AccumulatedOutput())
kinds := make([]Kind, 0, len(objectMap))
// Sort the keys
for kind := range objectMap {
kinds = append(kinds, kind)
}
slices.Sort(kinds)
for _, kind := range kinds {
v := objectMap[kind]
// Sort the keys
names := make([]Label, 0, len(v))
for name := range v {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
yamlString := v[name]
log.Debug(fmt.Sprintf("%s/%s", kind, name), "kind", kind, "name", name)
b = util.EnsureNewline(b)
header := fmt.Sprintf("---\n# Source: CUE apiObjects.%s.%s\n", kind, name)
b = append(b, []byte(header+yamlString)...)
b = util.EnsureNewline(b)
}
}
r.accumulatedOutput = string(b)
}
// kustomize replaces the accumulated output with the output of kustomize build
func (r *Result) kustomize(ctx context.Context) error {
log := logger.FromContext(ctx)
if r.ResourcesFile == "" {
log.DebugContext(ctx, "skipping kustomize: no resourcesFile")
return nil
}
if len(r.KustomizeFiles) < 1 {
log.DebugContext(ctx, "skipping kustomize: no kustomizeFiles")
return nil
}
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
// Write the main api object resources file for kustomize.
target := filepath.Join(tempDir, r.ResourcesFile)
b := []byte(r.AccumulatedOutput())
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write resources: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
// Write the kustomization tree, kustomization.yaml must be in this map for kustomize to work.
for file, content := range r.KustomizeFiles {
target := filepath.Join(tempDir, file)
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return errors.Wrap(err)
}
b := []byte(content)
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return errors.Wrap(err)
}
// Replace the accumulated output
r.accumulatedOutput = kOut.Stdout.String()
return nil
}
func (r *Result) WriteDeployFiles(ctx context.Context, path string) error {
log := logger.FromContext(ctx)
if len(r.DeployFiles) == 0 {
return nil
}
for k, content := range r.DeployFiles {
path := filepath.Join(path, k)
if err := r.Save(ctx, path, content); err != nil {
return errors.Wrap(err)
}
log.InfoContext(ctx, "wrote deploy file", "path", path, "bytes", len(content))
}
return nil
}
// Save writes the content to the filesystem for git ops.
func (r *Result) Save(ctx context.Context, path string, content string) error {
log := logger.FromContext(ctx)
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, os.FileMode(0775)); err != nil {
log.WarnContext(ctx, "could not mkdir", "path", dir, "err", err)
return errors.Wrap(err)
}
// Write the file content
if err := os.WriteFile(path, []byte(content), os.FileMode(0644)); err != nil {
log.WarnContext(ctx, "could not write", "path", path, "err", err)
return errors.Wrap(err)
}
log.DebugContext(ctx, "out: wrote "+path, "action", "write", "path", path, "status", "ok")
return nil
}

View File

@@ -1,20 +0,0 @@
package v1alpha1
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
func (tm *TypeMeta) GetKind() string {
return tm.Kind
}
func (tm *TypeMeta) GetAPIVersion() string {
return tm.APIVersion
}
// Discriminator is an interface to discriminate the kind api object.
type Discriminator interface {
GetKind() string
GetAPIVersion() string
}

View File

@@ -17,9 +17,6 @@ func TestMain(m *testing.M) {
}))
}
func TestGuides_v1alpha4(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha4", "guides")))
}
func TestGuides_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "guides")))
}
@@ -28,6 +25,10 @@ func TestSchemas_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "schemas")))
}
func TestIssues_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "issues")))
}
func TestCLI(t *testing.T) {
testscript.Run(t, params("cli"))
}

View File

@@ -0,0 +1,2 @@
# https://github.com/holos-run/holos/issues/334
exec holos

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,38 @@
# https://github.com/holos-run/holos/issues/332
env HOME=$WORK
# Mock with a stub helm command
env PATH=$WORK/bin:$PATH
chmod 755 bin/helm
# Initialize the platform
exec holos init platform v1alpha5 --force
# when helm update returns an error
! exec holos render platform
# holos should log the helm error to stderr
stderr 'Error: chart "podinfo" matching 0.0.0 not found in podinfo index'
-- bin/helm --
#! /bin/bash
echo 'Error: chart "podinfo" matching 0.0.0 not found in podinfo index' >&2
exit 2
-- platform/podinfo.cue --
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
-- components/podinfo/podinfo.cue --
package holos
// Produce a helm chart build plan.
holos: HelmChart.BuildPlan
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "0.0.0"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}

View File

@@ -0,0 +1,461 @@
# https://github.com/holos-run/holos/issues/331
# ensure holos show components --labels selects correctly.
# ensure BuildPlan includes labels and annotations from the platform component.
# ensure holos render platform injects the holos_component_labels and
# holos_component_annotations tags.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cmp stdout want/platform.yaml
# all buildplans are selected by default
exec holos show buildplans
cmp stdout want/all-buildplans.yaml
# one = works in the selector
exec holos show buildplans --selector app.holos.run/name=empty1-label
cmp stdout want/buildplans.1.yaml
# double == works in the selector
exec holos show buildplans --selector app.holos.run/name==empty2-label
cmp stdout want/buildplans.2.yaml
# not equal != negates the selection
exec holos show buildplans --selector app.holos.run/name!=empty3-label
cmp stdout want/buildplans.3.yaml
exec holos show buildplans --selector app.holos.run/name!=something-else
cmp stdout want/buildplans.4.yaml
-- platform/empty.cue --
package holos
Platform: Components: {
empty1: _
empty2: _
empty3: _
empty4: _
}
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": "\(name)-label"
annotations: "app.holos.run/description": "\(name)-annotation empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/platform.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components:
- annotations:
app.holos.run/description: empty1-annotation empty test case
labels:
app.holos.run/name: empty1-label
name: empty1
path: components/empty
- annotations:
app.holos.run/description: empty2-annotation empty test case
labels:
app.holos.run/name: empty2-label
name: empty2
path: components/empty
- annotations:
app.holos.run/description: empty3-annotation empty test case
labels:
app.holos.run/name: empty3-label
name: empty3
path: components/empty
- annotations:
app.holos.run/description: empty4-annotation empty test case
labels:
app.holos.run/name: empty4-label
name: empty4
path: components/empty
-- want/empty.yaml --
-- want/all-buildplans.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.1.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.2.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.3.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.4.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,144 @@
# https://github.com/holos-run/holos/issues/330
exec holos init platform v1alpha5 --force
exec helm template ./components/capabilities/vendor/0.1.0/capabilities
cmp stdout want/helm-template.yaml
exec holos render platform ./platform
# When no capabilities are specified
cmp deploy/components/capabilities/capabilities.gen.yaml want/when-no-capabilities-specified.yaml
# With APIVersions specified
cmp deploy/components/specified/specified.gen.yaml want/with-capabilities-specified.yaml
# With KubeVersion specified
cmp deploy/components/kubeversion1/kubeversion1.gen.yaml want/with-kubeversion-specified.yaml
# With both APIVersions and KubeVersion specified
cmp deploy/components/kubeversion2/kubeversion2.gen.yaml want/with-both-specified.yaml
-- want/with-both-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-kubeversion-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/when-no-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- platform/capabilities.cue --
package holos
import "encoding/json"
Platform: Components: capabilities: {
name: "capabilities"
path: "components/capabilities"
}
Platform: Components: specified: {
name: "specified"
path: "components/capabilities"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
Platform: Components: kubeversion1: {
name: "kubeversion1"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
}
Platform: Components: kubeversion2: {
name: "kubeversion2"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
-- components/capabilities/capabilities.cue --
package holos
import "encoding/json"
holos: Component.BuildPlan
Component: #Helm & {
Name: string @tag(holos_component_name, type=string)
Chart: name: "capabilities"
Chart: version: "0.1.0"
_APIVersions: string | *"[]" @tag(apiVersions, type=string)
APIVersions: json.Unmarshal(_APIVersions)
KubeVersion: string | *"v1.31.0" @tag(kubeVersion, type=string)
}
-- components/capabilities/vendor/0.1.0/capabilities/Chart.yaml --
apiVersion: v2
name: capabilities
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: "1.16.0"
-- components/capabilities/vendor/0.1.0/capabilities/templates/service.yaml --
apiVersion: v1
kind: Service
metadata:
{{- if .Capabilities.APIVersions.Has "foo/v1" }}
name: has-foo-v1
{{- else }}
name: has-foo-v1beta1
{{- end }}
annotations:
kubeVersion: {{ .Capabilities.KubeVersion }}
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
-- want/helm-template.yaml --
---
# Source: capabilities/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: has-foo-v1beta1
annotations:
kubeVersion: v1.31.0
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http

View File

@@ -4,10 +4,10 @@
cd $WORK
# Generate the directory structure we're going to work in.
exec holos generate platform v1alpha5 --force
exec holos init platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform ./platform
exec holos render platform
stderr -count=1 '^rendered platform'
# When author.#Kubernetes is empty
@@ -45,8 +45,3 @@ spec:
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
source:
component:
name: no-name
path: no-path
parameters: {}

View File

@@ -43,6 +43,11 @@ type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
@@ -83,6 +88,10 @@ type Helm struct {
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.

View File

@@ -56,10 +56,10 @@ Output fields are write\-once. It is an error for multiple Generators or Transfo
```go
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
```
@@ -75,15 +75,13 @@ Holos uses CUE to construct a BuildPlan. A future enhancement will support user
```go
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
// Source reflects the origin of the BuildPlan.
Source BuildPlanSource `json:"source,omitempty"`
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
```
@@ -95,7 +93,7 @@ BuildPlanSource reflects the origin of a [BuildPlan](<#BuildPlan>). Useful to sa
```go
type BuildPlanSource struct {
// Component reflects the component that produced the build plan.
Component Component `json:"component,omitempty"`
Component Component `json:"component,omitempty" yaml:"component,omitempty"`
}
```
@@ -107,9 +105,9 @@ BuildPlanSpec represents the specification of the [BuildPlan](<#BuildPlan>).
```go
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
```
@@ -121,13 +119,13 @@ Chart represents a [Helm](<#Helm>) Chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version"`
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
```
@@ -140,19 +138,25 @@ Component represents the complete context necessary to produce a [BuildPlan](<#B
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path"`
Path string `json:"path" yaml:"path"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty"`
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
@@ -164,7 +168,7 @@ File represents a simple single file copy [Generator](<#Generator>). Useful with
```go
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
Source FilePath `json:"source" yaml:"source"`
}
```
@@ -209,19 +213,19 @@ Each Generator in an [Artifact](<#Artifact>) must have a distinct Output value f
```go
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
```
@@ -233,14 +237,18 @@ Helm represents a [Chart](<#Chart>) manifest [Generator](<#Generator>).
```go
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
```
@@ -260,7 +268,7 @@ Join represents a [Transformer](<#Transformer>) using [bytes.Join](<https://pkg.
```go
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
```
@@ -290,9 +298,9 @@ Kustomize represents a kustomization [Transformer](<#Transformer>).
```go
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
```
@@ -304,7 +312,13 @@ Metadata represents data about the resource such as the Name.
```go
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
@@ -322,14 +336,14 @@ cue export --out yaml ./platform
```go
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
```
@@ -341,7 +355,7 @@ PlatformSpec represents the platform specification.
```go
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components"`
Components []Component `json:"components" yaml:"components"`
}
```
@@ -352,8 +366,8 @@ Repository represents a [Helm](<#Helm>) [Chart](<#Chart>) repository.
```go
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
}
```
@@ -386,17 +400,17 @@ Transformer combines multiple inputs from prior [Generator](<#Generator>) or [Tr
```go
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
```

View File

@@ -0,0 +1,16 @@
Integrate the `podinfo` component into the platform.
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```

View File

@@ -0,0 +1,34 @@
Create a directory for the example `podinfo` component we'll use to render
platform manifests.
```bash
mkdir -p components/podinfo
```
Create the CUE configuration for the example `podinfo` component.
```bash
cat <<EOF >components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
```bash
EOF
```

View File

@@ -1,32 +0,0 @@
```mermaid
---
title: Rendering Overview
---
graph LR
Platform[<a href="/docs/api/author/v1alpha4/#Platform">Platform</a>]
Component[<a href="/docs/api/author/v1alpha4/#ComponentConfig">Components</a>]
Helm[<a href="/docs/api/author/v1alpha4/#Helm">Helm</a>]
Kustomize[<a href="/docs/api/author/v1alpha4/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/api/author/v1alpha4/#Kubernetes">Kubernetes</a>]
BuildPlan[<a href="/docs/api/core/v1alpha4/#buildplan">BuildPlan</a>]
ResourcesArtifact[<a href="/docs/api/core/v1alpha4/#artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/api/core/v1alpha4/#artifact">GitOps<br/>Artifact</a>]
Generators[<a href="/docs/api/core/v1alpha4/#generators">Generators</a>]
Transformers[<a href="/docs/api/core/v1alpha4/#transformer">Transformers</a>]
Validators[Validators<br/>TBD]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Validators --> Files
```

View File

@@ -1,14 +1,17 @@
---
description: Get Support for Holos
slug: /support
sidebar_position: 900
slug: support
sidebar_position: 2000
---
# Support
## Community Support
You can ask questions in our community forums in [GitHub Discussions](https://github.com/holos-run/holos/discussions), [Discord](https://discord.gg/JgDVbNpye7), or [Google Groups](https://groups.google.com/g/holos-discuss).
You can ask questions in our community forums in [GitHub
Discussions](https://github.com/holos-run/holos/discussions),
[Discord](https://discord.gg/JgDVbNpye7), or [Google
Groups](https://groups.google.com/g/holos-discuss).
## Commercial Support and Services

View File

@@ -0,0 +1,18 @@
---
description: Architecture diagrams.
slug: architecture
sidebar_position: 100
---
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
import RenderComponentDiagram from '@site/src/diagrams/render-component-sequence.mdx';
# Architecture
## Platform Rendering Sequence
<RenderPlatformDiagram />
## Component Rendering Sequence
<RenderComponentDiagram />

View File

@@ -0,0 +1,224 @@
---
slug: argocd-application
title: ArgoCD Application
description: Configuring an Application for each Component.
sidebar_position: 110
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
import CommonComponentIntegrate from '../../common/example-component-integrate.mdx';
# ArgoCD Application
## Overview
This topic covers how to mix in an ArgoCD Application to all components. We'll
use the `Artifacts` field of [ComponentConfig] defined by the author schema.
## The Code
### Generating the structure
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-argocd-application && cd holos-argocd-application
```
```shell
holos init platform v1alpha5
```
### Creating an example Component
<CommonComponent />
<CommonComponentIntegrate />
## Adding ArgoCD Application
Configure Holos to render an [Application] by defining an [Artifact] for it in
every BuildPlan holos produces. We're unifying our custom configuration with
the existing `#ComponentConfig` defined in `schema.cue`.
```bash
cat <<EOF >argocd-application.cue
```
```cue showLineNumbers
package holos
import (
"path"
app "argoproj.io/application/v1alpha1"
)
#ComponentConfig: {
Name: _
OutputBaseDir: _
let ArtifactPath = path.Join([OutputBaseDir, "gitops", "\(Name).application.gen.yaml"], path.Unix)
let ResourcesPath = path.Join(["deploy", OutputBaseDir, "components", Name], path.Unix)
Artifacts: "\(Name)-application": {
artifact: ArtifactPath
generators: [{
kind: "Resources"
output: artifact
resources: Application: (Name): app.#Application & {
metadata: name: Name
metadata: namespace: "argocd"
spec: {
destination: server: "https://kubernetes.default.svc"
project: "default"
source: {
path: ResourcesPath
repoURL: "https://example.com/example.git"
targetRevision: "main"
}
}
}
}]
}
}
```
```bash
EOF
```
## Inspecting the BuildPlan
Our customized `#ComponentConfig` results in the following `BuildPlan`.
:::note
The second artifact around line 40 contains the configured `Application`
resource.
:::
<Tabs groupId="55075C71-02E8-4222-88C0-2D52C82D18FC">
<TabItem value="command" label="Command">
```bash
holos cue export --expression holos --out=yaml ./components/podinfo
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: podinfo
spec:
artifacts:
- artifact: components/podinfo/podinfo.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: podinfo
version: 6.6.2
release: podinfo
repository:
name: podinfo
url: https://stefanprodan.github.io/podinfo
values: {}
enableHooks: false
- kind: Resources
output: resources.gen.yaml
resources: {}
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: components/podinfo/podinfo.gen.yaml
kustomize:
kustomization:
labels:
- includeSelectors: false
pairs: {}
resources:
- helm.gen.yaml
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
- artifact: gitops/podinfo.application.gen.yaml
generators:
- kind: Resources
output: gitops/podinfo.application.gen.yaml
resources:
Application:
podinfo:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/components/podinfo
repoURL: https://example.com/example.git
targetRevision: main
source:
component:
name: podinfo
path: no-path
parameters: {}
```
</TabItem>
</Tabs>
## Rendering manifests
<Tabs groupId="E150C802-7162-4FBF-82A7-77D9ADAEE847">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
</Tabs>
## Reviewing the Application
The Artifact we added to `#ComponentConfig` will produce an ArgoCD Application
resource for every component in the platform. The output in this example is
located at:
```txt
deploy/gitops/podinfo.application.gen.yaml
```
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/components/podinfo
repoURL: https://example.com/example.git
targetRevision: main
```
[podinfo]: https://github.com/stefanprodan/podinfo
[CUE Module]: https://cuelang.org/docs/reference/modules/
[CUE Tags]: https://cuelang.org/docs/howto/inject-value-into-evaluation-using-tag-attribute/
[Application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[Component Parameters]: ../component-parameters.mdx
[Platform]: ../../api/author.md#Platform
[ComponentConfig]: ../../api/author.md#ComponentConfig
[Artifact]: ../../api/core.md#Artifact

View File

@@ -0,0 +1,19 @@
---
slug: .
title: GitOps
description: Managing resources with GitOps.
sidebar_position: 120
---
import DocCardList from '@theme/DocCardList';
# GitOps
This section has self contained articles covering how to manage resources using
GitOps tooling like [ArgoCD] and [Flux].
---
<DocCardList />
[ArgoCD]: https://argo-cd.readthedocs.io/en/stable/
[Flux]: https://fluxcd.io/

View File

@@ -1,7 +1,7 @@
---
description: Build a local cluster for use with Holos.
slug: local-cluster
sidebar_position: 100
sidebar_position: 50
---
import Tabs from '@theme/Tabs';
@@ -16,8 +16,7 @@ have a standard Kubernetes API server with proper DNS and TLS certificates.
You'll be able to easily reset the cluster to a known good state to iterate on
your own Platform.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
The [Glossary] page defines capitalized terms such as Platform and Component.
## Reset the Cluster
@@ -105,7 +104,7 @@ You're back to the same state as the first time you completed this guide.
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
1. [holos](../tutorial/setup.mdx) - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a k8s api server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
@@ -274,4 +273,7 @@ k3d cluster delete workload
## Next Steps
Now that you have a real cluster, apply and explore the manifests Holos renders
in the [Quickstart](/docs/quickstart) guide.
in the [Tutorial].
[Glossary]: ../glossary.mdx
[Tutorial]: ../tutorial.mdx

View File

@@ -0,0 +1,424 @@
---
slug: clusters
title: Clusters
description: Managing clusters - management and workload sets.
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
# Clusters
## Overview
This topic covers one common method to manage multiple clusters with Holos. We'll
define two schemas to hold cluster attributes. First, a single `#Cluster` then
a `#Clusters` collection. We'll use a `Clusters: #Clusters` struct to look up
configuration data using a key. We'll use the cluster name as the lookup key
identifying the cluster.
We'll also organize sets of similar clusters by defining `#ClusterSet` and
`#ClusterSets`. We'll use a `ClusterSets:
#ClusterSets` struct to configure a management cluster and iterate over all
workload clusters.
## The Code
### Initializing the structure
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-multiple-clusters && cd holos-multiple-clusters
```
```shell
holos init platform v1alpha5
```
### Example Component
<CommonComponent />
We'll integrate the component with the platform after we define clusters.
## Defining Clusters
We'll define a `#Cluster` schema and a `#Clusters` collection in this section.
We'll use these schemas to define a `Clusters` structure we use to manage
multiple clusters.
### Assumptions
We'll make the following assumptions, which hold true for many real world
environments.
1. There are two sets of clusters, workload clusters and management clusters.
2. There is one management cluster.
3. There are multiple workload clusters.
4. Each workload cluster is configured similarly, but not identically, to the
others.
### Prototyping the data
Before we define the schema, let's prototype the data structure we want to work
with. We want a structure that makes it easy to iterate over each cluster in
two distinct sets of clusters, management clusters and workload clusters. The
following `ClusterSets` struct accomplishes this goal.
```yaml showLineNumbers
management:
name: management
clusters:
management:
name: management
region: us-central1
set: management
workload:
name: workload
clusters:
e1:
name: e1
region: us-east1
set: workload
w1:
name: w1
region: us-west1
set: workload
```
:::tip
The `ClusterSets` data structure supports iterating over each cluster in each
cluster set.
:::
:::important
You're free to define your own fields and structures like we define `region` in
this topic.
:::
### Defining the schema
Armed with a concrete example of the structure, we can write a schema to define
and validate the data.
In CUE, schema definitions are usually defined at the root so they're accessible
in all subdirectories. The following is one example schema, you're free to
modify it to your situation. Holos is flexible, supporting schemas that match
your unique use case.
```bash
cat <<EOF > clusters.schema.cue
```
```cue showLineNumbers
package holos
import "strings"
// #Cluster represents one cluster
#Cluster: {
// name represents the cluster name.
name: string & =~"[a-z][a-z0-9]+" & strings.MinRunes(2) & strings.MaxRunes(63)
// Constrain the regions. No default, the region must be specified.
region: "us-east1" | "us-central1" | "us-west1"
// Each cluster must be in only one set of clusters. All but one cluster are
// workload clusters, so make it the default.
set: "management" | *"workload"
}
// #Clusters represents a cluster collection structure
#Clusters: {
// name is the lookup key for the collection.
[NAME=string]: #Cluster & {
// name must match the struct field name.
name: NAME
}
}
// #ClusterSet represents a set of clusters.
#ClusterSet: {
// name represents the cluster set name.
name: string & =~"[a-z][a-z0-9]+" & strings.MinRunes(2) & strings.MaxRunes(63)
clusters: #Clusters & {
// Constrain the cluster set to clusters having the same set. Ensures
// clusters are never mis-categorized.
[_]: set: name
}
}
// #ClusterSets represents a cluster set collection.
#ClusterSets: {
// name is the lookup key for the collection.
[NAME=string]: #ClusterSet & {
// name must match the struct field name.
name: NAME
}
}
```
```bash
EOF
```
### Defining the data
With a schema defined, we also define the data close to the root so it's
accessible through the unified configuration tree.
```bash
cat <<EOF > clusters.cue
```
```cue showLineNumbers
package holos
Clusters: #Clusters & {
// Management Cluster
management: region: "us-central1"
management: set: "management"
// Local Cluster
local: region: "us-west1"
// Some example clusters. Add new clusters to the Clusters struct like this.
e1: region: "us-east1"
e2: region: "us-east1"
e3: region: "us-east1"
w1: region: "us-west1"
w2: region: "us-west1"
w3: region: "us-west1"
}
// ClusterSets is dynamically built from the Clusters structure.
ClusterSets: #ClusterSets & {
// Map every cluster into the correct set.
for CLUSTER in Clusters {
(CLUSTER.set): clusters: (CLUSTER.name): CLUSTER
}
}
```
```bash
EOF
```
### Inspecting the data
We'll use the `holos cue` command to inspect the `ClusterSets` data structure we
just defined.
<Tabs groupId="9190BDAD-B4C5-4386-9C94-8E178AA6178A">
<TabItem value="command" label="Command">
```bash
holos cue export --expression ClusterSets --out=yaml ./
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
management:
name: management
clusters:
management:
name: management
region: us-central1
set: management
workload:
name: workload
clusters:
local:
name: local
region: us-west1
set: workload
e1:
name: e1
region: us-east1
set: workload
e2:
name: e2
region: us-east1
set: workload
e3:
name: e3
region: us-east1
set: workload
w1:
name: w1
region: us-west1
set: workload
w2:
name: w2
region: us-west1
set: workload
w3:
name: w3
region: us-west1
set: workload
```
</TabItem>
</Tabs>
This looks like our prototype, we're confident we can iterate over each cluster
in each set.
## Integrating Components
The `ClusterSets` data structure unlocks the capability to iterate over each
cluster in each cluster set. We'll use this capability to integrate the
`podinfo` component with each cluster in the platform.
### Configuring the Output directory
We need to configure `holos` to write output manifests into a cluster specific
output directory. We'll use the [ComponentConfig] `OutputBaseDir` field for
this purpose. We'll pass the value of this field as a component parameter.
```bash
cat <<EOF > componentconfig.cue
```
```cue showLineNumbers
package holos
#ComponentConfig: {
// Inject the output base directory from platform component parameters.
OutputBaseDir: string @tag(outputBaseDir, type=string)
}
```
```bash
EOF
```
### Integrating Podinfo
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
// Manage podinfo on all workload clusters.
for CLUSTER in ClusterSets.workload.clusters {
// We use the cluster name to disambiguate different podinfo build plans.
Platform: Components: "\(CLUSTER.name)-podinfo": {
name: "podinfo"
// Reuse the same component across multiple workload clusters.
path: "components/podinfo"
// Configure a cluster-unique message in the podinfo UI.
parameters: message: "Hello, I am cluster \(CLUSTER.name) in region \(CLUSTER.region)"
// Write to deploy/{outputBaseDir}/components/{name}/{name}.gen.yaml
parameters: outputBaseDir: "clusters/\(CLUSTER.name)"
}
}
```
```bash
EOF
```
## Rendering manifests
### Rendering the Platform
Render the platform to configure `podinfo` on each cluster.
<Tabs groupId="34A2D80B-0E86-4142-B65B-7DF70C47E1D2">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached podinfo 6.6.2
rendered podinfo in 164.278583ms
rendered podinfo in 165.48525ms
rendered podinfo in 165.186208ms
rendered podinfo in 165.831792ms
rendered podinfo in 166.845208ms
rendered podinfo in 167.000208ms
rendered podinfo in 167.012208ms
rendered platform in 167.06525ms
```
</TabItem>
</Tabs>
### Inspecting the Tree
Rendering the platform produces the following rendered manifests.
```bash
tree deploy
```
```txt showLineNumbers
deploy
└── clusters
├── e1
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── e2
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── e3
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── local
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── w1
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── w2
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── w3
└── components
└── podinfo
└── podinfo.gen.yaml
23 directories, 7 files
```
### Inspecting the Variation
Note how each component has slight variation using the component parameters.
```bash
diff -U2 deploy/clusters/{e,w}1/components/podinfo/podinfo.gen.yaml
```
```diff
--- deploy/clusters/e1/components/podinfo/podinfo.gen.yaml 2024-11-17 14:20:17
+++ deploy/clusters/w1/components/podinfo/podinfo.gen.yaml 2024-11-17 14:20:17
@@ -61,5 +61,5 @@
env:
- name: PODINFO_UI_MESSAGE
- value: Hello, I am cluster e1 in region us-east1
+ value: Hello, I am cluster w1 in region us-west1
- name: PODINFO_UI_COLOR
value: '#34577c'
```
## Concluding Remarks
In this topic we covered how to use CUE structures to organize multiple clusters
into various sets.
1. Clusters are defined in one place at the root of the configuration.
2. Clusters may be organized into sets by their purpose.
3. Most organizations have at least two sets, a set of workload clusters and a
set of management clusters.
4. Holos uses CUE, a super set of JSON. New clusters may be added by dropping a
JSON file into the root of the repository.
5. The pattern of defining a `#Cluster` and a `#Clusters` collection is a
general pattern. We'll see the same pattern for environments, projects, owners,
and more.
6. Component parameters are a flexible way to inject user defined configuration
from the platform level into a reusable component.
[ClusterSet]: https://multicluster.sigs.k8s.io/api-types/cluster-set/
[Environments]: ./environments.mdx
[Namespace Sameness - SIG Multicluster Position Statement]: https://github.com/kubernetes/community/blob/master/sig-multicluster/namespace-sameness-position-statement.md
[ComponentConfig]: ../../api/author.md#ComponentConfig

View File

@@ -0,0 +1,29 @@
---
slug: environments
title: Environments
description: Managing Environments - dev, test, stage, prod.
sidebar_position: 130
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Environments
## Overview
## The Code
### Generating the structure
### Using an example Component
## Defining Environments
### Defining one Environment
### Defining a collection of Environments
## Rendering manifests
## Reviewing the Manifests

View File

@@ -0,0 +1,25 @@
---
slug: .
title: Structures
description: Commonly used CUE structures.
sidebar_position: 120
---
import DocCardList from '@theme/DocCardList';
# Structures
This section has self contained articles covering commonly used CUE structures.
These structures are organized and presented as recipes you may adopt and adjust
to your unique organization.
:::important
Structures are defined by Holos Users, unlike the standardized [Core] and
[Author] schemas defined by the Holos Authors.
:::
---
<DocCardList />
[Core]: ../../api/core.md
[Author]: ../../api/author.md

View File

@@ -0,0 +1,29 @@
---
slug: owners
title: Owners
description: Managing and mapping projects to owners.
sidebar_position: 150
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Owners
## Overview
## The Code
### Generating the structure
### Using an example Component
## Defining Owners
### Defining one Owner
### Defining a collection of Owners
## Rendering manifests
## Reviewing the Manifests

View File

@@ -0,0 +1,29 @@
---
slug: projects
title: Projects
description: Managing components organizing them into projects.
sidebar_position: 140
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Projects
## Overview
## The Code
### Generating the structure
### Using an example Component
## Defining Projects
### Defining one Project
### Defining a collection of Projects
## Rendering manifests
## Reviewing the Manifests

View File

@@ -1,15 +0,0 @@
---
slug: cue-generator
title: CUE Generator
description: Render component manifests directly from CUE.
sidebar_position: 50
---
# CUE
Key points to cover:
1. Resources are validated against `#Resources` defined at the root.
2. Custom Resource Definitions need to be imported with timoni.
3. One component can have multiple generators, e.g. Helm and CUE together. This
is how resources are mixed in to helm charts.

278
doc/md/tutorial/cue.mdx Normal file
View File

@@ -0,0 +1,278 @@
---
slug: cue
title: CUE
description: Render component manifests directly from CUE.
sidebar_position: 50
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# CUE
## Overview
This tutorial demonstrates mixing additional resources into a component using
CUE. Holos components frequently mix in resources so we don't need to modify
existing charts or manifests. We'll add an [ExternalSecret] resource to the
podinfo Helm chart we configured in the [Hello Holos] tutorial.
Key concepts:
1. Resources are validated against `#Resources` defined at the root.
2. Custom Resource Definitions need to be imported with timoni.
3. Helm, Kustomize, and CUE can be mixed in together in the same component.
## The Code
### Generating the structure
Use `holos` to generate a minimal platform directory structure. First, create
and cd into a blank directory. Then use the `holos generate platform` command to
generate a minimal platform.
```shell
mkdir holos-cue-tutorial && cd holos-cue-tutorial
holos init platform v1alpha5
```
### Creating the component
Create the directory for the `podinfo` component. Create an empty file and then
add the following CUE configuration to it.
```bash
mkdir -p components/podinfo
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
// export the component build plan to holos
holos: Component.BuildPlan
// Component is a Helm chart
Component: #Helm & {
Name: "podinfo"
Namespace: "default"
// Add metadata.namespace to all resources with kustomize.
KustomizeConfig: Kustomization: namespace: Namespace
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
```bash
EOF
```
Integrate the component with the platform.
```bash
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="tutorial-hello-render-manifests">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```
cached podinfo 6.6.2
rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
</Tabs>
Add and commit the initial configuration.
```bash
git init . && git add . && git commit -m initial
```
### Mixing in Resources
We use the [ComponentConfig] `Resources` field to mix in resources to any
component kind. This field is a convenient wrapper around the core [BuildPlan]
[Resources] [Generator].
Create the mixins.cue file.
```bash
cat <<EOF > components/podinfo/mixins.cue
```
```cue showLineNumbers
package holos
// Component fields are unified with podinfo.cue
Component: {
// Concrete values are defined in podinfo.cue
Name: string
Namespace: string
// Resources represents mix-in resources organized as a struct.
Resources: ExternalSecret: (Name): {
// Name is consistent with the component name.
metadata: name: Name
// Namespace is consistent with the component namespace.
metadata: namespace: Namespace
spec: {
// Ensure the target secret name is consistent.
target: name: metadata.name
// Ensure the name in the SecretStore is consistent.
dataFrom: [{extract: {key: metadata.name}}]
refreshInterval: "30s"
secretStoreRef: kind: "SecretStore"
secretStoreRef: name: "default"
}
}
}
```
```bash
EOF
```
:::important
Holos uses CUE to validate mixed in resources against a schema. The `Resources`
field validates against the `#Resources` definition in [resources.cue].
:::
### Importing CRDs
Holos includes CUE schema definitions of the ExternalSecret custom resource
definition (CRD). These schemas are located in the `cue.mod` directory, written by
the `holos init platform` command we executed at the start of this tutorial.
Import your own custom resource definitions using [timoni]. We imported the
ExternalSecret CRDs embedded into `holos` with the following command.
<Tabs groupId="35B1A1A1-D7DF-4D27-A575-28556E182096">
<TabItem value="command" label="Command">
```bash
timoni mod vendor crds -f https://raw.githubusercontent.com/external-secrets/external-secrets/v0.10.5/deploy/crds/bundle.yaml
```
</TabItem>
<TabItem value="output" label="Output">
```txt
2:22PM INF schemas vendored: external-secrets.io/clusterexternalsecret/v1beta1
2:22PM INF schemas vendored: external-secrets.io/clustersecretstore/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/clustersecretstore/v1beta1
2:22PM INF schemas vendored: external-secrets.io/externalsecret/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/externalsecret/v1beta1
2:22PM INF schemas vendored: external-secrets.io/pushsecret/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/secretstore/v1alpha1
2:22PM INF schemas vendored: external-secrets.io/secretstore/v1beta1
2:22PM INF schemas vendored: generators.external-secrets.io/acraccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/ecrauthorizationtoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/fake/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/gcraccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/githubaccesstoken/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/password/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/uuid/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/vaultdynamicsecret/v1alpha1
2:22PM INF schemas vendored: generators.external-secrets.io/webhook/v1alpha1
```
</TabItem>
</Tabs>
:::tip
Take a look at
[cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue] to see
the imported definitions.
:::
Once imported, the last step is to add the resource kind to the `#Resources`
struct. This is most often accomplished by adding a new file which cue unifies
with the existing [resources.cue] file.
## Reviewing Changes
Render the platform with the `ExternalSecret` mixed into the podinfo component.
```shell
holos render platform ./platform
```
Take a look at the diff to see the mixed in `ExternalSecret`.
```shell
git diff deploy
```
```diff
diff --git a/deploy/components/podinfo/podinfo.gen.yaml b/deploy/components/podinfo/podinfo.gen.yaml
index 6e4aec0..f79e9d0 100644
--- a/deploy/components/podinfo/podinfo.gen.yaml
+++ b/deploy/components/podinfo/podinfo.gen.yaml
@@ -112,3 +112,19 @@ spec:
volumes:
- emptyDir: {}
name: data
+---
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+ name: podinfo
+ namespace: default
+spec:
+ dataFrom:
+ - extract:
+ key: podinfo
+ refreshInterval: 30s
+ secretStoreRef:
+ kind: SecretStore
+ name: default
+ target:
+ name: podinfo
```
We saw how to mix in resources using the `Resources` field of the
[ComponentConfig]. This technique approach works for every kind of component in
Holos. We did this without needing to fork the upstream Helm chart so we can
easily update to new podinfo versions as they're released.
## Trying Locally
Optionally apply the manifests Holos rendered to a [Local Cluster].
## Next Steps
This tutorial uses the `#Resources` structure to map resource kinds to their
schema definitions in CUE. This structure is defined in `resources.cue` at the
root of the tree. Take a look at [resources.cue] to see this mapping structure.
Continue to the next tutorial to learn how to define your own data structures
similar to this `#Resources` structure.
[Local Cluster]: ../topics/local-cluster.mdx
[ExternalSecret]: https://external-secrets.io/latest/api/externalsecret/
[Artifact]: ../api/core.md#Artifact
[Resources]: ../api/core.md#Resources
[Generator]: ../api/core.md#Generator
[Hello Holos]: ./hello-holos.mdx
[cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/cue.mod/gen/external-secrets.io/externalsecret/v1beta1/types_gen.cue#L13
[ComponentConfig]: ../api/author.md#ComponentConfig
[timoni]: https://timoni.sh/install/
[resources.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/v1alpha5/resources.cue#L33

View File

@@ -1,13 +0,0 @@
---
slug: Custom Schemas
title: Custom Schemas
description: Define your own custom data structures.
sidebar_position: 70
---
# Custom Schemas
- Work through defining a `#Cluster` schema and a `Clusters` struct.
- Direct the reader to [topics] for more recipes.
[topics]: ../topics.mdx

View File

@@ -1,14 +0,0 @@
---
slug: file-generator
title: File Generator
description: Provide plain files to the Kustomize transformer.
sidebar_position: 51
---
# File Generator
Key points to cover:
1. The file generator loads plain files.
2. Can be used to pass-through a plain file.
3. Can be used as an input to a transformer.

View File

@@ -7,48 +7,113 @@ sidebar_position: 30
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import RenderingOverview from '../diagrams/rendering-overview.mdx';
import PlatformSequence from '../diagrams/render-platform-sequence.mdx';
import ComponentSequence from '../diagrams/render-component-sequence.mdx';
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
import PlatformSequence from '@site/src/diagrams/render-platform-sequence.mdx';
import ComponentSequence from '@site/src/diagrams/render-component-sequence.mdx';
# Hello Holos
One of the first exercises you perform when learning a new programming language
is to print out the "Hello World!" greeting. For Holos, our "Hello Holos"
exercise involves modeling the [podinfo Helm chart][podinfo] that produces a
similar greeting message from a Kubernetes Pod.
## Overview
By the end of this tutorial you will gain the understanding of how to model
an individual Holos Component using a Helm chart as its source.
One of the first exercises we do when learning a new programming language is
printing out a "Hello World!" greeting. Hello Holos configures the [podinfo Helm
chart][podinfo] producing a similar greeting from a Kubernetes Service.
By the end of this tutorial you have an understanding of how to wrap a Helm
Chart as a Holos Component.
## The Code
### Generate a Platform
### Generating the structure
Use `holos` to generate a minimal platform directory structure. First, create
and cd into a blank directory. Then use the `holos generate platform` command to
generate a minimal platform.
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-tutorial
cd holos-tutorial
holos generate platform v1alpha5
mkdir holos-tutorial && cd holos-tutorial
```
Holos creates a `platform` directory containing a `platform.gen.cue` file. This
file is the entry point for your new platform configuration. You'll integrate
components into the platform using the CUE code in this `platform` directory.
Use the `holos init platform` command to initialize a minimal platform in the
blank directory.
### Create a Component
```shell
holos init platform v1alpha5
```
Here's the filesystem tree we'll build in this tutorial.
<Tabs groupId="80D04C6A-BC83-44D0-95CC-CE01B439B159">
<TabItem value="tree" label="Tree">
```text showLineNumbers
holos-tutorial/
├── components/
│   └── podinfo/
│   └── podinfo.cue
├── cue.mod/
├── platform/
│   ├── platform.gen.cue
│   └── podinfo.cue
├── resources.cue
├── schema.cue
└── tags.cue
```
</TabItem>
<TabItem value="details" label="Details">
<div style={{display: "flex"}}>
<div>
```text showLineNumbers
holos-tutorial/
├── components/
│   └── podinfo/
│   └── podinfo.cue
├── cue.mod/
├── platform/
│   ├── platform.gen.cue
│   └── podinfo.cue
├── resources.cue
├── schema.cue
└── tags.cue
```
</div>
<div>
- **Line 1** The platform root is the `holos-tutorial` directory we created.
- **Line 2** This tutorial places components in `components/`. They may reside
anywhere.
- **Line 3** A component is a collection of `*.cue` files at a path.
- **Line 4** We'll create this file and configure the podinfo helm chart in the
next section.
- **Line 5** The CUE module directory. Schema definitions for Kubernetes and
Holos resources reside within the `cue.mod` directory.
- **Line 6** The platform directory is the **main entrypoint** for the `holos
render platform` command.
- **Line 7** `platform.gen.cue` is initialized by `holos init platform` and
contains the Platform spec.
- **Line 8** `podinfo.cue` integrates podinfo with the platform by adding the
component to the platform spec. We'll add ths file after the next section.
- **Line 9** `resources.cue` Defines the Kubernetes resources available to
manage in CUE.
- **Line 10** `schema.cue` Defines the configuration common to all component
kinds.
- **Line 11** `tags.cue` Defines where component parameter values are injected
into the overall platform configuration. We don't need to be concerned with
this file until we cover component parameters.
- **Lines 9-11** Initialized by `holos init platform`, user editable after
initialization.
</div>
</div>
</TabItem>
</Tabs>
### Creating a component
Start by creating a directory for the `podinfo` component. Create an empty file
and then add the following CUE configuration to it.
<Tabs groupId="tutorial-hello-podinfo-helm-cue-code">
<TabItem value="components/podinfo/podinfo.cue" label="Podinfo Helm Chart">
```bash
mkdir -p components/podinfo
touch components/podinfo/podinfo.cue
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
@@ -65,20 +130,38 @@ HelmChart: #Helm & {
url: "https://stefanprodan.github.io/podinfo"
}
}
// Holos marshals Values into values.yaml for Helm.
Values: {
// message is a string with a default value. @tag indicates a value may
// be injected from the platform spec component parameters.
ui: {
message: string | *"Hello World" @tag(greeting, type=string)
}
}
}
```
</TabItem>
</Tabs>
### Integrate the Component
Integrate the `podinfo` component by creating a new file in the `platform`
directory with the following CUE code:
<Tabs groupId="tutorial-hello-register-podinfo-component">
<TabItem value="platform/podinfo.cue" label="Register Podinfo">
```bash
touch platform/podinfo.cue
EOF
```
:::important
CUE loads all of `*.cue` files in the component directory to define component,
similar to Go packages.
:::
:::note
CUE _also_ loads all `*.cue` files from the component leaf directory to the
platform root directory. In this example, `#Helm` on line 6 is defined in
`schema.cue` at the root.
:::
### Integrating the component
Integrate the `podinfo` component into the platform by creating a new CUE file
in the `platform` directory with the following content.
```bash
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
@@ -86,17 +169,25 @@ package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
// Inject a value into the component.
parameters: greeting: "Hello Holos!"
}
```
</TabItem>
</Tabs>
```bash
EOF
```
## Render the Manifests
:::tip
Component parameters may have any name as long as they don't start with
`holos_`.
:::
## Rendering manifests
Render a manifest for `podinfo` using the `holos render platform ./platform`
command.
command. The `platform/` directory is the main entrypoint for this command.
<Tabs groupId="tutorial-hello-render-manifests">
<Tabs groupId="E150C802-7162-4FBF-82A7-77D9ADAEE847">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
@@ -109,10 +200,19 @@ rendered podinfo in 1.938665041s
rendered platform in 1.938759417s
```
</TabItem>
<TabItem value="manifest" label="Manifest Data">
</Tabs>
:::important
Holos rendered the following manifest file by executing `helm template` after
caching `podinfo` locally.
:::
```txt
deploy/components/podinfo/podinfo.gen.yaml
```
<Tabs groupId="0E9C231D-D0E8-410A-A4A0-601842A086A6">
<TabItem value="service" label="Service">
```yaml showLineNumbers
apiVersion: v1
kind: Service
@@ -136,7 +236,10 @@ spec:
selector:
app.kubernetes.io/name: podinfo
type: ClusterIP
---
```
</TabItem>
<TabItem value="deployment" label="Deployment">
```yaml showLineNumbers
apiVersion: apps/v1
kind: Deployment
metadata:
@@ -175,6 +278,8 @@ spec:
- --random-delay=false
- --random-error=false
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!
- name: PODINFO_UI_COLOR
value: '#34577c'
image: ghcr.io/stefanprodan/podinfo:6.6.2
@@ -230,20 +335,34 @@ spec:
</TabItem>
</Tabs>
Holos rendered the `deploy/components/podinfo/podinfo.gen.yaml` file by
executing `helm template` after caching `podinfo` locally.
Holos renders the component with the greeting injected from the platform spec.
```shell
grep -B2 Hello deploy/components/podinfo/podinfo.gen.yaml
```
```yaml
env:
- name: PODINFO_UI_MESSAGE
value: Hello Holos!
```
## Breaking it down
Heres a quick review of the files we created and their purpose:
We run `holos render platform ./platform` because the CUE files in the platform
directory export a [Platform] resource to `holos`. The platform directory is
the entrypoint to the platform rendering process.
| File | Purpose |
| - | - |
| `components/podinfo/podinfo.cue` | Configures the `podinfo` Helm chart as a holos component. |
| `platform/podinfo.cue` | Adds the `podinfo` Helm chart to the list of Components that will be rendered by Holos. |
Components are the building blocks for a Platform. The `platform/podinfo.cue`
file integrates the `podinfo` Component with the Platform.
We run `holos render platform` against the `platform` directory because that
directory exports a [Platform] resource to `holos`.
Holos requires two fields to integrate a component with the platform.
1. A unique name for the component.
2. The component path to the directory containing the CUE files exporting a
`BuildPlan` defining the component.
Component parameters are optional. They allow re-use of the same component.
Refer to the [Component Parameters] topic for more information.
<Tabs groupId="67C1EE71-3EA8-4568-9F6D-0072BA09FF12">
<TabItem value="overview" label="Rendering Overview">
@@ -258,56 +377,14 @@ directory exports a [Platform] resource to `holos`.
</TabItem>
</Tabs>
Components are the building blocks for a Platform, and without them `holos
render platform` does nothing. The `platform/podinfo.cue` file registers the
`podinfo` Component with the Platform, and provides the path to the directory
where the Component's CUE files are located.
:::important
Components can be parameterized.
:::
The Platform spec can re-use the same component path providing it varying input
parameters. This is covered in the [Component Parameters] topic.
:::tip
Holos makes it easy to re-use a Helm chart with multiple customers and
environments. This is not well supported by Helm alone.
:::
The `components/podinfo/podinfo.cue` file unifies the `#Helm`
definition, indicating that we're configuring a Helm chart, along with the
`podinfo` chart's version and repository information. If we wanted to customize
the `podinfo` chart and change any of the chart's values, we would make those
changes here. For example, we could change the message being displayed by
passing the `ui.message` value to the chart:
```cue
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: {
ui: message: "Hello Holos from Podinfo!"
}
}
```
Holos repeats this process for every Component added to the Platform, and since `podinfo`
is the only Component, we're done!
## Next Steps
We've shown how to add a single Helm chart to the Platform, but what if you have
more than one Helm chart and they all need to access the same data? Check
out the [Helm Generator] tutorial to learn more.
We've shown how to integrate one Helm chart to the Platform, but we haven't yet
covered multiple Helm charts. Continue on with the next tutorial to learn how
Holos makes it easy to inject values into multiple components safely and easily.
[podinfo]: https://github.com/stefanprodan/podinfo
[Helm Generator]: ./helm-generator.mdx
[CUE Module]: https://cuelang.org/docs/reference/modules/
[CUE Tags]: https://cuelang.org/docs/howto/inject-value-into-evaluation-using-tag-attribute/
[Platform]: ../api/author.md#Platform
[Component Parameters]: ../topics/component-parameters.mdx

View File

@@ -1,13 +0,0 @@
---
slug: helm-generator
title: Helm Generator
description: Generate manifests from existing Helm Charts
sidebar_position: 40
---
# Helm
Key points to cover:
1. Holos makes it easy to define data once and pass it to multiple charts.
2. Work through the Prometheus and Blackbox example from the v1alpha4 Helm Guide.

View File

@@ -0,0 +1,526 @@
---
slug: helm-values
title: Helm Values
description: Holos provides values to multiple charts easily and safely.
sidebar_position: 40
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<head>
<meta property="og:title" content="Helm Values | Holos" />
<meta property="og:image" content="https://holos.run/img/cards/guides-helm-2.png" />
</head>
# Helm Values
## Overview
Holos makes it easier to integrate multiple Helm charts together. Holos adds
valuable capabilities to Helm and Kustomize.
1. Inject the same value into two or more charts to integrate them safer than Helm alone.
2. Add strong type checking and validation of constraints for Helm input values.
3. Implementation of the [rendered manifests pattern].
We'll manage the [prometheus] and [blackbox] Helm Charts in this tutorial.
Unfortunately, the upstream `values.yaml` files are misconfigured by default.
Prometheus tries to connect to Blackbox at the wrong host and port.
:::tip
Holos and CUE makes it easy to fix this problem by integrating them correctly.
The integrated charts will stay in lock step over time.
:::
## The Code
### Generating the structure
Use `holos` to generate a minimal platform directory structure. First, create
and cd into a blank directory. Then use the `holos init platform` command.
```shell
mkdir holos-helm-values-tutorial
cd holos-helm-values-tutorial
holos init platform v1alpha5
```
Make a commit to track changes.
```bash
git init . && git add . && git commit -m initial
```
### Managing the Components
Create the `prometheus` and `blackbox` component directories, then add each of
the following file contents.
```bash
mkdir -p components/prometheus components/blackbox
```
<Tabs groupId="D15A3008-1EFC-4D34-BED1-15BC0C736CC3">
<TabItem value="prometheus.cue" label="prometheus.cue">
```bash
cat <<EOF > components/prometheus/prometheus.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus"
version: "25.27.0"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
```
```bash
EOF
```
</TabItem>
<TabItem value="blackbox.cue" label="blackbox.cue">
```bash
cat <<EOF > components/blackbox/blackbox.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
holos: Helm.BuildPlan
Helm: #Helm & {
Chart: {
name: "prometheus-blackbox-exporter"
version: "9.0.1"
repository: {
name: "prometheus-community"
url: "https://prometheus-community.github.io/helm-charts"
}
}
}
```
```bash
EOF
```
</TabItem>
</Tabs>
### Integrating the Components
Integrate the components with the platform by adding the following file to the
platform directory.
```bash
cat <<EOF > platform/prometheus.cue
```
```cue showLineNumbers
package holos
Platform: Components: {
prometheus: {
name: "prometheus"
path: "components/prometheus"
}
blackbox: {
name: "blackbox"
path: "components/blackbox"
}
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="33D6BFED-62D8-4A42-A26A-F3121D57C4E5">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached prometheus-blackbox-exporter 9.0.1
rendered blackbox in 3.825430417s
cached prometheus 25.27.0
rendered prometheus in 4.840089667s
rendered platform in 4.840137792s
```
</TabItem>
</Tabs>
Commit the results.
<Tabs groupId="446CC550-A634-45C0-BEC7-992E5C56D4FA">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add blackbox and prometheus'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main b5df111] add blackbox and prometheus
5 files changed, 1550 insertions(+)
create mode 100644 components/blackbox/blackbox.cue
create mode 100644 components/prometheus/prometheus.cue
create mode 100644 deploy/components/blackbox/blackbox.gen.yaml
create mode 100644 deploy/components/prometheus/prometheus.gen.yaml
create mode 100644 platform/prometheus.cue
```
</TabItem>
</Tabs>
### Importing Helm Values
Holos renders the helm charts with their default values. We can import these
default values into CUE so we can easily work with them as structured data
instead of text markup.
```bash
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/prometheus/values.cue \
components/prometheus/vendor/25.27.0/prometheus/values.yaml
```
```bash
holos cue import \
--package holos \
--path 'Helm: Values:' \
--outfile components/blackbox/values.cue \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
```
These commands convert the YAML data into CUE code and nest the values under the
`Values` field of the `Helm` struct.
:::important
CUE unifies `values.cue` with the other `*.cue` files in the same directory.
:::
Render the platform and commit the results.
<Tabs groupId="BDDCD65A-2E9D-4BA6-AAE2-8099494D5E4B">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered blackbox in 365.936792ms
rendered prometheus in 371.855875ms
rendered platform in 372.109916ms
```
</TabItem>
</Tabs>
<Tabs groupId="1636C619-258E-4D49-8052-F64B588C9177">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'import values'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 52e90ea] import values
2 files changed, 1815 insertions(+)
create mode 100644 components/blackbox/values.cue
create mode 100644 components/prometheus/values.cue
```
</TabItem>
</Tabs>
### Managing Common Configuration
Define a structure to hold the configuration values we want both helm charts to
use. We add this configuration to the `components` directory so it's in scope
for all components.
```bash
cat <<EOF > components/blackbox.cue
```
```cue showLineNumbers
package holos
// Schema Definition
#Blackbox: {
// host constrained to a lower case dns label
host: string & =~"^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
// port constrained to a valid range
port: int & >0 & <=65535
}
// Concrete values must validate against the schema.
Blackbox: #Blackbox & {
host: "blackbox"
port: 9115
}
```
```bash
EOF
```
:::important
1. CUE loads and unifies all `*.cue` files from the root directory containing
`cue.mod` to the leaf component path directory.
2. CUE validates types _and_ constraints. Validation with CUE is better than
languages with only type checking.
:::
Add and commit the configuration.
<Tabs groupId="A738CCE4-F0C6-4CC7-BE1F-2B92F0E86FDC">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add blackbox configuration'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 1adcd08] add blackbox configuration
1 file changed, 15 insertions(+)
create mode 100644 components/blackbox.cue
```
</TabItem>
</Tabs>
### Referring to Common Configuration
Referencing common configuration from multiple components is easy and safe with
Holos and CUE.
Patch the two `values.cue` files, or edit them by hand, to reference
`Blackbox.host` and `Blackbox.port`.
<Tabs groupId="5FFCE892-B8D4-4F5B-B2E2-39EC9E9F87A4">
<TabItem value="command" label="Command">
```bash
patch -p1 < values.patch
```
</TabItem>
<TabItem value="patch" label="values.patch">
```diff
--- a/components/blackbox/values.cue
+++ b/components/blackbox/values.cue
@@ -1,6 +1,8 @@
package holos
Helm: Values: {
+ fullnameOverride: Blackbox.host
+
global: {
//# Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
//#
@@ -192,7 +194,7 @@ Helm: Values: {
annotations: {}
labels: {}
type: "ClusterIP"
- port: 9115
+ port: Blackbox.port
ipDualStack: {
enabled: false
ipFamilies: ["IPv6", "IPv4"]
--- a/components/prometheus/values.cue
+++ b/components/prometheus/values.cue
@@ -1083,7 +1083,7 @@ Helm: Values: {
target_label: "__param_target"
}, {
target_label: "__address__"
- replacement: "blackbox"
+ replacement: "\(Blackbox.host):\(Blackbox.port)"
}, {
source_labels: ["__param_target"]
target_label: "instance"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
patching file 'components/blackbox/values.cue'
patching file 'components/prometheus/values.cue'
```
</TabItem>
</Tabs>
:::important
Both charts now use the same values in lock step. Holos and CUE integrate them
safely and easily.
:::
Remove the patch file, then commit the changes.
<Tabs groupId="6498B00E-FADA-4EB2-885C-808F1D22E04D">
<TabItem value="command" label="Command">
```bash
rm values.patch
```
```bash
git add . && git commit -m 'integrate blackbox and prometheus together'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 4221803] integrate blackbox and prometheus together
2 files changed, 4 insertions(+), 2 deletions(-)
```
</TabItem>
</Tabs>
## Reviewing Changes
Holos makes it easy to see and review platform wide changes. Render the
platform to see how both prometheus and blackbox change in lock step.
<Tabs groupId="E7F6D8B1-22FA-4075-9B44-D9F2815FE0D3">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered blackbox in 374.810666ms
rendered prometheus in 382.899334ms
rendered platform in 383.270625ms
```
</TabItem>
</Tabs>
Changes are easily visible with version control.
<Tabs groupId="9789A0EF-24D4-4FB9-978A-3895C2778789">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/components/blackbox/blackbox.gen.yaml b/deploy/components/blackbox/blackbox.gen.yaml
index 3db20cd..5336f44 100644
--- a/deploy/components/blackbox/blackbox.gen.yaml
+++ b/deploy/components/blackbox/blackbox.gen.yaml
@@ -7,7 +7,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -31,7 +31,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
---
apiVersion: v1
@@ -43,7 +43,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
ports:
@@ -65,7 +65,7 @@ metadata:
app.kubernetes.io/name: prometheus-blackbox-exporter
app.kubernetes.io/version: v0.25.0
helm.sh/chart: prometheus-blackbox-exporter-9.0.1
- name: prometheus-blackbox-exporter
+ name: blackbox
namespace: default
spec:
replicas: 1
@@ -119,8 +119,8 @@ spec:
name: config
hostNetwork: false
restartPolicy: Always
- serviceAccountName: prometheus-blackbox-exporter
+ serviceAccountName: blackbox
volumes:
- configMap:
- name: prometheus-blackbox-exporter
+ name: blackbox
name: config
diff --git a/deploy/components/prometheus/prometheus.gen.yaml b/deploy/components/prometheus/prometheus.gen.yaml
index 9e02bce..ab638f0 100644
--- a/deploy/components/prometheus/prometheus.gen.yaml
+++ b/deploy/components/prometheus/prometheus.gen.yaml
@@ -589,7 +589,7 @@ data:
- source_labels:
- __address__
target_label: __param_target
- - replacement: blackbox
+ - replacement: blackbox:9115
target_label: __address__
- source_labels:
- __param_target
```
</TabItem>
</Tabs>
From the diff we know this change will:
1. Reconfigure the blackbox exporter host from `prometheus-blackbox-exporter` to `blackbox`.
2. Have no effect on the blackbox service port. It was already using the default 9115.
3. Reconfigure prometheus to query the blackbox exporter at the correct host and
port, `blackbox:9115`.
Without this change prometheus incorrectly assumed blackbox was listening at
`blackbox` on port `80` when it was actually listening at
`prometheus-blackbox-exporter` port `9115`. Going forward, changing the
blackbox host or port will reconfigure both charts correctly.
Commit the changes and move on to deploying them.
<Tabs groupId="F8C9A98D-DE1E-4EF6-92C1-017A9166F6C7">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'render integrated blackbox and prometheus manifests'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 67efe0d] render integrated blackbox and prometheus manifests
2 files changed, 7 insertions(+), 7 deletions(-)
```
</TabItem>
</Tabs>
## Trying Locally
Optionally apply the manifests Holos rendered to a [Local Cluster].
## Next Steps
In this tutorial we learned how Holos makes it easier to holistically integrate
the [prometheus] and [blackbox] charts so they're configured in lock step with
each other. If we relied on Helm alone, there is no good way to configure both
charts to use the same service endpoint.
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
[prometheus]: https://github.com/prometheus-community/helm-charts/tree/prometheus-25.27.0/charts/prometheus
[blackbox]: https://github.com/prometheus-community/helm-charts/tree/prometheus-blackbox-exporter-9.0.1/charts/prometheus-blackbox-exporter
[httpbin]: https://github.com/mccutchen/go-httpbin/tree/v2.15.0
[Config Schema]: #config-schema
[Technical Overview]: ./overview.mdx
[Local Cluster]: ../topics/local-cluster.mdx

View File

@@ -1,10 +0,0 @@
---
slug: kustomize-transformer
title: Kustomize Transformer
description: Holos makes it easy to Kustomize a Helm chart.
sidebar_position: 60
---
# Helm Kustomization
Introduce Kustomization transformer.

View File

@@ -0,0 +1,407 @@
---
slug: kustomize
title: Kustomize
description: Holos makes it easy to Kustomize configuration.
sidebar_position: 45
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Kustomize
## Overview
In the previous tutorial we learned how Holos makes it easier to holistically
integrate the [prometheus] and [blackbox] charts so they're configured in lock
step with each other.
This tutorial goes further by integrating the [httpbin] service with prometheus
and blackbox to automatically probe for availability.
We'll explore how Holos manages [kustomize] bases similar to the Helm kind
covered in the [Helm Values] tutorial.
## The Code
### Generating the structure
<Tabs>
<TabItem value="optional" label="Optional">
:::note Skip this step if you completed the [Helm Values] tutorial.
Otherwise click the **Generate** tab to generate a blank platform now.
:::
</TabItem>
<TabItem value="generate" label="Generate">
Use `holos` to generate a minimal platform directory structure. First, create
and cd into a blank directory. Then use the `holos init platform` command.
```shell
mkdir holos-kustomize-tutorial
cd holos-kustomize-tutorial
holos init platform v1alpha5
```
Make a commit to track changes.
```bash
git init . && git add . && git commit -m initial
```
</TabItem>
</Tabs>
### Managing the Component
Create the `httpbin` component directory and add the `httpbin.cue` and
`httpbin.yaml` files to it.
<Tabs groupId="800C3AE7-E7F8-4AFC-ABF1-6AFECD945958">
<TabItem value="setup" label="Setup">
```bash
mkdir -p components/httpbin
```
</TabItem>
<TabItem value="components/httpbin/httpbin.cue" label="httpbin.cue">
```bash
cat <<EOF > components/httpbin/httpbin.cue
```
```cue showLineNumbers
package holos
// Produce a Kustomize BuildPlan for Holos
holos: Kustomize.BuildPlan
// https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/README.md
Kustomize: #Kustomize & {
KustomizeConfig: {
// Files tells Holos to copy the file from the component path to the
// temporary directory Holos uses for BuildPlan execution.
Files: {
"httpbin.yaml": _
}
CommonLabels: {
"app.kubernetes.io/name": "httpbin"
}
// Kustomization represents a kustomization.yaml file in CUE. Holos
// marshals this field into a `kustomization.yaml` while processing a
// BuildPlan. See
// https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
Kustomization: {
images: [{name: "mccutchen/go-httpbin"}]
// Use a hidden field to compose patches easily with a struct. Hidden
// fields are not included in exported structures.
_patches: {}
// Convert the hidden struct to a list.
patches: [for x in _patches {x}]
}
}
}
```
```bash
EOF
```
</TabItem>
<TabItem value="components/httpbin/httpbin.yaml" label="httpbin.yaml">
```bash
cat <<EOF > components/httpbin/httpbin.yaml
```
```yaml showLineNumbers
# https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/resources.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
template:
spec:
containers:
- name: httpbin
image: mccutchen/go-httpbin
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /status/200
port: http
readinessProbe:
httpGet:
path: /status/200
port: http
resources: {}
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
appProtocol: http
```
```bash
EOF
```
</TabItem>
</Tabs>
Holos knows the `httpbin.yaml` file is part of the BuildPlan because of the
`KustomizeConfig: Files: "httpbin.yaml": _` line in the `httpbin.cue`.
### Integrating the Components
Integrate `httpbin` with the platform by adding the following file to the
platform directory.
```bash
cat <<EOF > platform/httpbin.cue
```
```cue showLineNumbers
package holos
Platform: Components: {
httpbin: {
name: "httpbin"
path: "components/httpbin"
}
}
```
```bash
EOF
```
Render the platform.
<Tabs groupId="B120D5D1-0EAB-41E0-AD21-15526EBDD53D">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered httpbin in 707.554666ms
rendered platform in 707.9845ms
```
</TabItem>
</Tabs>
Commit the results.
<Tabs groupId="446CC550-A634-45C0-BEC7-992E5C56D4FA">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'add httpbin'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main c05f9ef] add httpbin
4 files changed, 118 insertions(+)
create mode 100644 components/httpbin/httpbin.cue
create mode 100644 components/httpbin/httpbin.yaml
create mode 100644 deploy/components/httpbin/httpbin.gen.yaml
create mode 100644 platform/httpbin.cue
```
</TabItem>
</Tabs>
### Inspecting the BuildPlan
We can see the [BuildPlan] exported to `holos` by the `holos:
Kustomize.BuildPlan` line in `httpbin.cue`. Holos processes this build plan to
produce the fully rendered manifests.
<Tabs groupId="DD697D65-5BEC-4B92-BB33-59BE4FEC112F">
<TabItem value="command" label="Command">
```bash
holos cue export --expression holos --out=yaml ./components/httpbin
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
- kind: File
output: httpbin.yaml
file:
source: httpbin.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
- httpbin.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/name: httpbin
patches: []
images:
- name: mccutchen/go-httpbin
resources:
- resources.gen.yaml
- httpbin.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
source:
component:
name: no-name
path: no-path
parameters: {}
```
</TabItem>
</Tabs>
### Transforming manifests
Reviewing the BuildPlan exported in the previous command:
1. The [File Generator] copies the plain `httpbin.yaml` file into the build.
2. The [Kustomize Transformer] uses `httpbin.yaml` as an input resource.
3. The final artifact is the output from Kustomize.
This BuildPlan transforms the raw yaml by labeling all of the resources with
`"app.kubernetes.io/name": "httpbin"` using the [KustomizeConfig] `CommonLabels`
field. We still need to integrate `httpbin` with `prometheus`. Annotate the
Service with `prometheus.io/probe: "true"` to complete the integration. Holos
makes this easier with CUE. We don't need to edit any yaml files.
Add a new `patches.cue` file to the `httpbin` component with the following
content.
```bash
cat <<EOF > components/httpbin/patches.cue
```
```cue showLineNumbers
package holos
import "encoding/yaml"
// Mix in a Kustomize patch to the configuration.
Kustomize: KustomizeConfig: Kustomization: _patches: {
probe: {
target: kind: "Service"
target: name: "httpbin"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/prometheus.io~1probe"
value: "true"
}])
}
}
```
```bash
EOF
```
:::note
We use a hidden `_patches` field to easily unify data into a struct, then
convert the struct into a list for export.
:::
## Reviewing Changes
Render the platform to see the result of the kustomization patch.
<Tabs groupId="5D1812DD-8E7B-4F97-B349-275214F38B6E">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered httpbin in 197.030208ms
rendered platform in 197.416416ms
```
</TabItem>
</Tabs>
Holos is configuring Kustomize to patch the plain `httpbin.yaml` file with the
annotation.
<Tabs groupId="3D80279E-8EDE-4B3E-9269-50F5D1C1CA42">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/components/httpbin/httpbin.gen.yaml b/deploy/components/httpbin/httpbin.gen.yaml
index 298b9a8..a16bd1a 100644
--- a/deploy/components/httpbin/httpbin.gen.yaml
+++ b/deploy/components/httpbin/httpbin.gen.yaml
@@ -1,6 +1,8 @@
apiVersion: v1
kind: Service
metadata:
+ annotations:
+ prometheus.io/probe: "true"
labels:
app.kubernetes.io/name: httpbin
name: httpbin
```
</TabItem>
</Tabs>
Add and commit the final changes.
<Tabs groupId="54C335C8-B382-4277-AE87-0D6556921955">
<TabItem value="command" label="Command">
```bash
git add . && git commit -m 'annotate httpbin for prometheus probes'
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 6eeeadb] annotate httpbin for prometheus probes
2 files changed, 3 insertions(+), 1 deletion(-)
```
</TabItem>
</Tabs>
## Trying Locally
Optionally apply the manifests Holos rendered to a [Local Cluster].
## Next Steps
We learned how Holos makes it easier to manage [httpbin], distributed as a
Kustomize base, using a kustomize Component similar to the helm component we saw
previously. Holos offers a clear way to kustomize any component, patching an
annotation onto the `httpbin` Service in this example.
Continue on with the tutorial to explore how Holos makes it easier to manage
certificates and make services accessible outside of a cluster.
[httpbin]: https://github.com/mccutchen/go-httpbin/tree/v2.15.0
[prometheus]: https://github.com/prometheus-community/helm-charts/tree/prometheus-25.27.0/charts/prometheus
[blackbox]: https://github.com/prometheus-community/helm-charts/tree/prometheus-blackbox-exporter-9.0.1/charts/prometheus-blackbox-exporter
[kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
[Helm Values]: ./helm-values.mdx
[File Generator]: ../api/core.md#File
[Kustomize Transformer]: ../api/core.md#Kustomize
[BuildPlan]: ../api/core.md#BuildPlan
[KustomizeConfig]: ../api/author.md#KustomizeConfig
[Local Cluster]: ../topics/local-cluster.mdx

View File

@@ -5,12 +5,12 @@ description: Learn how Holos integrates software into a holistic platform.
sidebar_position: 10
---
import RenderingOverview from '../diagrams/rendering-overview.mdx';
import RenderPlatformDiagram from '../diagrams/render-platform-sequence.mdx';
import RenderComponentDiagram from '../diagrams/render-component-sequence.mdx';
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
# Tutorial
## Overview
Holos is a configuration management tool for Kubernetes resources. It provides
the building blocks needed for implementing the [rendered manifests pattern].
It gives the flexibility to manage a wide range of configurations, from large
@@ -27,27 +27,19 @@ At a high level, Holos provides a few major components:
<RenderingOverview />
{/* TODO: Replace this with the Advantages diagram we talked about. */}
## Holos' role in your organization
Platform engineers run the `holos render platform` command locally and in CI to
produce Kubernetes manifests which are committed to version control. GitOps
tools like ArgoCD or Flux deploy the manifests produced by Holos.
Other tools focus on individual applications. Holos focuses holistically on the
integration of applications into a whole platform. In this way a single one
line configuration change, like changing the domain name, is clearly visible
across the whole platform.
Rendering a platform may produce a single ConfigMap resource or produce millions
of lines of fully rendered manifests configuring multiple environments,
clusters, and regions.
<RenderPlatformDiagram />
Holos works well with your existing Helm charts, kustomize bases, and any other
configuration data you currently store in version control.
## Advantages of Holos
This section outlines some advantages of Holos.
### Safe
Holos uses [CUE] to provide strong typing and constraints to configuration data.
@@ -60,11 +52,6 @@ Holos offers a consistent way to incorporate a wide variety of tools into a well
defined data pipeline. Configuration produced from CUE, Helm, and Kustomize are
all handled with the same consistent process.
### Easy
Holos makes it easy to define configuration data once, then safely use the data
in multiple Helm charts, Kustomize bases, or any other supported component kind.
### Flexible
Holos is designed to be flexible. Holos offers flexible building blocks for
@@ -76,40 +63,6 @@ For example, environments and clusters are explicitly kept out of the core and
instead are provided as flexible, user-customizable [topics] organized as a
recipes.
<RenderComponentDiagram />
## When not to use Holos
Holos is useful for many organizations but there are situations where Holos is
not a good fit.
### Implicit GitOps
Holos takes an explicit view of GitOps, meaning the fully rendered manifests are
complete. These manifests are stored in version control without an implicit
dependency on external systems. If your team uses an implicit approach, meaning
some configuration comes from elsewhere, then Holos may not be a good fit.
For example, if container image tags are managed by a system outside of version
control then holos may not be a good fit.
Holos is a good fit if you're open to integrate external systems with Holos such
that the external data becomes an input to produce the fully rendered manifests.
### CUE is a Non Starter
Holos heavily uses CUE. For example, when using Holos with Helm and Kustomize,
we no longer need to write YAML files and text templates. All configuration is
expressed in CUE and `holos` handles the YAML on our behalf.
CUE is a relatively simple domain specific language, but it is very different
from most other languages because of it's roots in logic programming languages.
Holos is not a good fit if your team prefers to write code in YAML templates or
turing complete general purpose languages.
For more info see [Why CUE for Configuration].
## Getting Help
If you get stuck, you can get help on [Discord] or [GitHub discussions]. Don't

View File

@@ -7,107 +7,78 @@ sidebar_position: 20
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import RenderPlatformDiagram from '../diagrams/render-platform-sequence.mdx';
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
# Setup
## Overview
This tutorial will guide you through the installation of Holos and its
dependencies, as well as the initialization of a minimal Platform that you can
extend to meet your specific needs.
## Prerequisites
## Installing
Holos integrates with the following tools that should be installed to enable
their functionality.
Holos is distributed as a single file executable that can be installed in a
couple of ways.
* [Helm] to fetch and render Helm chart Components [Kubectl] to [kustomize]
* components.
<Tabs groupId="FE2C74C8-B3A3-4AEA-BBD3-F57FAA654B6F">
<TabItem value="brew" label="Install with brew">
```bash
brew install holos-run/tap/holos
```
</TabItem>
<TabItem value="go" label="Go">
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
</TabItem>
</Tabs>
## Install Holos
### Completion
Holos is distributed as a single file executable that can be installed in a couple of ways.
<Tabs groupId="65F79D28-2E57-4A90-8EBA-3D8758C80233">
<TabItem value="zsh" label="zsh">
```bash
source <(holos completion zsh)
```
</TabItem>
<TabItem value="bash" label="bash">
```bash
source <(holos completion bash)
```
</TabItem>
<TabItem value="fish" label="fish">
```bash
source <(holos completion fish)
```
</TabItem>
<TabItem value="powershell" label="powershell">
```bash
holos completion powershell | Invoke-Expression
```
</TabItem>
</Tabs>
### Releases
Download `holos` from the [releases] page and place the executable into your
shell path.
### Go install
### Dependencies
Alternatively, install directly into your go bin path using:
Holos integrates with the following tools that should be installed to enable
their functionality.
```shell
go install github.com/holos-run/holos/cmd/holos@latest
```
- [Helm] to fetch and render Helm chart Components.
- [Kubectl] to [kustomize] components.
## Generate a new Platform
Use `holos` to generate a minimal platform using the recommended directory
structure. First, create and cd into a blank directory. Then use the `holos
generate platform` command to generate a minimal platform.
```shell
mkdir holos-tutorial
cd holos-tutorial
holos generate platform v1alpha5
```
Holos creates a `platform` directory containing a `platform.gen.cue` file
serving as the entry point for your new platform configuration. You'll register
software components with the platform using the CUE code in this `platform`
directory.
## Render the Platform
We usually make configuration changes with holos in the following loop:
1. Edit some CUE code.
2. Render the whole platform.
3. Review the fully rendered manifests.
4. Repeat
Rendering the platform to plain manifest files allows us to see the changes
clearly.
<Tabs groupId="66379FE3-206D-4DF1-BAF0-29C9A6C17D00">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered platform in 8.292µs
```
</TabItem>
<TabItem value="diagram" label="Diagram">
<RenderPlatformDiagram />
</TabItem>
</Tabs>
In this initial state `holos` doesn't do anything because no components are
registered with the platform. You can inspect the Platform definition to get
this insight.
<Tabs groupId="218658D2-1305-46D3-8F55-8DA8EB45F114">
<TabItem value="command" label="Command">
```bash
holos cue export --expression holos --out=yaml ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```yaml
kind: Platform
apiVersion: v1alpha5
metadata:
name: default
spec:
components: []
```
Note the `spec.components` field is empty.
</TabItem>
</Tabs>
:::note
Holos is tested with Helm version `v3.16.2`.
:::
Please try upgrading helm if you encounter `Error: chart requires kubeVersion
...` errors.
## Next Steps
@@ -115,21 +86,6 @@ You've got the structure of your platform configuration in place. Continue on to
[Hello Holos] where you'll learn how easy it is to manage a Helm chart with
holos.
## Do I need Kubernetes?
No. Holos generates fully rendered Kubernetes manifests but does not apply
them. You don't need a Kubernetes cluster to start using Holos and reviewing
the files it creates.
You will need a cluster to apply the rendered manifests and interact with the
resources and services Holos configures. We recommend using our [Local Cluster]
guide to set up a minimal [k3d] Kubernetes cluster with [OrbStack] or [Docker].
You might like our [Local Cluster] guide because it:
1. Includes a script to quickly reset the cluster to a known good state.
2. Configures proper DNS and TLS certificates.
[Helm]: https://github.com/helm/helm/releases
[Kubectl]: https://kubernetes.io/docs/tasks/tools/
[kustomize]: https://kustomize.io/

View File

@@ -7,6 +7,9 @@ image: /img/cards/announcing-holos.png
description: Holistically manage Helm and Kustomize with CUE
---
import RenderingOverview from '@site/src/diagrams/rendering-overview.mdx';
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';
<head>
<title>Announcing Holos</title>
<meta property="og:title" content="Announcing Holos" />
@@ -21,39 +24,10 @@ manifests pattern as a data pipeline to fully render manifests generated from
[Kustomize]: https://kustomize.io/
[CUE]: https://cuelang.org/
```mermaid
---
title: Rendered Manifest Pipeline
---
graph LR
Platform[<a href="/docs/api/author/v1alpha4/#Platform">Platform</a>]
Component[<a href="/docs/api/author/v1alpha4/#ComponentConfig">Components</a>]
<RenderingOverview />
Helm[<a href="/docs/api/author/v1alpha4/#Helm">Helm</a>]
Kustomize[<a href="/docs/api/author/v1alpha4/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/api/author/v1alpha4/#Kubernetes">Kubernetes</a>]
{/* truncate */}
BuildPlan[<a href="/docs/api/core/v1alpha4/#buildplan">BuildPlan</a>]
ResourcesArtifact[<a href="/docs/api/core/v1alpha4/#artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/api/core/v1alpha4/#artifact">GitOps<br/>Artifact</a>]
Generators[<a href="/docs/api/core/v1alpha4/#generators">Generators</a>]
Transformers[<a href="/docs/api/core/v1alpha4/#transformer">Transformers</a>]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Files
```
<!-- truncate -->
At the start of the pandemic I was migrating our platform from VMs managed by
Puppet to Kubernetes. My primary goal was to build an observability system
@@ -85,6 +59,8 @@ a Go command line tool to implement the pattern as a data pipeline. Id been
thinking about the comments from the [Why are we templating YAML] posts and
wondering what an answer to this question would look like.
<RenderPlatformDiagram />
The Go command line tool was an incremental improvement over the CI scripts, but
we still didnt have a good way to handle the data values. We were still
templating YAML which didnt catch errors early enough. It was too easy to
@@ -101,22 +77,16 @@ Take a look at Holos if youre looking to implement the rendered manifests
pattern or cant shake that feeling it should be easier to integrate third party
software into Kubernetes like we felt.
1. [Helm Guide] Walks through how we solved the challenges we faced with the prometheus Helm charts.
2. [Quickstart] Works through how a platform team can define golden paths for other teams using CUE.
3. [Author API] provides an ergonomic way to work with Helm, Kustomize, and CUE resources.
1. [Tutorial] takes a tour of Holos features starting from scratch.
2. [Topics] cover how to customize and tailor Holos to your unique needs.
[Helm Guide]: /docs/guides/helm/
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story
[Holos]: https://holos.run/
[Quickstart]: /docs/quickstart/
[Tutorial]: /docs/v1alpha5/tutorial/overview/
[Topics]: /docs/v1alpha5/topics/
[Holos]: https://holos.run/
[Helm]: https://helm.sh/
[Kustomize]: https://kustomize.io/
[CUE]: https://cuelang.org/

View File

@@ -59,6 +59,14 @@ const config: Config = {
showLastUpdateAuthor: true,
showLastUpdateTime: true,
sidebarPath: './sidebars.ts',
// https://docusaurus.io/docs/versioning#configuring-versioning-behavior
lastVersion: 'current',
versions: {
current: {
label: 'v1alpha5',
path: 'v1alpha5',
}
}
},
blog: {
path: "blog",
@@ -90,7 +98,7 @@ const config: Config = {
}
},
navbar: {
title: '',
title: 'Holos',
logo: {
src: 'img/logo.svg',
srcDark: 'img/logo-dark.svg',
@@ -98,17 +106,9 @@ const config: Config = {
items: [
{
type: 'doc',
docId: 'guides/quickstart',
position: 'left',
label: 'Quickstart',
},
{ to: '/docs/technical-overview', label: 'Docs', position: 'left' },
{ to: '/docs/guides', label: 'Guides', position: 'left' },
{
type: 'doc',
docId: 'api',
position: 'left',
label: 'API',
docId: 'tutorial/overview',
label: 'Docs',
position: 'left'
},
{ to: '/blog', label: 'Blog', position: 'left' },
{
@@ -121,71 +121,17 @@ const config: Config = {
label: 'Discord',
position: 'right',
},
{
type: 'docsVersionDropdown',
position: 'right',
// dropdownItemsAfter: [{ to: '/versions', label: 'All versions' }],
dropdownActiveClassDisabled: true,
},
],
},
footer: {
style: 'dark',
links: [
{
title: 'Docs',
items: [
{
label: 'Quickstart',
to: '/docs/quickstart',
},
{
label: 'Concepts',
to: '/docs/concepts',
},
{
label: 'Documentation',
to: '/docs',
},
{
label: 'API Reference',
to: '/docs/api',
},
],
},
{
title: 'Community',
items: [
{
label: 'Support',
href: '/docs/support',
},
{
label: 'Discord',
href: 'https://discord.gg/JgDVbNpye7',
},
{
label: 'Discussion List',
href: 'https://groups.google.com/g/holos-discuss',
},
{
label: 'Discussion Forum',
href: 'https://github.com/holos-run/holos/discussions',
},
],
},
{
title: 'More',
items: [
{
label: 'Blog',
to: '/blog',
},
{
label: 'GitHub',
href: 'https://github.com/holos-run/holos',
},
{
label: 'GoDoc',
href: 'https://pkg.go.dev/github.com/holos-run/holos?tab=doc',
}
],
},
],
links: [],
copyright: `Copyright © ${new Date().getFullYear()} The Holos Authors.`,
},
prism: {

File diff suppressed because it is too large Load Diff

View File

@@ -15,10 +15,10 @@
"typecheck": "tsc"
},
"dependencies": {
"@docusaurus/core": "^3.6.0",
"@docusaurus/plugin-client-redirects": "^3.6.0",
"@docusaurus/preset-classic": "^3.6.0",
"@docusaurus/theme-mermaid": "^3.6.0",
"@docusaurus/core": "^3.6.1",
"@docusaurus/plugin-client-redirects": "^3.6.1",
"@docusaurus/preset-classic": "^3.6.1",
"@docusaurus/theme-mermaid": "^3.6.1",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
@@ -26,9 +26,9 @@
"react-dom": "^18.0.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.6.0",
"@docusaurus/tsconfig": "^3.6.0",
"@docusaurus/types": "^3.6.0",
"@docusaurus/module-type-aliases": "^3.6.1",
"@docusaurus/tsconfig": "^3.6.1",
"@docusaurus/types": "^3.6.1",
"@wcj/html-to-markdown-cli": "^2.1.1",
"cspell": "^8.10.4",
"html-to-markdown": "^1.0.0",

View File

@@ -15,14 +15,14 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/base00/undraw_software_engineer_re_tnjc.svg').default,
description: (
<>
<p align="left">
<p style={{ textAlign: 'left' }}>
<ul>
<li>Provide simple definitions for other teams to use as golden paths.</li>
<li>Define integrations in <a href="https://cuelang.org/">CUE</a> with strong type checking. No more text templates or bash scripts.</li>
<li>Reuse your existing Helm charts and Kustomize bases.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
<a href="/docs/v1alpha5/tutorial/overview/">Learn More</a>
</>
),
},
@@ -31,14 +31,14 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/base00/undraw_through_the_park_lxnl.svg').default,
description: (
<>
<p align="left">
<p style={{ textAlign: 'left' }}>
<ul>
<li>Move faster using paved paths from your platform and security teams.</li>
<li>Develop locally or in the cloud.</li>
<li>Spend more time developing software and fewer cycles fighting infrastructure challenges.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
<a href="/docs/v1alpha5/tutorial/overview/">Learn More</a>
</>
),
},
@@ -47,14 +47,14 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/base00/undraw_security_on_re_e491.svg').default,
description: (
<>
<p align="left">
<p style={{ textAlign: 'left' }}>
<ul>
<li>Define security policy as reusable, typed configurations.</li>
<li>Automatically enforce security policy on new projects.</li>
<li>Ensure a consistent security posture cross-platform with fewer code changes.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
<a href="/docs/v1alpha5/tutorial/overview/">Learn More</a>
</>
),
}

View File

@@ -1,6 +1,6 @@
```mermaid
---
title: holos render component
title: holos render component sequence diagram
---
sequenceDiagram
participant HRC as holos<br />render component

View File

@@ -1,6 +1,6 @@
```mermaid
---
title: holos render platform
title: holos render platform sequence diagram
---
sequenceDiagram
participant HRP as holos<br />render platform

View File

@@ -0,0 +1,32 @@
```mermaid
---
title: Rendering Overview
---
graph LR
Platform[<a href="/docs/v1alpha5/api/author/#Platform">Platform</a>]
Component[<a href="/docs/v1alpha5/api/author/#ComponentConfig">Components</a>]
Helm[<a href="/docs/v1alpha5/api/author/#Helm">Helm</a>]
Kustomize[<a href="/docs/v1alpha5/api/author/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/v1alpha5/api/author/#Kubernetes">Kubernetes</a>]
BuildPlan[<a href="/docs/v1alpha5/api/core/#BuildPlan">BuildPlan</a>]
ResourcesArtifact[<a href="/docs/v1alpha5/api/core/#Artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/v1alpha5/api/core/#Artifact">GitOps<br/>Artifact</a>]
Generators[<a href="/docs/v1alpha5/api/core/#Generator">Generators</a>]
Transformers[<a href="/docs/v1alpha5/api/core/#Transformer">Transformers</a>]
Validators[Validators]
Files[Manifest<br/>Files]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Validators --> Files
```

View File

@@ -23,17 +23,11 @@ function HomepageHeader() {
<div className={styles.buttons}>
<Link
className="button button--secondary button--lg"
to="docs/quickstart">
Get Started
</Link>
<span className={styles.divider}></span>
<Link
className="button button--primary button--lg"
to="docs/technical-overview/">
to="docs/v1alpha5/tutorial/overview/">
Learn More
</Link>
<span className={styles.divider}></span>
</div>
</div>
</div>
</div >
</header >

View File

Before

Width:  |  Height:  |  Size: 248 KiB

After

Width:  |  Height:  |  Size: 248 KiB

View File

Before

Width:  |  Height:  |  Size: 206 KiB

After

Width:  |  Height:  |  Size: 206 KiB

View File

@@ -1,724 +0,0 @@
---
description: Try Holos with this quick start guide.
slug: /archive/2024-09-15-quickstart
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Quickstart
In this guide, you'll experience how Holos makes the process of operating a
Platform safer, easier, and more consistent. We'll use Holos to manage a
vendor-provided Helm chart as a Component. Next, we'll mix in our own custom
resources to manage the Component with GitOps. Finally, you'll see how Holos
makes it safer and easier to maintain software over time by surfacing the exact
changes that will be applied when upgrading the vendor's chart to a new version,
before they are actually made.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap upstream Helm charts.
Optionally, if you'd like to apply the rendered manifests to a real Cluster,
first complete the [Local Cluster Guide](/docs/guides/local-cluster).
## Install Holos
Install Holos with the following command or other methods listed on the
[Installation](/docs/install/) page.
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
## Create a Git Repository
Start by initializing an empty Git repository. Holos operates on local files
stored in a Git repository.
<Tabs groupId="init">
<TabItem value="command" label="Command">
```bash
mkdir holos-quickstart
cd holos-quickstart
git init
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Initialized empty Git repository in /holos-quickstart/.git/
```
</TabItem>
</Tabs>
This guide assumes you will run commands from the root directory of the Git
repository unless stated otherwise.
## Generate the Platform {#Generate-Platform}
Generate the Platform code in the repository root. A Platform refers to the
entire set of software holistically integrated to provide a software development
platform for your organization. In this guide, the Platform will include a
single Component to demonstrate how the concepts fit together.
```bash
holos generate platform quickstart
```
Commit the generated platform config to the repository.
<Tabs groupId="commit-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate platform quickstart - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main (root-commit) 0b17b7f] holos generate platform quickstart
213 files changed, 72349 insertions(+)
...
```
</TabItem>
</Tabs>
## Generate a Component {#generate-component}
The platform you generated is currently empty. Run the following command to
generate the CUE code that defines a Helm Component.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
holos generate component podinfo --component-version 6.6.1
```
</TabItem>
<TabItem value="output" label="Output">
```txt
generated component
```
</TabItem>
</Tabs>
The --component-version 6.6.1 flag intentionally installs an older release.
You'll see how Holos assists with software upgrades later in this guide.
The generate component command creates two files: a leaf file,
`components/podinfo/podinfo.gen.cue`, and a root file, `podinfo.gen.cue`. Holos
leverages the fact that [order is
irrelevant](https://cuelang.org/docs/tour/basics/order-irrelevance/) in CUE to
register the component with the Platform by adding a file to the root of the Git
repository. The second file defines the component in the leaf component
directory.
<Tabs groupId="podinfo-files">
<TabItem value="components/podinfo/podinfo.gen.cue" label="Leaf">
`components/podinfo/podinfo.gen.cue`
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
(#Helm & Chart).Output
let Chart = {
Name: "podinfo"
Version: "6.6.1"
Namespace: "default"
Repo: name: "podinfo"
Repo: url: "https://stefanprodan.github.io/podinfo"
Values: {}
}
```
</TabItem>
<TabItem value="podinfo.gen.cue" label="Root">
`podinfo.gen.cue`
```cue showLineNumbers
package holos
// Manage podinfo on workload clusters only
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name)/podinfo": {
path: "components/podinfo"
cluster: Cluster.name
}
}
```
</TabItem>
</Tabs>
In this example, we provide the minimal information needed to manage the Helm
chart: the name, version, Kubernetes namespace for deployment, and the chart
repository location.
This chart deploys cleanly without any values provided, but we include an empty
Values struct to show how Holos improves consistency and safety in Helm by
leveraging the strong type-checking in CUE. You can safely pass shared values,
such as the organizations domain name, to all Components across all clusters in
the Platform by defining them at the root of the configuration.
Commit the generated component config to the repository.
<Tabs groupId="commit-component">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate component podinfo - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main cc0e90c] holos generate component podinfo
2 files changed, 24 insertions(+)
create mode 100644 components/podinfo/podinfo.gen.cue
create mode 100644 podinfo.gen.cue
```
</TabItem>
</Tabs>
## Render the Component
You can render individual components without adding them to a Platform, which is
helpful when developing a new component.
<Tabs groupId="render-podinfo">
<TabItem value="command" label="Command">
```bash
holos render component ./components/podinfo --cluster-name=default
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached
rendered podinfo
```
</TabItem>
</Tabs>
First, the command caches the Helm chart locally to speed up subsequent
renderings. Then, the command runs Helm to produce the output and writes it into
the deploy directory.
<Tabs groupId="tree-podinfo">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
└── default
└── components
└── podinfo
└── podinfo.gen.yaml
5 directories, 1 file
```
</TabItem>
</Tabs>
The component deploys to one cluster named `default`. In practice, the same
component is often deployed to multiple clusters, such as `east` and `west` to
provide redundancy and increase availability.
:::tip
This example is equivalent to running `helm template` on the chart and saving
the output to a file. Holos simplifies this task, making it safer and more
consistent when managing many charts.
:::
## Mix in an ArgoCD Application
We've seen how Holos works with Helm, but we haven't yet explored how Holos
makes it easier to consistently and safely manage all of the software in a
Platform.
Holos allows you to easily mix in resources that differentiate your Platform.
We'll use this feature to mix in an ArgoCD [Application][application] to manage
the podinfo Component with GitOps. We'll define this configuration in a way that
can be automatically and consistently reused across all future Components added
to the Platform.
Create a new file named `argocd.cue` in the root of your Git repository with the
following contents:
<Tabs groupId="argocd-config">
<TabItem value="command" label="argocd.cue">
```cue showLineNumbers
package holos
#ArgoConfig: {
Enabled: true
RepoURL: "https://github.com/holos-run/holos-quickstart-guide"
}
```
</TabItem>
</Tabs>
:::tip
If you plan to apply the rendered output to a real cluster, change the
`example.com` RepoURL to the URL of the Git repository you created in this
guide. You don't need to change the example if you're just exploring Holos by
inspecting the rendered output without applying it to a live cluster.
:::
With this file in place, render the component again.
<Tabs groupId="render-podinfo-argocd">
<TabItem value="command" label="Command">
```bash
holos render component ./components/podinfo --cluster-name=default
```
</TabItem>
<TabItem value="output" label="Output">
```txt
wrote deploy file
rendered gitops/podinfo
rendered podinfo
```
</TabItem>
</Tabs>
Holos uses the locally cached chart to improve performance and reliability. It
then renders the Helm template output along with an ArgoCD Application resource
for GitOps.
:::tip
By defining the ArgoCD configuration at the root, we again take advantage of the
fact that [order is
irrelevant](https://cuelang.org/docs/tour/basics/order-irrelevance/) in CUE.
:::
Defining the configuration at the root ensures all future leaf Components take
the ArgoCD configuration and render an Application manifest for GitOps
management.
<Tabs groupId="tree-podinfo-argocd">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
└── default
├── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── gitops
└── podinfo.application.gen.yaml
6 directories, 2 files
```
</TabItem>
</Tabs>
Notice the new `podinfo.application.gen.yaml` file created by enabling ArgoCD in
the Helm component. The Application resource in the file looks like this:
<Tabs groupId="podinfo-application">
<TabItem value="file" label="podinfo.application.gen.yaml">
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: ./deploy/clusters/default/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
</Tabs>
:::tip
Holos generates a similar Application resource for every additional Component
added to your Platform.
:::
Finally, add and commit the results to your Platform's Git repository.
<Tabs groupId="commit-argo">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos render component ./components/podinfo --cluster-name=default"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main f95cef1] holos render component ./components/podinfo --cluster-name=default
3 files changed, 134 insertions(+)
create mode 100644 argocd.cue
create mode 100644 deploy/clusters/default/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/default/gitops/podinfo.application.gen.yaml
```
</TabItem>
</Tabs>
In this section, we learned how Holos simplifies mixing resources into
Components, like an ArgoCD Application. Holos ensures consistency by managing an
Application resource for every Component added to the Platform through the
configuration you define in `argocd.cue` at the root of the repository.
## Define Workload Clusters {#workload-clusters}
We've generated a Component to manage podinfo and integrated it with our
Platform, but rendering the Platform doesn't render podinfo. Podinfo isn't
rendered because we haven't assigned any Clusters to the workload Fleet.
Define two new clusters, `east` and `west`, and assign them to the workload
Fleet. Create a new file named `clusters.cue` in the root of your Git repository
with the following contents:
<Tabs groupId="clusters">
<TabItem value="clusters.cue" label="clusters.cue">
```cue showLineNumbers
package holos
// Define two workload clusters for disaster recovery.
#Fleets: workload: clusters: {
// In CUE _ indicates values are defined elsewhere.
east: _
west: _
}
```
</TabItem>
</Tabs>
This example shows how Holos simplifies configuring multiple clusters with
similar configuration by grouping them into a Fleet.
:::tip
Fleets help segment a group of Clusters into one leader and multiple followers
by designating one cluster as the primary. Holos makes it safer, easier, and
more consistent to reconfigure which cluster is the primary. The primary can be
set to automatically restore persistent data from backups, while non-primary
clusters can be configured to automatically replicate from the primary.
Automatic database backup, restore, and streaming replication is an advanced
topic enabled by Cloud Native PG and CUE. Check back for a guide on this and
other Day 2 operations topics.
:::
## Render the Platform {#render-platform}
Render the Platform to render the podinfo Component for each of the workload
clusters.
<Tabs groupId="render-platform">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered components/podinfo for cluster west in 99.480792ms
rendered components/podinfo for cluster east in 99.882667ms
```
</TabItem>
</Tabs>
The render platform command iterates over every Cluster in the Fleet and renders
each Component assigned to the Fleet. Notice the two additional subdirectories
created under the deploy directory, one for each cluster: `east` and `west`.
<Tabs groupId="tree-platform">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
├── default
│   ├── components
│   │   └── podinfo
│   │   └── podinfo.gen.yaml
│   └── gitops
│   └── podinfo.application.gen.yaml
# highlight-next-line
├── east
│   ├── components
│   │   └── podinfo
│   │   └── podinfo.gen.yaml
│   └── gitops
│   └── podinfo.application.gen.yaml
# highlight-next-line
└── west
├── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── gitops
└── podinfo.application.gen.yaml
14 directories, 6 files
```
</TabItem>
</Tabs>
Holos ensures consistency and safety by defining the ArgoCD Application once,
with strong type checking, at the configuration root.
New Application resources are automatically generated for the `east` and `west`
workload Clusters.
<Tabs groupId="applications">
<TabItem value="east" label="east">
`deploy/clusters/east/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/east/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
<TabItem value="west" label="west">
`deploy/clusters/west/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/west/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
<TabItem value="default" label="default">
`deploy/clusters/default/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/default/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
</Tabs>
Add and commit the rendered Platform and workload Clusters.
<Tabs groupId="commit-render-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos render platform ./platform - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 5aebcf5] holos render platform ./platform - 0.93.2
5 files changed, 263 insertions(+)
create mode 100644 clusters.cue
create mode 100644 deploy/clusters/east/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/east/gitops/podinfo.application.gen.yaml
create mode 100644 deploy/clusters/west/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/west/gitops/podinfo.application.gen.yaml
```
</TabItem>
</Tabs>
## Upgrade a Helm Chart
Holos is designed to ease the burden of Day 2 operations. With Holos, upgrading
software, integrating new software, and making safe platform-wide configuration
changes become easier.
Let's upgrade the podinfo Component to see how this works in practice. First,
update the Component version field to the latest upstream Helm chart version.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
holos generate component podinfo --component-version 6.6.2
```
</TabItem>
<TabItem value="output" label="Output">
```txt
generated component
```
</TabItem>
</Tabs>
Remove the cached chart version.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
rm -rf components/podinfo/vendor
```
</TabItem>
</Tabs>
Now re-render the Platform.
<Tabs groupId="render-platform2">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered components/podinfo for cluster east in 327.10475ms
rendered components/podinfo for cluster west in 327.796541ms
```
</TabItem>
</Tabs>
Notice we're still using the upstream chart without modifying it. The Holos
component wraps around the chart to mix in additional resources and integrate
the component with the broader Platform.
## Visualize the Changes
Holos makes it easier to see exactly what changes are made and which resources
will be applied to the API server. By design, Holos operates on local files,
leaving the task of applying them to ecosystem tools like `kubectl` and ArgoCD.
This allows platform operators to inspect changes during code review, or before
committing the change at all.
For example, using `git diff`, we see that the only functional change when
upgrading this Helm chart is the deployment of a new container image tag to each
cluster. Additionally, we can roll out this change gradually by applying it to
the east cluster first, then to the west cluster, limiting the potential blast
radius of a problematic change.
<Tabs groupId="git-diff">
<TabItem value="command" label="Command">
```bash
git diff deploy/clusters/east
```
</TabItem>
<TabItem value="output" label="Output">
```diff showLineNumbers
diff --git a/deploy/clusters/east/components/podinfo/podinfo.gen.yaml b/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
index 7cc3332..8c1647d 100644
--- a/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
+++ b/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
@@ -5,9 +5,9 @@ kind: Service
metadata:
name: podinfo
labels:
- helm.sh/chart: podinfo-6.6.1
+ helm.sh/chart: podinfo-6.6.2
app.kubernetes.io/name: podinfo
- app.kubernetes.io/version: "6.6.1"
+ app.kubernetes.io/version: "6.6.2"
app.kubernetes.io/managed-by: Helm
spec:
type: ClusterIP
@@ -29,9 +29,9 @@ kind: Deployment
metadata:
name: podinfo
labels:
- helm.sh/chart: podinfo-6.6.1
+ helm.sh/chart: podinfo-6.6.2
app.kubernetes.io/name: podinfo
- app.kubernetes.io/version: "6.6.1"
+ app.kubernetes.io/version: "6.6.2"
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
@@ -53,7 +53,7 @@ spec:
terminationGracePeriodSeconds: 30
containers:
- name: podinfo
# highlight-next-line
- image: "ghcr.io/stefanprodan/podinfo:6.6.1"
# highlight-next-line
+ image: "ghcr.io/stefanprodan/podinfo:6.6.2"
imagePullPolicy: IfNotPresent
command:
- ./podinfo
```
</TabItem>
</Tabs>
:::tip
Holos is designed to surface the _fully rendered_ manifests intended for the
Kubernetes API server, making it easier to see and reason about platform-wide
configuration changes.
:::
## Recap {#recap}
In this quickstart guide, we learned how Holos makes it easier, safer, and more
consistent to manage a Platform composed of multiple Clusters and upstream Helm
charts.
We covered how to:
1. Generate a Git repository for the Platform config.
2. Wrap the unmodified upstream podinfo Helm chart into a Component.
3. Render an individual Component.
4. Mix-in your Platform's unique resources to all Components. For example, ArgoCD Application resources.
5. Define multiple similar, but not identical, workload clusters.
6. Render the manifests for the entire Platform with the `holos render platform` command.
7. Upgrade a Helm chart to the latest version as an important Day 2 task.
8. Visualize and surface the details of planned changes Platform wide.
## Dive Deeper
If you'd like to dive deeper, check out the [Schema API][schema] and [Core
API][core] reference docs. The main difference between the schema and core
packages is that the schema is used by users to write refined CUE, while the
core package is what the schema produces for `holos` to execute. Users rarely
need to interact with the Core API when on the happy path, but can use the core
package as an escape hatch when the happy path doesn't go where you want.
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[schema]: /docs/api/author/
[core]: /docs/api/core/

View File

@@ -1,106 +0,0 @@
---
description: Self service platform resource management for project teams.
slug: /archive/guides/2024-09-17-manage-a-project
sidebar_position: 250
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Manage a Project
In this guide we'll explore how Holos easily, safely, and consistently manages
platform resources for teams to develop the projects they're working on.
Intended Audience: Platform Engineers and Software Engineers.
Goal is to demonstrate how the platform team can consistently, easily, and
safely provide platform resources to software engineers.
Assumption is software engineers have a container they want to deploy onto the
platform and make accessible. We'll use httpbin as a stand-in for the dev
team's container.
Project is roughly equivalent to Dev Team for the purpose of this guide, but in
practice multiple teams work on a given project over the lifetime of the
project, so we structure the files into projects instead of teams.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Helm Components.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Kustomize Components.
If you'd like to apply the manifests we render in this guide complete the
following optional, but recommended, steps.
a. Complete the [Local Cluster] guide to set up a local cluster to work with.
b. You'll need a GitHub account to fork the repository associated with this
guide.
## Fork the Guide Repository
<Tabs groupId="fork">
<TabItem value="command" label="Command">
```bash
```
</TabItem>
<TabItem value="output" label="Output">
```txt showLineNumbers
```
</TabItem>
</Tabs>
This guide assumes you will run commands from the root directory of this
repository unless stated otherwise.
[Quickstart]: /docs/quickstart
[Local Cluster]: /docs/guides/local-cluster
## Render the Platform
So we can build the basic platform. Don't dwell on the platform bits.
## Apply the Manifests
Deploy ArgoCD, but not any of the Application resources.
## Browse to ArgoCD
Note there is nothing here yet.
## Switch to your Fork
Note all of the Applications change consistently.
## Apply the Applications
Note how ArgoCD takes over management, no longer need to k apply.
## Create a Project
Project is a conceptual, not technical, thing in Holos. Mainly about how components are laid out in the filesystem tree.
We use a schematic built into holos as an example, the platform team could use the same or provide a similar template and instructions for development teams to self-serve.
## Render the Platform
Notice:
1. Project is registered with the platform at the root.
2. HTTPRoute and Namespace resources are added close to the root in `projects`
3. Deployment and Service resources are added at the leaf in `projects/httpbin/backend`
## Update the image tag
Add a basic schematic to demonstrate this. May need to add two new flags for image url and image tag to the generate subcommand, but should just be two new fields on the struct.
## Dive Deeper
Set the stage for constraints. Ideas: Limit what resources can be added,
namespaces can be operated in, enforce labels, etc...
Simple, consistent, easy constraints.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 934 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 703 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1014 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 728 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1014 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 854 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

View File

@@ -1,6 +0,0 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# ArgoCD
Coming soon.

View File

@@ -1,3 +0,0 @@
# Backstage
Coming soon.

View File

@@ -1,3 +0,0 @@
# Observability
Coming soon.

View File

@@ -1,17 +0,0 @@
# Overview
<!-- https://kubernetes.io/docs/contribute/style/diagram-guide/ -->
This tutorial covers the following process of getting started with Holos.
```mermaid
graph LR
A[1. Install <br>holos] -->
B[2. Register <br>account] -->
C[3. Generate <br>platform] -->
D[4. Render <br>platform] -->
E[5. Apply <br>config]
classDef box fill:#fff,stroke:#000,stroke-width:1px,color:#000;
class A,B,C,D,E box
```

View File

@@ -1,62 +0,0 @@
# Registration
Holos leverages a simple web app to collect and store platform attributes with a web form. Register an account with the web app to create and retrieve the platform model.
```
holos register user
```
:::tip
Holos allows you to customize all of the sections and fields of your platform model.
:::
## Generate your Platform
Generate your platform configuration from the holos reference platform embedded in the `holos` executable. Platform configuration is stored in a git repository.
```bash
mkdir holos-infra
cd holos-infra
holos generate platform holos
```
The generate command writes many files organized by platform component into the current directory
TODO: Put a table here describing key elements?
:::tip
Take a peek at `holos generate platform --help` to see other platforms embedded in the holos executable.
:::
## Push the Platform Form
```
holos push platform form .
```
## Fill in the form
TODO
## Pull the Platform Model
Once the platform model is saved, pull it into the holos-infra repository:
```
holos pull platform model .
```
## Render the Platform
With the platform model and the platform spec, you're ready to render the complete platform configuration:
```
holos render platform ./platform
```
## Summary

Binary file not shown.

Before

Width:  |  Height:  |  Size: 624 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 116 KiB

View File

@@ -1,687 +0,0 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Try Holos Locally
This guide walks through the process of building and managing a software
development platform with Holos. The k3d platform built in this guide is a
slimmed down version of the larger, more holistic, Holos reference platform.
Holos is different from existing tools in a few important ways.
1. Holos provides a **unified configuration model** purpose built to improve on
unmodified Helm charts, Kustomize bases, or anything else that produces
structured configuration data.
2. Holos all but **eliminates the need to template yaml**, a common source of
frustration and errors in production.
3. Holos platforms are **composable** and have breadth. The toolchain and
techniques scale down to one machine and up to multiple clusters across
multiple regions.
4. The unified configuration model is well suited to a **Zero Trust security
model**. Platform wide policy configuration is easier to manage with Holos.
---
This guide assumes commands are run locally. Capitalized terms have specific
definitions described in the [Glossary](/docs/glossary).
## What you'll need {#Requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a k8s api server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
5. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to interact with the k8s api server.
6. [mkcert](https://github.com/FiloSottile/mkcert?tab=readme-ov-file#installation) - to make trusted TLS certificates.
7. [jq](https://jqlang.github.io/jq/download/) - to fiddle with JSON output.
:::note
Registering an account **is recommended** to try out proper authentication and
authorization in Holos, but you can complete this guide without signing up.
:::
## Goal {#Goal}
By the end of this guide you'll have built the foundation of a software
development platform. The foundation provides Zero Trust security by
holistically integrating off-the-shelf open source software.
1. Istio is configured to authenticate and authorize requests using an OIDC
ID-Token issued by ZITADEL before requests reach backend services.
2. The platform provides single sign-on and role based access control for all
services running on the platform.
This guide strives to keep things neat and tidy. All of the resources are
located in one k3d cluster and one local Git repository. If you want to clean
up at any point, do so with:
```bash
k3d cluster delete workload
rm -rf holos-k3d
```
## Sign In or Out {#Sign-In}
Holos provides integrated authentication and authorization which we'll use in
this guide to protect a service. We recommend registering an account to see
this in action. Registration also enables you to explore the customizable web
form that simplifies complex configuration.
If you opt-out, the platform will be configured to use a fake identity in place
of real id tokens.
<Tabs groupId="registration">
<TabItem value="registered" label="Sign In">
```bash
holos register user
```
</TabItem>
<TabItem value="unregistered" label="Opt Out">
```bash
holos logout
```
</TabItem>
</Tabs>
## Create the Platform {#Create-Platform}
A server-side platform resource in Holos stores the web form used to simplify
platform wide configuration.
First, initialize an empty Git repository:
```bash
mkdir holos-k3d
cd holos-k3d
git init
```
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
Use `holos` to make the rpc call to create the server-side platform
resource.
```bash
holos create platform --name k3d --display-name "Try Holos Locally"
```
</TabItem>
<TabItem value="unregistered" label="Signed Out">
Create a blank `platform.metadata.json` file so subsequent holos commands
skip rpc calls.
```bash
touch platform.metadata.json
```
</TabItem>
</Tabs>
### Generate the Platform {#Generate-Platform}
Generate the platform code in the repository root.
```bash
holos generate platform k3d
```
Commit the generated platform config to the repository.
```bash
git add .
git commit -m "holos generate platform k3d - $(holos --version)"
```
### Push the Platform Form
Each Holos platform has a Platform Form used to submit top level, platform-wide
configuration values. The purpose of the form is to validate configuration
values and simplify complicated configurations and integrations.
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
Push the Platform Form to publish it. Browse to the printed URL to view the
form.
```bash
holos push platform form .
```
</TabItem>
<TabItem value="unregistered" label="Signed Out">
You will update the Platform Model locally in a later step so there's
nothing to do in this step. Only signed-in users can push a Platform Form
to the Holos web server.
```bash
# holos push platform form .
```
</TabItem>
</Tabs>
The Platform Form is defined locally in `forms/platform/platform-form.cue`.
On the web it looks like:
![Platform Form Default Values](./form-pushed.png)
### Update the Platform Model {#Platform-Model}
Holos needs initial, top level configuration values to render the platform. The
Platform Model is the term we use for these values. In this section you will
configure role based access control by way of updating the Platform Model.
In the k3d platform you're building now, role based access control is
implemented by asserting against the oidc id token subject. Update the form
with the `sub` claim value from your id token. This will ensure only you have
access to platform services.
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
Copy and paste the `sub` value into your Platform Form's Subject field.
```bash
holos login --print-claims --log-level=error | jq -r .sub
```
After pasting the `sub` value, click Submit on the form.
</TabItem>
<TabItem value="unregistered" label="Signed Out">
You don't have an id token when you're signed out, so there's nothing for
you to do in this step.
```bash
# holos login --print-claims --log-level=error | jq -r .sub
```
The platform will be configured to assert against the User-Agent header
instead.
</TabItem>
</Tabs>
### Pull the Platform Model {#Pull-the-Platform-Model}
The Platform Model needs to be pulled into the local Git repository after the
form has been submitted. Next, we'll run `holos render` which operates
exclusively on local files.
Holos stores the Platform Model in the `platform.config.json` file. Holos
provides this file as input to CUE when rendering the platform. This file is
intended to be added to version control.
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
Pull the updated Platform Model into the local repository.
```bash
holos pull platform model .
git add platform.config.json
git commit -m "Add platform model"
```
</TabItem>
<TabItem value="unregistered" label="Signed Out">
The holos generate platform k3d command created an initial Platform Model in
`platform.config.json`. As a result there's nothing to do in this step.
```bash
# holos pull platform model .
# git add platform.config.json
# git commit -m "Add platform model"
```
</TabItem>
</Tabs>
## Render the Platform {#Render-the-Platform}
Holos has everything necessary to render the platform once the
`platform.config.json` file and the code from `holos generate` are in the
current directory.
Rendering a platform is the process of iterating over each platform component
and rendering it into plain yaml. Holos does not apply the resulting manifests.
Other tools like kubectl, ArgoCD, or Flux are responsible for applying the
manifests.
```bash
holos render platform ./platform
```
The render command writes the manifest files to the `deploy/` directory. Commit
the files so they can be applied via GitOps later.
```bash
git add deploy
git commit -m "holos render platform ./platform"
```
:::info[Don't blink, this is where Holos builds the platform]
It usually takes no more than a few seconds.
Rendering the holos reference platform currently results in about 500K lines of
yaml. In contrast, roughly 80K lines are produced by this slimmed down k3d
platform.
We mention this because the scale doesn't matter as much as it does with other
tools. Manage millions of lines of configuration with Holos the same way this
guide manages thousands. This is made possible by the unique way CUE unifies
all configuration into one single model.
:::
## Configure DNS {#DNS}
Configure your machine to resolve `*.holos.localhost` to your loopback
interface. This is necessary for requests to reach the workload cluster.
<Tabs>
<TabItem value="macos" label="macOS" default>
Cache sudo credentials.
Admin access is necessary to setup a local dnsmasq instance and configure
macOS's DNS resolver.
```bash
sudo -v
```
Resolve *.holos.localhost DNS queries to 127.0.0.1.
```bash
bash ./scripts/local-dns
```
</TabItem>
<TabItem value="linux" label="Linux">
[NSS-myhostname](http://man7.org/linux/man-pages/man8/nss-myhostname.8.html)
ships with many Linux distributions and should resolve *.localhost
automatically to 127.0.0.1.
Otherwise it is installable with:
```bash
sudo apt install libnss-myhostname
```
</TabItem>
<TabItem value="windows" label="Windows">
Ensure the loopback interface has at least the following names in `C:\windows\system32\drivers\etc\hosts`
```
127.0.0.1 httpbin.holos.localhost app.holos.localhost
```
</TabItem>
</Tabs>
## Create the Cluster {#Create-Cluster}
The Workload Cluster is where your applications and services will be deployed.
In production this is usually an EKS, GKE, or AKS cluster.
:::tip
Holos supports all compliant Kubernetes clusters. Holos was developed and tested
on GKE, EKS, Talos, k3s, and Kubeadm clusters.
:::
<Tabs>
<TabItem value="evaluate" label="Try Holos" default>
Use this command when exploring Holos.
```bash
k3d cluster create workload \
--port "443:443@loadbalancer" \
--k3s-arg "--disable=traefik@server:0"
```
</TabItem>
<TabItem value="develop" label="Develop Holos">
Use this command when developing Holos.
```bash
k3d registry create registry.holos.localhost --port 5100
```
```bash
k3d cluster create workload \
--registry-use k3d-registry.holos.localhost:5100 \
--port "443:443@loadbalancer" \
--k3s-arg "--disable=traefik@server:0"
```
</TabItem>
</Tabs>
Traefik is disabled because Istio provides the same functionality.
## Apply the Platform Components {#Apply-Platform-Components}
Use `kubectl` to apply each platform component. In production, it's common to
fully automate this process with ArgoCD, but we use `kubectl` to the same
effect.
### Local CA {#Local-CA}
Holos platforms use cert manager to issue tls certificates. The browser and
tools we're using need to trust these certificates to work together.
Admin access is necessary for `mkcert` to manage the certificate into your trust
stores.
```bash
sudo -v
```
Manage the local CA and copy the CA key to the workload cluster so that cert
manager can manage trusted certificates.
```bash
bash ./scripts/local-ca
```
:::warning
Take care to run the local-ca script each time you create the workload cluster
so that Certificates are issued correctly.
:::
### Service Mesh
The platform service mesh provides an ingress gateway and connectivity useful
for observability, reliability, and security.
#### Namespaces
With Holos, components are automatically added to the namespaces component,
useful for centrally managed policies.
```bash
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/namespaces
```
#### Custom Resource Definitions
```bash
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/gateway-api
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/istio-base
```
#### Cert Manager {#cert-manager}
Apply the cert-manager controller.
```bash
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/cert-manager
```
Apply the ClusterIssuer which issues Certificate resources using the local
certificate authority.
```bash
kubectl -n cert-manager wait pod -l app.kubernetes.io/component=webhook --for=condition=Ready
kubectl apply --server-side=true -f deploy/clusters/workload/components/local-ca
kubectl apply --server-side=true -f deploy/clusters/workload/components/certificates
kubectl -n istio-gateways wait certificate httpbin.holos.localhost --for=condition=Ready
```
:::warning
The certificate will time out before becoming ready if the [local-ca](#Local-CA)
script was not run after the cluster was created.
:::
#### Istio {#Istio}
Istio implements the Service Mesh.
```bash
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/istio-cni
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/istiod
kubectl apply --server-side=true -f ./deploy/clusters/workload/components/gateway
```
Verify the Gateway is programmed and the listeners have been accepted:
```bash
kubectl -n istio-gateways wait gateway default --for=condition=Accepted
```
#### httpbin {#httpbin}
httpbin is a simple backend service useful for end-to-end testing.
```bash
kubectl apply --server-side=true -f deploy/clusters/workload/components/httpbin-backend
kubectl apply --server-side=true -f deploy/clusters/workload/components/httpbin-routes
kubectl -n holos-system wait pod -l app.kubernetes.io/instance=httpbin --for=condition=Ready
```
:::info
Browse to [https://httpbin.holos.localhost/](https://httpbin.holos.localhost/)
to verify end to end connectivity. You should see the httpbin index page.
:::
### Authenticating Proxy
The auth proxy is responsible for authenticating browser requests, handling the
oidc authentication flow, and providing a signed id token to the rest of the
services in the mesh.
#### Cookie Secret
The auth proxy stores session information in an encrypted cookie. Generate a
random cookie encryption Secret and apply.
```bash
LC_ALL=C tr -dc A-Za-z0-9 </dev/urandom \
| head -c 32 \
| kubectl create secret generic "authproxy" \
--from-file=cookiesecret=/dev/stdin \
--dry-run=client -o yaml \
| kubectl apply -n istio-gateways -f-
```
#### Deployment
The auth proxy Deployment receives requests from web browsers and responds with
an authentication decision.
```bash
kubectl apply --server-side=true -f deploy/clusters/workload/components/authproxy
kubectl apply --server-side=true -f deploy/clusters/workload/components/authroutes
```
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
<Admonition type="info">
Verify authentication is working by browsing to
[https://httpbin.holos.localhost/holos/authproxy](https://httpbin.holos.localhost/holos/authproxy).
We want a simple `Authenticated` response.
<Admonition type="tip">
You may need to refresh the page a few times while the platform configures
itself.
</Admonition>
</Admonition>
Istio will respond with `no healthy upstream` until the pod becomes ready.
Wait for the pod to become ready with:
```bash
kubectl -n holos-system wait pod -l app.kubernetes.io/instance=httpbin --for=condition=Ready
```
Once authenticated, visit
[https://httpbin.holos.localhost/holos/authproxy/userinfo](https://httpbin.holos.localhost/holos/authproxy/userinfo)
which returns a subset of claims from your id token.
<Admonition type="warning">
If you get `Unauthorized` instead of a json response body, make sure you
[authenticated](https://httpbin.holos.localhost/holos/authproxy) first.
</Admonition>
```json
{
"user": "275552236589843464",
"email": "demo@holos.run",
"preferredUsername": "demo"
}
```
</TabItem>
<TabItem value="unregistered" label="Signed Out">
The auth proxy will always try to sign you in when you are signed out, so
there isn't much to do here. Please do take a moment to glance at the
Signed In tab to see how this would work if you were signed in.
The `k3d` platform relies on `https://login.holos.run` to issue id tokens.
Authorization has been configured against fake request headers instead of
the real `x-oidc-id-token` header.
</TabItem>
</Tabs>
### Authorization Policy
Configure authorization policies using attributes of the authenticated request.
Authorization policies route web requests through the auth proxy and then
validate all requests against the `x-oidc-id-token` header.
```bash
kubectl apply --server-side=true -f deploy/clusters/workload/components/authpolicy
```
Istio make take a few seconds to program the Gateway with the
AuthorizationPolicy resources.
## Try out Zero Trust
A basic Zero Trust security model is now in place. The platform authenticates
and authorizes requests before they reach the backend service.
### Browser
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
The platform has been configured to authorize requests with a `x-oidc-id-token` header.
1. Verify authentication is working by browsing to [https://httpbin.holos.localhost/dump/request](https://httpbin.holos.localhost/dump/request).
- Refresh the page a few times.
- The `httpbin` backend pods should echo back the `x-oidc-id-token`
header injected by the auth proxy.
2. Note the `x-oidc-id-token` header is not sent by your browser but is
received by the backend service.
- This design reduces the risk of exposing id tokens in the browser.
- Browser request size remains constant as more claims are added to id
tokens.
- Reliability improves because id tokens often overflow request header
buffers when they pass through middle boxes across the internet.
</TabItem>
<TabItem value="unregistered" label="Signed Out">
The platform has been configured to authorize requests with a `User-Agent: anonymous` header.
1. Open an incognito window (Cmd+Shift+N) to verify the platform is
enforcing the authorization policy.
2. Browse to
[https://httpbin.holos.localhost/dump/request](https://httpbin.holos.localhost/dump/request)
you should be redirected to the sign in page by the auth proxy.
- You **do not** need to register or sign in.
- This step verifies the platform is redirecting unauthenticated
requests to the identity provider.
- Navigate back or close and re-open an incognito window.
3. Set your `User-Agent` header to `anonymous` using your browser developer tools.
- For Chrome the process is described
[here](https://developer.chrome.com/docs/devtools/device-mode/override-user-agent#override_the_user_agent_string).
- The purpose is to simulate an authenticated request.
4. Browse to
[https://httpbin.holos.localhost/dump/request](https://httpbin.holos.localhost/dump/request).
- The platform should allow the request through to the backend pod.
- `httpbin` should echo back your request which should contain `User-Agent: anonymous`.
</TabItem>
</Tabs>
### Command Line
Verify unauthenticated requests are blocked by default outside the browser.
```bash
curl -I https://httpbin.holos.localhost/dump/request
```
You should receive a `HTTP/2 302` response that redirects to `location:
https://login.holos.run` to start the oauth login flow.
Next, verify authenticated requests are allowed.
<Tabs groupId="registration">
<TabItem value="registered" label="Signed In">
The platform is configured to authenticate the id token present in the
`x-oidc-id-token` header.
💡 It also works with `grpcurl`.
```bash
curl -H x-oidc-id-token:$(holos token) https://httpbin.holos.localhost/dump/request
```
</TabItem>
<TabItem value="unregistered" label="Signed Out">
The platform is configured to authorize any request with `User-Agent:
anonymous` in place of validating the oidc id token.
💡 Take a moment to click the Signed In tab, I don't want you to miss how
cool `$(holos token)` is.
```bash
curl -A anonymous https://httpbin.holos.localhost/dump/request
```
</TabItem>
</Tabs>
You should receive a response showing the request headers the backend received.
:::tip
Note how the platform secures both web browser and command line api access to
the backend httpbin service. httpbin itself has no authentication or
authorization functionality.
:::
## Summary
Thank you for taking the time to try out Holos. In this guide, you built the
foundation of a software development platform that:
1. Provides a unified configuration model with CUE that
- Supports unmodified Helm Charts, Kustomize Kustomizations, plain YAML.
- Provides a web form to pass top level parameters.
2. Reduces errors by eliminating the need to template unstructured text.
3. Is composable and scales down to a local machine.
4. Provides an way to safely configure broad authentication and authorization
policy.
## Next Steps
Dive deeper with the following resources that build on the foundation you have now.
1. Explore the [Rendering Process](/docs/concepts#rendering) in Holos.
2. Dive deeper into the [Platform Manifests](./platform-manifests) rendered in this guide.
3. Deploy [ArgoCD](../argocd) onto the foundation you built.
4. Deploy [Backstage](../backstage) as a portal to the integrated platform components.
## Clean-Up
If you'd like to clean up the resources you created in this guide, remove them
with:
```bash
k3d cluster delete workload
rm -rf holos-k3d
```

View File

@@ -1,137 +0,0 @@
# Platform Manifests
This document provides an example of how Holos uses CUE and Helm to unify and
render the platform configuration. It refers to the manifests rendered in the
Try Holos Locally guide.
Take a moment to review the manifests `holos` rendered to build the platform.
### ArgoCD Application
Note the Git URL in the Platform Model is used to derive the ArgoCD
`Application` resource for all of the platform components.
```yaml
# deploy/clusters/workload/gitops/namespaces.application.gen.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: namespaces
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: /deploy/clusters/workload/components/namespaces
# highlight-next-line
repoURL: https://github.com/holos-run/holos-k3d.git
# highlight-next-line
targetRevision: HEAD
```
One ArgoCD `Application` resource is produced for each Holos component by
default. The CUE definition which produces the rendered output is defined in
`buildplan.cue` around line 222.
:::tip
Note how CUE does not use error-prone text templates, the language is well
specified and typed which reduces errors when unifying the configuration with
the Platform Model in the following `#Argo` definition.
:::
```cue
// buildplan.cue
// #Argo represents an argocd Application resource for each component, written
// using the #HolosComponent.deployFiles field.
#Argo: {
ComponentName: string
Application: app.#Application & {
metadata: name: ComponentName
metadata: namespace: "argocd"
spec: {
destination: server: "https://kubernetes.default.svc"
project: "default"
source: {
// highlight-next-line
path: "\(_Platform.Model.argocd.deployRoot)/deploy/clusters/\(_ClusterName)/components/\(ComponentName)"
// highlight-next-line
repoURL: _Platform.Model.argocd.repoURL
// highlight-next-line
targetRevision: _Platform.Model.argocd.targetRevision
}
}
}
// deployFiles represents the output files to write along side the component.
deployFiles: "clusters/\(_ClusterName)/gitops/\(ComponentName).application.gen.yaml": yaml.Marshal(Application)
}
```
### Helm Chart
The `cert-manger` component renders using the upstream Helm chart. The build
plan that defines the helm chart to use along with the values to provide looks
like the following.
:::tip
Holos fully supports your existing Helm charts. Consider leveraging `holos` as
an alternative to umbrella charts.
:::
```cue
// components/cert-manager/cert-manager.cue
package holos
// Produce a helm chart build plan.
(#Helm & Chart).Output
let Chart = {
Name: "cert-manager"
Version: "1.14.5"
Namespace: "cert-manager"
Repo: name: "jetstack"
Repo: url: "https://charts.jetstack.io"
// highlight-next-line
Values: {
installCRDs: true
startupapicheck: enabled: false
// Must not use kube-system on gke autopilot. GKE Warden blocks access.
// highlight-next-line
global: leaderElection: namespace: Namespace
// https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-resource-requests#min-max-requests
resources: requests: {
cpu: "250m"
memory: "512Mi"
"ephemeral-storage": "100Mi"
}
// highlight-next-line
webhook: resources: Values.resources
// highlight-next-line
cainjector: resources: Values.resources
// highlight-next-line
startupapicheck: resource: Values.resources
// https://cloud.google.com/kubernetes-engine/docs/how-to/autopilot-spot-pods
nodeSelector: {
"kubernetes.io/os": "linux"
if _ClusterName == "management" {
"cloud.google.com/gke-spot": "true"
}
}
webhook: nodeSelector: Values.nodeSelector
cainjector: nodeSelector: Values.nodeSelector
startupapicheck: nodeSelector: Values.nodeSelector
}
}
```

View File

@@ -1,28 +0,0 @@
# Local Development
This document captures notes on locally developing Holos.
Follow the steps in [Try Holos Locally](../guides/try-holos), but take care
to select `Develop` tabs when creating the k3d cluster so you have a local
registry to push to.
## Apply Resources
Work will be done in the `dev-holos` namespace.
Apply the infrastructure, which should persist when tilt is started / stopped.
```bash
kubectl apply --server-side=true -f ./hack/tilt/k8s/dev-holos-infra
```
This creates the PostgresCluster, service account, etc...
## Start tilt
Tilt will build the go executable, build the container, then push it to the
local repository associated with k3d.
```bash
./hack/tilt/bin/tilt up
```

View File

@@ -23,7 +23,7 @@ deployed services safely and consistently.
<DocCardList />
[Quickstart]: /docs/quickstart/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Change a Service]: /docs/guides/change-a-service/
[Technical Overview]: /docs/technical-overview/
[Quickstart]: ./guides/quickstart.mdx
[Deploy a Service]: ./guides/deploy-a-service.mdx
[Change a Service]: ./guides/change-a-service.mdx
[Technical Overview]: ./technical-overview.md

View File

@@ -18,12 +18,11 @@ This guide builds on the concepts covered in the [Quickstart] and [Deploy a Serv
Like our other guides, this guide is intended to be useful without needing to
run each command. If you'd like to apply the manifests to a real Cluster,
complete the [Local Cluster Guide](/docs/guides/local-cluster) before this
guide.
complete the [Local Cluster Guide] before this guide.
You'll need the following tools installed to run the commands in this guide.
1. [holos](/docs/install) - to build the Platform.
1. [holos] - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap Helm charts.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Holos
@@ -706,14 +705,17 @@ Thanks for taking the time to work through this guide which covered:
value is defined, making it faster and easier to troubleshoot problems.
[Quickstart]: /docs/quickstart/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Change a Service]: /docs/guides/change-a-service/
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize
[ComponentFields]: /docs/api/author/v1alpha3/#ComponentFields
[platform-files]: /docs/quickstart/#how-platform-rendering-works
[Quickstart]: ./quickstart.mdx
[Local Cluster Guide]: ./local-cluster.mdx
[platform-files]: ./quickstart.mdx#how-platform-rendering-works
[Deploy a Service]: ./deploy-a-service.mdx
[Change a Service]: ./change-a-service.mdx
[Helm]: ../api/author.md#Helm
[Kubernetes]: ../api/author.md#Kubernetes
[Kustomize]: ../api/author.md#Kustomize
[ComponentFields]: ../api/author.md#ComponentFields
[holos]: ../start/install.md
[AppProject]: https://argo-cd.readthedocs.io/en/stable/user-guide/projects/
[unification operator]: https://cuelang.org/docs/reference/spec/#unification
[code-owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
@@ -727,7 +729,6 @@ value is defined, making it faster and easier to troubleshoot problems.
[comprehension]: https://cuelang.org/docs/reference/spec/#comprehensions
[code owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[ReferenceGrant]: https://gateway-api.sigs.k8s.io/api-types/referencegrant/
[Local Cluster Guide]: /docs/guides/local-cluster
[Bank of Holos]: https://github.com/holos-run/bank-of-holos
[default value]: https://cuelang.org/docs/tour/types/defaults/
[constrain]: https://cuelang.org/docs/tour/basics/constraints/

View File

@@ -30,8 +30,7 @@ platform team.
Like our other guides, this guide is intended to be useful without needing to
run the commands. If you'd like to render the platform and apply the manifests
to a real Cluster, complete the [Local Cluster
Guide](/docs/guides/local-cluster) before this guide.
to a real Cluster, complete the [Local Cluster Guide] before this guide.
:::important
This guide relies on the concepts we covered in the [Quickstart] guide.
@@ -39,7 +38,7 @@ This guide relies on the concepts we covered in the [Quickstart] guide.
You'll need the following tools installed to run the commands in this guide.
1. [holos](/docs/install) - to build the Platform.
1. [holos] - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap Helm charts.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Holos
@@ -469,8 +468,7 @@ for Cluster in _Fleets.workload.clusters {
:::tip
The behavior of files in the `platform/` directory is covered in detail in the
[how platform rendering works](/docs/quickstart/#how-platform-rendering-works)
section of the Quickstart guide.
[how platform rendering works] section of the Quickstart guide.
:::
Before we render the platform, we want to make sure our podinfo component, and
@@ -1451,6 +1449,7 @@ configuration change a team needs to roll out after a service has been deployed
for some time.
[Quickstart]: ./quickstart.mdx
[how platform rendering works]: ./quickstart.mdx#how-platform-rendering-works
[Change a Service]: ./change-a-service.mdx
[Helm]: ../api/author.md#Helm
[Kubernetes]: ../api/author.md#Kubernetes
@@ -1472,3 +1471,4 @@ for some time.
[code owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[ReferenceGrant]: https://gateway-api.sigs.k8s.io/api-types/referencegrant/
[Bank of Holos]: https://github.com/holos-run/bank-of-holos
[holos]: ../start/install.md

View File

@@ -4852,8 +4852,8 @@ Overview].
[prometheus]: https://github.com/prometheus-community/helm-charts/tree/prometheus-25.27.0/charts/prometheus
[blackbox]: https://github.com/prometheus-community/helm-charts/tree/prometheus-blackbox-exporter-9.0.1/charts/prometheus-blackbox-exporter
[httpbin]: https://github.com/mccutchen/go-httpbin/tree/v2.15.0
[Local Cluster]: /docs/guides/local-cluster/
[guides]: /docs/guides/
[Technical Overview]: /docs/technical-overview/
[Installation]: /docs/install/
[Local Cluster]: ./local-cluster.mdx
[guides]: ../guides.md
[Technical Overview]: ../technical-overview.md
[Installation]: ../start/install.md
[Config Schema]: #config-schema

View File

@@ -16,8 +16,7 @@ have a standard Kubernetes API server with proper DNS and TLS certificates.
You'll be able to easily reset the cluster to a known good state to iterate on
your own Platform.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
The [Concepts] page defines capitalized terms such as Platform and Component.
## Reset the Cluster
@@ -105,7 +104,7 @@ You're back to the same state as the first time you completed this guide.
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
1. [holos] - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a k8s api server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
@@ -274,4 +273,8 @@ k3d cluster delete workload
## Next Steps
Now that you have a real cluster, apply and explore the manifests Holos renders
in the [Quickstart](/docs/quickstart) guide.
in the [Quickstart] guide.
[Concepts]: ../start/concepts.md
[Quickstart]: ./quickstart.mdx
[holos]: ../start/install.md

View File

@@ -47,12 +47,11 @@ of Holos.
This guide is intended to be informative without needing to run the commands.
If you'd like to render the platform and apply the manifests to a real Cluster,
complete the [Local Cluster Guide](/docs/guides/local-cluster) before this
guide.
complete the [Local Cluster Guide] before this guide.
You'll need the following tools installed to run the commands in this guide.
1. [holos](/docs/install) - to build the Platform.
1. [holos][Installation] - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap Helm charts.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Holos
@@ -61,8 +60,7 @@ Components that render with Kustomize.
## Install Holos
Start by installing the `holos` command line tool with the following command.
If you don't have Go, refer to [Installation](/docs/install/) to download the
executable.
If you don't have Go, refer to [Installation] to download the executable.
<Tabs groupId="go-install">
<TabItem value="command" label="Command">
@@ -1121,11 +1119,12 @@ Thank you for finishing the Quickstart guide. Dive deeper with the next guide
on how to [Deploy a Service] which explains how to take one of your existing
Helm charts or Deployments and manage it with Holos.
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[core]: /docs/api/core/v1alpha4/
[Core API]: /docs/api/core/
[Author API]: /docs/api/author/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Manage a Project]: /docs/guides/manage-a-project/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern/
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[core]: ../api/core.md
[Core API]: ../api/core.md
[Author API]: ../api/author.md
[Deploy a Service]: ./deploy-a-service.mdx
[Local Cluster Guide]: ./local-cluster.mdx
[Installation]: ../start/install.md
[^1]: [The Basics of CUE](https://cuelang.org/docs/tour/basics/json-superset/)

View File

@@ -43,11 +43,11 @@ I rewrote our scripts and charts using CUE and Go, replacing the glue layer. The
Thanks for reading. Take Holos for a spin on your local machine with our [Quickstart] guide.
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[Guides]: ./guides.md
[API Reference]: ./api.md
[Quickstart]: ./guides/quickstart.mdx
[CUE]: https://cuelang.org/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Author API]: ./api/author.md
[Core API]: ./api/core.md
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story

View File

@@ -10,4 +10,4 @@ These documents provide additional context to supplement the [Quickstart] guide.
<DocCardList />
[Quickstart]: /docs/quickstart/
[Quickstart]: ./guides/quickstart.mdx

View File

@@ -9,8 +9,7 @@ sidebar_position: 200
## Introduction
This page is intended as a high level conceptual overview of the key concepts in
Holos. Refer to the [Core API](/docs/api/core/) for low level reference
documentation.
Holos. Refer to the [Core API] for low level reference documentation.
Holos is a tool built for platform engineers. The Holos authors share three
core values which guide our design decisions for the tool.
@@ -368,3 +367,4 @@ typically has two Fleets, one for management and one for workloads.
[krm]: https://docs.google.com/document/d/1RmHXdLhNbyOWPW_AtnnowaRfGejw-qlKQIuLKQWlwzs/view#heading=h.sa6p0aye4ide
[Platform]: ../api/core.md#Platform
[BuildPlan]: ../api/core.md#BuildPlan
[Core API]: ../api/core.md

15
hack/claude Normal file
View File

@@ -0,0 +1,15 @@
#! /bin/bash
TOPLEVEL="$(cd $(dirname "$0") && git rev-parse --show-toplevel)"
cd "${TOPLEVEL}"
# The text files use about 85% of the knowledge base.
mkdir -p tmp/claude
for x in $(git ls-files internal/\*.go api/author/v1alpha5/\*.go api/core/v1alpha5/\*.go doc/md/\*.md{,x}); do
y="${x//\//__}.txt"
[[ $y =~ "__ent__" ]] && continue
cp "$x" "tmp/claude/$y"
done

108
holos.go
View File

@@ -1,18 +1,6 @@
// Package holos defines types for the rest of the system.
package holos
import (
"bytes"
"context"
"encoding/json"
"fmt"
"log/slog"
"strings"
"cuelang.org/go/cue"
"github.com/holos-run/holos/internal/errors"
)
// A PathCueMod is a string representing the absolute filesystem path of a cue
// module. It is given a unique type so the API is clear.
type PathCueMod string
@@ -27,99 +15,3 @@ type FilePath string
// FileContent represents the contents of a file as a string.
type FileContent string
// TypeMeta represents the kind and version of a resource holos needs to
// process. Useful to discriminate generated resources.
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
// Builder builds file artifacts.
type Builder interface {
Build(context.Context, ArtifactMap) error
}
// ArtifactMap sets and gets data for file artifacts.
//
// Concrete values must ensure Set is write once, returning an error if a given
// FilePath was previously Set. Concrete values must be safe for concurrent
// reads and writes.
type ArtifactMap interface {
Get(path string) (data []byte, ok bool)
Set(path string, data []byte) error
Save(dir, path string) error
}
// Discriminator is useful to discriminate by type meta, the kind and api
// version of something.
type Discriminator interface {
Discriminate(ctx context.Context) (TypeMeta, error)
}
type Unifier interface {
Unify(ctx context.Context) (BuildData, error)
}
// BuildData represents the data necessary to produce a build plan. It is a
// convenience wrapper to store relevant fields to inform the user.
type BuildData struct {
Value cue.Value
ModuleRoot string
InstancePath InstancePath
Dir string
}
func (bd *BuildData) TypeMeta() (tm TypeMeta, err error) {
v, err := bd.value()
if err != nil {
return tm, errors.Wrap(err)
}
kind := v.LookupPath(cue.ParsePath("kind"))
if err := kind.Err(); err != nil {
return tm, errors.Wrap(err)
}
if tm.Kind, err = kind.String(); err != nil {
return tm, errors.Wrap(err)
}
version := v.LookupPath(cue.ParsePath("apiVersion"))
if err := version.Err(); err != nil {
return tm, errors.Wrap(err)
}
if tm.APIVersion, err = version.String(); err != nil {
return tm, errors.Wrap(err)
}
return
}
func (bd *BuildData) Decoder() (*json.Decoder, error) {
v, err := bd.value()
if err != nil {
return nil, errors.Wrap(err)
}
jsonBytes, err := v.MarshalJSON()
if err != nil {
return nil, errors.Wrap(err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
return decoder, nil
}
func (bd *BuildData) value() (v cue.Value, err error) {
v = bd.Value.LookupPath(cue.ParsePath("holos"))
if err := v.Err(); err != nil {
if strings.HasPrefix(err.Error(), "field not found") {
slog.Warn(fmt.Sprintf("%s: deprecated usage: nest output under holos: %s", err, bd.Dir), "err", err)
v = bd.Value
return v, nil
}
err = errors.Wrap(err)
return v, err
}
return
}

View File

@@ -9,21 +9,35 @@ import (
"github.com/holos-run/holos/internal/errors"
)
func New() *Artifact {
return &Artifact{m: make(map[string][]byte)}
// NewStore should provide a concrete Store.
var _ Store = NewStore()
// Store sets and gets data for file artifacts.
//
// Concrete values must ensure Set is write once, returning an error if a given
// FilePath was previously Set. Concrete values must be safe for concurrent
// reads and writes. Use [NewStore] to create a new concrete value.
type Store interface {
Get(path string) (data []byte, ok bool)
Set(path string, data []byte) error
Save(dir, path string) error
}
// Artifact represents the fully rendered manifests build from the holos
func NewStore() *MapStore {
return &MapStore{m: make(map[string][]byte)}
}
// MapStore represents the fully rendered manifests build from the holos
// rendering pipeline. Files are organized by keys representing paths relative
// to the current working directory. Values represent the file content.
type Artifact struct {
type MapStore struct {
mu sync.RWMutex
m map[string][]byte
}
// Set sets an artifact file with write locking. Set returns an error if the
// artifact was previously set.
func (a *Artifact) Set(path string, data []byte) error {
func (a *MapStore) Set(path string, data []byte) error {
a.mu.Lock()
defer a.mu.Unlock()
if _, ok := a.m[path]; ok {
@@ -34,7 +48,7 @@ func (a *Artifact) Set(path string, data []byte) error {
}
// Get gets the content of an artifact with read locking.
func (a *Artifact) Get(path string) (data []byte, ok bool) {
func (a *MapStore) Get(path string) (data []byte, ok bool) {
a.mu.RLock()
defer a.mu.RUnlock()
data, ok = a.m[path]
@@ -42,7 +56,7 @@ func (a *Artifact) Get(path string) (data []byte, ok bool) {
}
// Save writes a file to the filesystem.
func (a *Artifact) Save(dir, path string) error {
func (a *MapStore) Save(dir, path string) error {
fullPath := filepath.Join(dir, path)
msg := fmt.Sprintf("could not save %s", fullPath)
data, ok := a.Get(path)
@@ -58,7 +72,7 @@ func (a *Artifact) Save(dir, path string) error {
return nil
}
func (a *Artifact) Keys() []string {
func (a *MapStore) Keys() []string {
a.mu.RLock()
defer a.mu.RUnlock()
keys := make([]string, 0, len(a.m))

View File

@@ -1,504 +0,0 @@
// Package builder is responsible for building fully rendered kubernetes api
// objects from various input directories. A directory may contain a platform
// spec or a component spec.
package builder
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
"cuelang.org/go/cue/load"
"github.com/holos-run/holos"
core_v1alpha2 "github.com/holos-run/holos/api/core/v1alpha2"
core_v1alpha3 "github.com/holos-run/holos/api/core/v1alpha3"
meta_v1alpha2 "github.com/holos-run/holos/api/meta/v1alpha2"
"github.com/holos-run/holos/api/v1alpha1"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/render"
)
const (
KubernetesObjects = core_v1alpha3.KubernetesObjectsKind
// Helm is the value of the kind field of holos build output indicating helm
// values and helm command information.
Helm = core_v1alpha3.HelmChartKind
// Skip is the value when the instance should be skipped
Skip = "Skip"
// KustomizeBuild is the value of the kind field of cue output indicating
// holos should process the component using kustomize build to render output.
KustomizeBuild = v1alpha1.KustomizeBuildKind
)
// An Option configures a Builder
type Option func(*config)
type config struct {
args []string
cluster string
tags []string
}
type Builder struct {
cfg config
ctx *cue.Context
}
type buildPlanWrapper struct {
buildPlan *core_v1alpha3.BuildPlan
}
func (b *buildPlanWrapper) validate() error {
if b == nil {
return fmt.Errorf("invalid BuildPlan: is nil")
}
bp := b.buildPlan
if bp == nil {
return fmt.Errorf("invalid BuildPlan: is nil")
}
errs := make([]string, 0, 2)
if bp.Kind != core_v1alpha3.BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", v1alpha1.BuildPlanKind, bp.Kind))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (b *buildPlanWrapper) resultCapacity() (count int) {
if b == nil {
return 0
}
bp := b.buildPlan
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}
// New returns a new *Builder configured by opts Option.
func New(opts ...Option) *Builder {
var cfg config
for _, f := range opts {
f(&cfg)
}
b := &Builder{
cfg: cfg,
ctx: cuecontext.New(),
}
return b
}
// Entrypoints configures the leaf directories Builder builds.
func Entrypoints(args []string) Option {
return func(cfg *config) { cfg.args = args }
}
// Cluster configures the cluster name for the holos component instance.
func Cluster(name string) Option {
return func(cfg *config) { cfg.cluster = name }
}
// Tags configures tags to pass to cue when building the instance.
func Tags(tags []string) Option {
return func(cfg *config) { cfg.tags = tags }
}
// Cluster returns the cluster name of the component instance being built.
func (b *Builder) Cluster() string {
return b.cfg.cluster
}
func (b *Builder) Discriminate(ctx context.Context) (tm holos.TypeMeta, err error) {
cueModDir, err := b.findCueMod()
if err != nil {
err = errors.Wrap(err)
return
}
cueConfig := load.Config{
Dir: string(cueModDir),
ModuleRoot: string(cueModDir),
}
bd := &holos.BuildData{ModuleRoot: string(cueModDir)}
if len(b.cfg.args) > 1 {
return tm, errors.Wrap(errors.New("cannot provide more than one argument"))
}
// Make args relative to the module directory
args := make([]string, 0, len(b.cfg.args)+2)
for _, path := range b.cfg.args {
target, err := filepath.Abs(path)
if err != nil {
return tm, errors.Wrap(fmt.Errorf("could not find absolute path: %w", err))
}
relPath, err := filepath.Rel(bd.ModuleRoot, target)
if err != nil {
return tm, errors.Wrap(fmt.Errorf("invalid argument, must be relative to cue.mod: %w", err))
}
bd.InstancePath = holos.InstancePath(target)
bd.Dir = relPath
relPath = "./" + relPath
args = append(args, relPath)
}
instances := load.Instances(args, &cueConfig)
values, err := b.ctx.BuildInstances(instances)
if err != nil {
return tm, errors.Wrap(err)
}
bd.Value = values[0]
tm, err = bd.TypeMeta()
return
}
// Unify returns a cue.Value representing the kind of build holos is meant to
// execute. This function unifies a cue package entrypoint with
// platform.config.json and user data json files located recursively within the
// userdata directory at the cue module root.
//
// Deprecated: use Discriminate instead.
func (b *Builder) Unify(ctx context.Context, cfg *client.Config) (bd holos.BuildData, err error) {
// Ensure the value is from the same runtime, otherwise cue panics.
bd.Value = b.ctx.CompileString("")
cueModDir, err := b.findCueMod()
if err != nil {
err = errors.Wrap(err)
return
}
bd.ModuleRoot = string(cueModDir)
platformConfigData, err := os.ReadFile(filepath.Join(bd.ModuleRoot, client.PlatformConfigFile))
if err != nil {
return bd, errors.Wrap(fmt.Errorf("could not load platform model: %w", err))
}
// TODO(jeff): Changing these tag names breaks backwards compatibility. We
// need to refactor this unification into a versioned builder, at least at the
// component level. Right now it's executed when rendering the initial
// Platform spec, which should be backwards compatible but isn't because this
// package is shared by all versions.
tags := make([]string, 0, len(b.cfg.tags)+2)
// TODO: Use instance.FillPath to fill the platform config.
// Refer to https://pkg.go.dev/cuelang.org/go/cue#Value.FillPath
tags = append(tags, "holos_platform_config="+string(platformConfigData))
// TODO(jeff): This is hacky after I switched to reserved holos_ tag names in
// v1alpha4. Could use some serious clean up now that --cluster-name is
// deprecated for --inject holos_cluster=foo, but it was kind of nice to have
// a required argument.
if cluster := cfg.Holos().ClusterName(); cluster != "" {
tags = append(tags, "holos_cluster="+cluster)
}
tags = append(tags, b.cfg.tags...)
cueConfig := load.Config{
Dir: bd.ModuleRoot,
ModuleRoot: bd.ModuleRoot,
Tags: tags,
}
// Make args relative to the module directory
args := make([]string, 0, len(b.cfg.args)+2)
for _, path := range b.cfg.args {
target, err := filepath.Abs(path)
if err != nil {
return bd, errors.Wrap(fmt.Errorf("could not find absolute path: %w", err))
}
relPath, err := filepath.Rel(bd.ModuleRoot, target)
if err != nil {
return bd, errors.Wrap(fmt.Errorf("invalid argument, must be relative to cue.mod: %w", err))
}
// WATCH OUT: Assumes only one instance path is provided via args, which is
// true when I added this, but may be a poor assumption by the time you read
// this.
bd.InstancePath = holos.InstancePath(target)
bd.Dir = relPath
relPath = "./" + relPath
args = append(args, relPath)
}
instances := load.Instances(args, &cueConfig)
values, err := b.ctx.BuildInstances(instances)
if err != nil {
err = errors.Wrap(err)
return
}
// Unify into a single Value
for _, v := range values {
bd.Value = bd.Value.Unify(v)
}
// Fill in #UserData
userData, err := loadUserData(b.ctx, bd.ModuleRoot)
if err != nil {
err = errors.Wrap(err)
return
}
bd.Value = bd.Value.FillPath(cue.ParsePath("#UserData"), userData)
return
}
// loadUserData recursively unifies userdata/**/*.json files into cue.Value val.
func loadUserData(ctx *cue.Context, moduleRoot string) (val cue.Value, err error) {
// Ensure the value is from the same runtime, otherwise cue panics.
val = ctx.CompileString("")
userdataPath := filepath.Join(moduleRoot, "userdata")
if err = os.MkdirAll(userdataPath, 0755); err != nil {
return val, errors.Wrap(err)
}
err = filepath.Walk(userdataPath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return errors.Wrap(err)
}
if !info.IsDir() && filepath.Ext(info.Name()) == ".json" {
userData, err := os.ReadFile(path)
if err != nil {
return errors.Wrap(err)
}
val = val.Unify(ctx.CompileBytes(userData, cue.Filename(path)))
}
return nil
})
return val, errors.Wrap(err)
}
// Run builds the cue entrypoint into zero or more Results. Exactly one CUE
// package entrypoint is expected in the args slice. The platform config is
// provided to the entrypoint through a json encoded string tag named
// platform_config. The resulting cue.Value is unified with all user data files
// at the path "#UserData".
//
// Deprecated: Use holos.Builder instead
func (b *Builder) Run(ctx context.Context, cfg *client.Config) (results []*render.Result, err error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building instances")
bd, err := b.Unify(ctx, cfg)
if err != nil {
return nil, err
}
return b.build(ctx, bd)
}
func (b *Builder) build(ctx context.Context, bd holos.BuildData) (results []*render.Result, err error) {
log := logger.FromContext(ctx).With("dir", bd.InstancePath)
value := bd.Value
if err := value.Err(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not build %s: %w", bd.InstancePath, err))
}
log.DebugContext(ctx, "cue: validating instance")
if err := value.Validate(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not validate: %w", err))
}
log.DebugContext(ctx, "cue: decoding holos build plan")
jsonBytes, err := value.MarshalJSON()
if err != nil {
return nil, errors.Wrap(fmt.Errorf("could not marshal cue instance %s: %w", bd.Dir, err))
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
// Discriminate the type of build plan.
tm := &v1alpha1.TypeMeta{}
err = decoder.Decode(tm)
if err != nil {
return nil, errors.Wrap(fmt.Errorf("invalid BuildPlan: %s: %w", bd.Dir, err))
}
log.DebugContext(ctx, "cue: discriminated build kind: "+tm.Kind, "kind", tm.Kind, "apiVersion", tm.APIVersion)
// New decoder for the full object
decoder = json.NewDecoder(bytes.NewReader(jsonBytes))
// TODO: When we release v1, explicitly allow unknown fields so we can add
// fields without needing to bump the major version. Disallow until we reach
// v1 for clear error reporting.
decoder.DisallowUnknownFields()
switch tm.Kind {
case "BuildPlan":
var bp core_v1alpha3.BuildPlan
if err = decoder.Decode(&bp); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode BuildPlan %s: %w", bd.Dir, err))
return
}
results, err = b.buildPlan(ctx, &bp, bd.InstancePath)
if err != nil {
return results, err
}
default:
err = errors.Wrap(fmt.Errorf("unknown kind: %v", tm.Kind))
}
return results, err
}
func (b *Builder) buildPlan(ctx context.Context, buildPlan *core_v1alpha3.BuildPlan, path holos.InstancePath) (results []*render.Result, err error) {
log := logger.FromContext(ctx)
bpw := buildPlanWrapper{buildPlan: buildPlan}
if err := bpw.validate(); err != nil {
log.WarnContext(ctx, "could not validate", "skipped", true, "err", err)
return nil, errors.Wrap(fmt.Errorf("could not validate %w", err))
}
if buildPlan.Spec.Disabled {
log.DebugContext(ctx, "skipped: spec.disabled is true", "skipped", true)
return
}
results = make([]*render.Result, 0, bpw.resultCapacity())
log.DebugContext(ctx, "allocated results slice", "cap", bpw.resultCapacity())
for _, component := range buildPlan.Spec.Components.Resources {
ko := render.KubernetesObjects{Component: component}
if result, err := ko.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
for _, component := range buildPlan.Spec.Components.KubernetesObjectsList {
ko := render.KubernetesObjects{Component: component}
if result, err := ko.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
for _, component := range buildPlan.Spec.Components.HelmChartList {
hc := render.HelmChart{Component: component}
if result, err := hc.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
for _, component := range buildPlan.Spec.Components.KustomizeBuildList {
kb := render.KustomizeBuild{Component: component}
if result, err := kb.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
log.DebugContext(ctx, "returning results", "len", len(results))
return results, nil
}
// findCueMod returns the root module location containing the cue.mod file or
// directory or an error if the builder arguments do not share a common root
// module.
func (b *Builder) findCueMod() (dir holos.PathCueMod, err error) {
for _, origPath := range b.cfg.args {
absPath, err := filepath.Abs(origPath)
if err != nil {
return "", err
}
path := holos.PathCueMod(absPath)
for {
if _, err := os.Stat(filepath.Join(string(path), "cue.mod")); err == nil {
if dir != "" && dir != path {
return "", fmt.Errorf("multiple modules not supported: %v is not %v", dir, path)
}
dir = path
break
} else if !os.IsNotExist(err) {
return "", err
}
parentPath := holos.PathCueMod(filepath.Dir(string(path)))
if parentPath == path {
return "", fmt.Errorf("no cue.mod from root to leaf: %v", origPath)
}
path = parentPath
}
}
return dir, nil
}
// Platform builds a platform
// TODO: Refactor, lift up into NewPlatform RunE.
func (b *Builder) Platform(ctx context.Context, cfg *client.Config) (*core_v1alpha2.Platform, error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building platform instance")
bd, err := b.Unify(ctx, cfg)
if err != nil {
return nil, errors.Wrap(err)
}
return b.runPlatform(ctx, bd)
}
func (b *Builder) runPlatform(ctx context.Context, bd holos.BuildData) (*core_v1alpha2.Platform, error) {
path := holos.InstancePath(bd.Dir)
log := logger.FromContext(ctx).With("dir", path)
value := bd.Value
if err := bd.Value.Err(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not load: %w", err))
}
log.DebugContext(ctx, "cue: validating instance")
if err := value.Validate(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not validate: %w", err))
}
log.DebugContext(ctx, "cue: decoding holos platform")
jsonBytes, err := value.MarshalJSON()
if err != nil {
return nil, errors.Wrap(fmt.Errorf("could not marshal cue instance %s: %w", bd.Dir, err))
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
// Discriminate the type of build plan.
tm := &meta_v1alpha2.TypeMeta{}
err = decoder.Decode(tm)
if err != nil {
return nil, errors.Wrap(fmt.Errorf("invalid platform: %s: %w", bd.Dir, err))
}
log.DebugContext(ctx, "cue: discriminated build kind: "+tm.GetKind(), "kind", tm.GetKind(), "apiVersion", tm.GetAPIVersion())
decoder = json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
var pf core_v1alpha2.Platform
switch tm.GetKind() {
case "Platform":
if err = decoder.Decode(&pf); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode platform %s: %w", bd.Dir, err))
return nil, err
}
return &pf, nil
default:
err = errors.Wrap(fmt.Errorf("unknown kind: %v", tm.GetKind()))
}
return nil, err
}

View File

@@ -0,0 +1,44 @@
package builder
import (
"github.com/holos-run/holos/internal/builder/v1alpha5"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
)
type BuildPlan struct {
holos.BuildPlan
}
func LoadBuildPlan(i *Instance, opts holos.BuildOpts) (bp BuildPlan, err error) {
err = i.Discriminate(func(tm holos.TypeMeta) error {
if tm.Kind != "BuildPlan" {
return errors.Format("unsupported kind: %s, want BuildPlan", tm.Kind)
}
switch version := tm.APIVersion; version {
case "v1alpha5":
bp = BuildPlan{&v1alpha5.BuildPlan{Opts: opts}}
default:
return errors.Format("unsupported version: %s", version)
}
return nil
})
if err != nil {
return bp, errors.Wrap(err)
}
// Get the holos: field value from cue.
v, err := i.HolosValue()
if err != nil {
return bp, errors.Wrap(err)
}
// Load the platform from the cue value.
if err := bp.Load(v); err != nil {
return bp, errors.Wrap(err)
}
return bp, err
}

Some files were not shown because too many files have changed in this diff Show More