Compare commits

...

19 Commits

Author SHA1 Message Date
Jeff McCune
3fd06f594b schemas: fix kustomize patch name field (#348)
Without this patch trying to use a Kustomize patch with the optional
name field omitted results in the following error:

  could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
  holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
      $WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2

This patch fixes the problem by providing a default value for the name
field matching the Go zero value for a string.
2024-11-20 15:08:44 -08:00
Jeff McCune
a75338f21c docs: add bug report issue template, disable dev deploy 2024-11-20 14:50:25 -08:00
Jeff McCune
6002040360 version 0.100.0 2024-11-20 11:34:04 -08:00
Jeff McCune
864d7d442b build: pass platform component labels and annotations to BuildPlan
Without this patch the BuildPlan resulting from a Platform that has
components with labels and annotations does not have the labels or
annotations of the source component.

Holos should copy the labels and annotations defined on each of the
Platform.spec.components to the resulting BuildPlan so end users can see
clearly where a BuildPlan originated from, and filter with selectors the
intermediate output BuildPlan the same way we filter with selectors the
original Platform spec components list.

Result:

```
holos init platform v1alpha5 --force
holos show buildplans  | head
```

```yaml
kind: BuildPlan
apiVersion: v1alpha5
metadata:
  name: podinfo
  labels:
    app.holos.run/cluster: local
    app.holos.run/name: podinfo
  annotations:
    app.holos.run/description: podinfo for cluster local
```
2024-11-20 11:34:03 -08:00
Jeff McCune
791c0a9ffd util: fix misleading RunCmd error message source location
The error message misleads the reader to the utility function.  It
should lead to the caller.
2024-11-20 11:23:54 -08:00
Jeff McCune
bfe4a8d7c4 remove unmaintained embedded platforms
v1alpha5 is current and maintained.
2024-11-20 11:23:54 -08:00
Jeff McCune
eaa508a6f8 remove v1alpha1 schemas and support 2024-11-20 11:23:53 -08:00
Jeff McCune
63256a2845 remove embedded k3d platform
No longer used, v1alpha5 is used in the docs and maintained.
2024-11-20 11:23:53 -08:00
Jeff McCune
937c1dc953 show: fix inconsistent ordering of output
Without this patch the holos show buildplans command returns results in
an inconsistent order.  This is a problem because the output should be
idempotent.

This patch fixes the problem by adding an EncodeSeq(idx int, v any) method to
the encoder interface.  idx represents the index position of the
Platform.spec.components list after selector filtering has been applied.

This patch modifies the json and yaml encoders to buffer out of order
results from the concurrent go routines.

Result:

Concurrent execution is preserved. The buffer is kept to a reasonable
size, entries are deleted once they're encoded in the correct order.

Most importantly the output is consistent and idempotent so we can write
effective integration tests.
2024-11-20 11:23:52 -08:00
Jeff McCune
11bd50e2eb show: fix buildplans selector inconsistency
Sometimes, but not always, the holos show buildplans command produces no
output.

```
❯ holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug
finalized config from flags
rendered platform in 13.458µs
```

It only happens when there's a selector.  It doesn't happen without the
selector flag.  It only happens with ==, not with =.

This test fails quickly.

```
while [[ $(holos show buildplans --selector app.holos.run/cluster==w3 --log-level=debug | wc -l) -eq 39 ]]; do true; done
```

This test runs until killed.

```
while [[ $(holos show buildplans --log-level=debug | wc -l) -eq 279 ]]; do true; done
```

Solution:

The problem is the use of the map.  Iterating over the keys happens in a
random order.  With the fix we check in an explicit order.
2024-11-20 11:23:52 -08:00
Jeff McCune
7cfcf55565 add yaml struct tags to core v1alpha5 schemas
Without this patch the `holos show buildplans` BuildPlan output has
incorrect yaml with the v3 encoder.  For example apiversion: v1alpha5
instead of apiVersion.
2024-11-20 11:23:52 -08:00
Jeff McCune
8b22ba04e1 cli: add show command and refactor interfaces (#331)
Show subcommand:

This is large change that accomplishes a number of goals.  First, there
was no convenient way to show a build plan without using the debug logs
to indentify the tags to inject, then calling the cue command with the
right incantation to inspect the BuildPlan.

This patch addresses the problem by adding a `holos show buildplans`
command.  The command loads the Platform spec from the platform
directory, then iterates over all Components to produce the BuildPlan.

This patch adds labels and annotations to the platform Components
collection in order to select and filter the output.

Result:

```
❯ holos show components --selector app.holos.run/cluster=local --format=yaml | head
kind: BuildPlan
apiversion: v1alpha5
metadata:
  name: podinfo
spec:
  artifacts:
    - artifact: clusters/local/components/podinfo/podinfo.gen.yaml
      generators:
        - kind: Helm
          output: helm.gen.yaml
```

---

Interface refactor:

This refactors the interface between the `holos` Go CLI layer and the
various core schema data structures.  We now use a proper Go interface.
Concurrent execution over platform components has been improved to
accept a closure function so we can use the same interface method to
process the components.  We use this to show each component and render
each component from different subcommands using the same interface
embedded in the builder.Platform struct.

The embedded interface allows us to easily swap in different versions,
e.g. v1beta1 and eventually v1.  The number of interface methods are
quite small.  14 methods across 4 interfaces in holos/interface.go.

---

Remove old versions:

This patch removes support for versions prior to v1alpha5 in an effort
to clean up cruft.
2024-11-20 11:23:51 -08:00
Nate McCurdy
7d5187873b docs: Remove non-existent CLI completions, add pwsh
Cobra provides completions for bash, zsh, fish, and powershell, not
ksh.
2024-11-18 10:27:33 -08:00
Jeff McCune
03b796312a cli: gate grpc client and auth flags behind feature flag
Previously the holos render platform and component subcommands had flags
for oidc authentication and client access to the gRPC service.  These
flags aren't currently used, they're remnants from the json powered form
prototype.

This patch gates the flags behind a feature flag which is disabled by
default.

Result:

  holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int   number of components to render concurrently (default 8)
  -v, --version           version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")

---

  HOLOS_FEATURE_CLIENT=1 holos render platform --help

render an entire platform

Usage:
  holos render platform DIRECTORY [flags]

Examples:
  holos render platform ./platform

Flags:
      --concurrency int             number of components to render concurrently (default 8)
      --oidc-client-id string       oidc client id. (default "270319630705329162@holos_platform")
      --oidc-extra-scopes strings   optional oidc scopes
      --oidc-force-refresh          force refresh
      --oidc-issuer string          oidc token issuer url. (default "https://login.holos.run")
      --oidc-scopes strings         required oidc scopes (default openid,email,profile,groups,offline_access)
      --server string               server to connect to (default "https://app.holos.run:443")
  -v, --version                     version for platform

Global Flags:
      --log-drop strings    log attributes to drop (example "user-agent,version")
      --log-format string   log format (text|json|console) (default "console")
      --log-level string    log level (debug|info|warn|error) (default "info")
2024-11-17 16:06:57 -08:00
Jeff McCune
20fb39e49b docs: add clusters topic (#343)
Previously we didn't have good documentation on how to manage multiple
sets of clusters.

This patch adds a clusters topic in the structures category.  Each one
of the environments, projects, owners, etc... structures follow the same
pattern as #Clusters and #ClusterSets, so it makes sense to put them
into a dedicated sidebar category for specific CUE structures.
2024-11-17 14:45:32 -08:00
Jeff McCune
c9c8c13810 docs: replace touch with cat 2024-11-15 09:54:09 -07:00
Jeff McCune
374cd872e9 docs: CUE not cue and typo fix in hello holos 2024-11-15 09:40:02 -07:00
Jeff McCune
8db06dd0e1 releaser: fix brew test command (#327)
holos version isn't a valid command but holos --version is.
2024-11-14 16:14:26 -07:00
Jeff McCune
66acadf86d docs: support brew install (#327) 2024-11-14 16:07:13 -07:00
580 changed files with 2715 additions and 202473 deletions

View File

@@ -11,6 +11,7 @@
"admissionregistration",
"alertmanager",
"alertmanagers",
"anchore",
"anthos",
"apiextensions",
"apimachinery",
@@ -34,6 +35,7 @@
"balancereader",
"blackbox",
"buildplan",
"buildplans",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
@@ -56,6 +58,7 @@
"Cmds",
"CNCF",
"CODEOWNERS",
"componentconfig",
"configdir",
"configmap",
"configmapargs",
@@ -75,6 +78,7 @@
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devel",
"devicecode",
"dnsmasq",
"dscacheutil",
@@ -137,6 +141,7 @@
"httproute",
"httproutes",
"iampolicygenerator",
"incpatch",
"Infima",
"intstr",
"isatty",
@@ -256,6 +261,7 @@
"rolebinding",
"rootfs",
"ropc",
"sboms",
"seccomp",
"secretargs",
"SECRETKEY",
@@ -298,10 +304,12 @@
"tokencache",
"Tokener",
"tolerations",
"TOPLEVEL",
"Traceid",
"traefik",
"transactionhistory",
"tsdb",
"txtar",
"typemeta",
"udev",
"uibutton",

124
.github/ISSUE_TEMPLATE/bug-report.md vendored Normal file
View File

@@ -0,0 +1,124 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: NeedsInvestigation, Triage
assignees: ''
---
<!--
Please answer these questions before submitting your issue. Thanks!
To ask questions, see https://github.com/holos-run/holos/discussions
-->
### What version of holos are you using (`holos --version`)?
```shell
holos --version
```
### Does this issue reproduce with the latest release?
<!--
Get the latest release with:
brew install holos-run/tap/holos
Or see https://holos.run/docs/v1alpha5/tutorial/setup/
-->
### What did you do?
<!--
Please provide a testscript that should pass, but does not because of the bug.
See the below example.
You can create a txtar from a directory with:
holos txtar ./path/to/dir
Refer to: https://github.com/rogpeppe/go-internal/tree/master/cmd/testscript
-->
Steps to reproduce:
```shell
brew install testscript
```
```shell
testscript -v -continue example.txt
```
```txt
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work.
exec holos --version
exec holos init platform v1alpha5 --force
# want a BuildPlan shown
exec holos show buildplans
stdout 'kind: BuildPlan'
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}
```
### What did you expect to see?
The testscript should pass.
### What did you see instead?
The testscript fails because of the bug.
```shell
testscript -v -continue example.txt
```
```txt
# Have: an error related to the imported Kustomize schemas.
# Want: holos show buildplans to work. (0.073s)
> exec holos --version
[stdout]
0.100.0
> exec holos init platform v1alpha5 --force
# want a BuildPlan shown (0.085s)
> exec holos show buildplans
[stderr]
could not run: holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string at builder/v1alpha5/builder.go:218
holos.spec.artifacts.0.transformers.0.kustomize.kustomization.patches.0.target.name: cannot convert non-concrete value string:
$WORK/cue.mod/gen/sigs.k8s.io/kustomize/api/types/var_go_gen.cue:33:2
[exit status 1]
FAIL: example.txt:7: unexpected command failure
> stdout 'kind: BuildPlan'
FAIL: example.txt:8: no match for `kind: BuildPlan` found in stdout
# want this error to go away (0.000s)
> ! stderr 'cannot convert non-concrete value string'
FAIL: example.txt:10: unexpected match for `cannot convert non-concrete value string` found in stderr: cannot convert non-concrete value string
failed run
```

View File

@@ -2,7 +2,7 @@ name: Dev Deploy
on:
push:
branches: ['main', 'dev-deploy']
branches: ['dev-deploy']
jobs:
deploy:

View File

@@ -35,6 +35,9 @@ jobs:
with:
go-version: stable
- name: Setup Syft
uses: anchore/sbom-action/download-syft@1ca97d9028b51809cf6d3c934c3e160716e1b605 # v0.17.5
# Necessary to run these outside of goreleaser, otherwise
# /home/runner/_work/holos/holos/internal/frontend/node_modules/.bin/protoc-gen-connect-query is not in PATH
- name: Install Tools
@@ -54,11 +57,19 @@ jobs:
- name: Git diff
run: git diff
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
version: '~> v2'
args: release --clean
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -6,7 +6,7 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
version: 2
before:
hooks:
@@ -50,3 +50,39 @@ changelog:
exclude:
- "^docs:"
- "^test:"
source:
enabled: true
name_template: '{{ .ProjectName }}_{{ .Version }}_source_code'
sboms:
- id: source
artifacts: source
documents:
- "{{ .ProjectName }}_{{ .Version }}_sbom.spdx.json"
brews:
- name: holos
repository:
owner: holos-run
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
homepage: "https://holos.run"
description: "Holos CLI"
dependencies:
- name: helm
type: optional
- name: kubectl
type: optional
install: |
bin.install "holos"
bash_output = Utils.safe_popen_read(bin/"holos", "completion", "bash")
(bash_completion/"holos").write bash_output
zsh_output = Utils.safe_popen_read(bin/"holos", "completion", "zsh")
(zsh_completion/"_holos").write zsh_output
fish_output = Utils.safe_popen_read(bin/"holos", "completion", "fish")
(fish_completion/"holos.fish").write fish_output
test: |
system "#{bin}/holos --version"

View File

@@ -46,6 +46,11 @@ type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.

View File

@@ -23,23 +23,21 @@ package core
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
// Source reflects the origin of the BuildPlan.
Source BuildPlanSource `json:"source,omitempty"`
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
// BuildPlanSource reflects the origin of a [BuildPlan]. Useful to save a build
@@ -47,7 +45,7 @@ type BuildPlanSpec struct {
// component collection.
type BuildPlanSource struct {
// Component reflects the component that produced the build plan.
Component Component `json:"component,omitempty"`
Component Component `json:"component,omitempty" yaml:"component,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
@@ -71,10 +69,10 @@ type BuildPlanSource struct {
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
@@ -90,19 +88,19 @@ type Artifact struct {
// 3. [File] - Generates data by reading a file from the component directory.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
// Resource represents one kubernetes api object.
@@ -119,24 +117,24 @@ type Resources map[Kind]map[InternalLabel]Resource
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
Source FilePath `json:"source" yaml:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty"`
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty"`
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
@@ -145,19 +143,19 @@ type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version"`
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
@@ -171,17 +169,17 @@ type Repository struct {
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
@@ -191,15 +189,15 @@ type Transformer struct {
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
@@ -229,7 +227,13 @@ type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
@@ -242,20 +246,20 @@ type Metadata struct {
// cue export --out yaml ./platform
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components"`
Components []Component `json:"components" yaml:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
@@ -263,17 +267,23 @@ type PlatformSpec struct {
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path"`
Path string `json:"path" yaml:"path"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty"`
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,56 +0,0 @@
package v1alpha1
import (
"errors"
"fmt"
"strings"
)
// BuildPlan is the primary interface between CUE and the Holos cli.
type BuildPlan struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
Spec BuildPlanSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
}
type BuildPlanSpec struct {
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
Components BuildPlanComponents `json:"components,omitempty" yaml:"components,omitempty"`
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
type BuildPlanComponents struct {
HelmChartList []HelmChart `json:"helmChartList,omitempty" yaml:"helmChartList,omitempty"`
KubernetesObjectsList []KubernetesObjects `json:"kubernetesObjectsList,omitempty" yaml:"kubernetesObjectsList,omitempty"`
KustomizeBuildList []KustomizeBuild `json:"kustomizeBuildList,omitempty" yaml:"kustomizeBuildList,omitempty"`
Resources map[string]KubernetesObjects `json:"resources,omitempty" yaml:"resources,omitempty"`
}
func (bp *BuildPlan) Validate() error {
errs := make([]string, 0, 2)
if bp.Kind != BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", BuildPlanKind, bp.Kind))
}
if bp.APIVersion != APIVersion {
errs = append(errs, fmt.Sprintf("apiVersion invalid: want: %s have: %s", APIVersion, bp.APIVersion))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (bp *BuildPlan) ResultCapacity() (count int) {
if bp == nil {
return 0
}
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}

View File

@@ -1,30 +0,0 @@
package v1alpha1
// HolosComponent defines the fields common to all holos component kinds including the Render Result.
type HolosComponent struct {
TypeMeta `json:",inline" yaml:",inline"`
// Metadata represents the holos component name
Metadata ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
// APIObjectMap holds the marshalled representation of api objects. Think of
// these as resources overlaid at the back of the render pipeline.
APIObjectMap APIObjectMap `json:"apiObjectMap,omitempty" yaml:"apiObjectMap,omitempty"`
// Kustomization holds the marshalled representation of the flux kustomization
// which reconciles resources in git with the api server.
Kustomization `json:",inline" yaml:",inline"`
// Kustomize represents a kubectl kustomize build post-processing step.
Kustomize `json:",inline" yaml:",inline"`
// Skip causes holos to take no action regarding the component.
Skip bool
}
func (hc *HolosComponent) NewResult() *Result {
return &Result{HolosComponent: *hc}
}
func (hc *HolosComponent) GetAPIVersion() string {
return hc.APIVersion
}
func (hc *HolosComponent) GetKind() string {
return hc.Kind
}

View File

@@ -1,11 +0,0 @@
package v1alpha1
const (
APIVersion = "holos.run/v1alpha1"
BuildPlanKind = "BuildPlan"
HelmChartKind = "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
ChartDir = "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
ResourcesFile = "resources.yaml"
)

View File

@@ -1,2 +0,0 @@
// Package v1alpha1 defines the api boundary between CUE and Holos.
package v1alpha1

View File

@@ -1,13 +0,0 @@
package v1alpha1
import object "github.com/holos-run/holos/service/gen/holos/object/v1alpha1"
// Form represents a collection of Formly json powered form.
type Form struct {
TypeMeta `json:",inline" yaml:",inline"`
Spec FormSpec `json:"spec" yaml:"spec"`
}
type FormSpec struct {
Form object.Form `json:"form" yaml:"form"`
}

View File

@@ -1,184 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// A HelmChart represents a helm command to provide chart values in order to render kubernetes api objects.
type HelmChart struct {
HolosComponent `json:",inline" yaml:",inline"`
// Namespace is the namespace to install into. TODO: Use metadata.namespace instead.
Namespace string `json:"namespace"`
Chart Chart `json:"chart"`
ValuesContent string `json:"valuesContent"`
EnableHooks bool `json:"enableHooks"`
}
type Chart struct {
Name string `json:"name"`
Version string `json:"version"`
Release string `json:"release"`
Repository Repository `json:"repository,omitempty"`
}
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
func (hc *HelmChart) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: hc.HolosComponent}
if err := hc.helm(ctx, &result, path); err != nil {
return nil, err
}
result.addObjectMap(ctx, hc.APIObjectMap)
if err := result.kustomize(ctx); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not kustomize: %w", err))
}
return &result, nil
}
// runHelm provides the values produced by CUE to helm template and returns
// the rendered kubernetes api objects in the result.
func (hc *HelmChart) helm(ctx context.Context, r *Result, path holos.InstancePath) error {
log := logger.FromContext(ctx).With("chart", hc.Chart.Name)
if hc.Chart.Name == "" {
log.WarnContext(ctx, "skipping helm: no chart name specified, use a different component type")
return nil
}
cachedChartPath := filepath.Join(string(path), ChartDir, filepath.Base(hc.Chart.Name))
if isNotExist(cachedChartPath) {
// Add repositories
repo := hc.Chart.Repository
if repo.URL != "" {
out, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo add: %w", err))
}
// Update repository
out, err = util.RunCmd(ctx, "helm", "repo", "update", repo.Name)
if err != nil {
log.ErrorContext(ctx, "could not run helm", "stderr", out.Stderr.String(), "stdout", out.Stdout.String())
return errors.Wrap(fmt.Errorf("could not run helm repo update: %w", err))
}
} else {
log.DebugContext(ctx, "no chart repository url proceeding assuming oci chart")
}
// Cache the chart
if err := cacheChart(ctx, path, ChartDir, hc.Chart); err != nil {
return fmt.Errorf("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos")
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, tempDir)
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, []byte(hc.ValuesContent), 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "helm: wrote values", "path", valuesPath, "bytes", len(hc.ValuesContent))
// Run charts
chart := hc.Chart
args := []string{"template"}
if !hc.EnableHooks {
args = append(args, "--no-hooks")
}
namespace := hc.Namespace
args = append(args, "--include-crds", "--values", valuesPath, "--namespace", namespace, "--kubeconfig", "/dev/null", "--version", chart.Version, chart.Release, cachedChartPath)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Wrap(fmt.Errorf("could not run helm template: %w", err))
}
r.accumulatedOutput = helmOut.Stdout.String()
return nil
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// It is assumed that the only method responsible for writing to chartDir is
// cacheChart itself.
//
// This relies on the atomicity of moving temporary directories into place on
// the same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is an
// expected scenario when this function is called concurrently.
func cacheChart(ctx context.Context, path holos.InstancePath, chartDir string, chart Chart) error {
log := logger.FromContext(ctx)
cacheTemp, err := os.MkdirTemp(string(path), chartDir)
if err != nil {
return errors.Wrap(fmt.Errorf("could not make temp dir: %w", err))
}
defer util.Remove(ctx, cacheTemp)
chartName := chart.Name
if chart.Repository.Name != "" {
chartName = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, chartName)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
cachePath := filepath.Join(string(path), chartDir)
if err := os.MkdirAll(cachePath, 0777); err != nil {
return errors.Wrap(fmt.Errorf("could not mkdir: %w", err))
}
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
for _, item := range items {
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cachePath, item.Name())
log.DebugContext(ctx, "rename", "src", src, "dst", dst)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
}
}
log.InfoContext(ctx, "cached", "chart", chart.Name, "chart_version", chart.Version, "path", cachePath)
return nil
}
func isNotExist(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}

View File

@@ -1,21 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
const KubernetesObjectsKind = "KubernetesObjects"
// KubernetesObjects represents CUE output which directly provides Kubernetes api objects to holos.
type KubernetesObjects struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces kubernetes api objects from the APIObjectMap
func (o *KubernetesObjects) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
result := Result{HolosComponent: o.HolosComponent}
result.addObjectMap(ctx, o.APIObjectMap)
return &result, nil
}

View File

@@ -1,7 +0,0 @@
package v1alpha1
// Kustomization holds the rendered flux kustomization api object content for git ops.
type Kustomization struct {
// KsContent is the yaml representation of the flux kustomization for gitops.
KsContent string `json:"ksContent,omitempty" yaml:"ksContent,omitempty"`
}

View File

@@ -1,47 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
const KustomizeBuildKind = "KustomizeBuild"
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process raw yaml file resources in a holos component directory.
// 2. Post process a HelmChart to inject istio, add custom labels, etc...
type Kustomize struct {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
KustomizeFiles FileContentMap `json:"kustomizeFiles,omitempty" yaml:"kustomizeFiles,omitempty"`
// ResourcesFile is the file name used for api objects in kustomization.yaml
ResourcesFile string `json:"resourcesFile,omitempty" yaml:"resourcesFile,omitempty"`
}
// KustomizeBuild renders plain yaml files in the holos component directory using kubectl kustomize build.
type KustomizeBuild struct {
HolosComponent `json:",inline" yaml:",inline"`
}
// Render produces a Result by executing kubectl kustomize on the holos
// component path. Useful for processing raw yaml files.
func (kb *KustomizeBuild) Render(ctx context.Context, path holos.InstancePath) (*Result, error) {
log := logger.FromContext(ctx)
result := Result{HolosComponent: kb.HolosComponent}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", string(path))
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return nil, errors.Wrap(err)
}
// Replace the accumulated output
result.accumulatedOutput = kOut.Stdout.String()
// Add CUE based api objects.
result.addObjectMap(ctx, kb.APIObjectMap)
return &result, nil
}

View File

@@ -1,14 +0,0 @@
package v1alpha1
// Label is an arbitrary unique identifier. Defined as a type for clarity and type checking.
type Label string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
type Kind string
// APIObjectMap is the shape of marshalled api objects returned from cue to the
// holos cli. A map is used to improve the clarity of error messages from cue.
type APIObjectMap map[Kind]map[Label]string
// FileContentMap is a map of file names to file contents.
type FileContentMap map[string]string

View File

@@ -1,15 +0,0 @@
package v1alpha1
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are by holos objects distinct
// from kubernetes api objects.
type ObjectMeta struct {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
Name string `json:"name,omitempty" yaml:"name,omitempty"`
// Namespace confines a holos component to a single namespace via kustomize if set.
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Labels are not used but are copied from api machinery ObjectMeta for completeness.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations are not used but are copied from api machinery ObjectMeta for completeness.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}

View File

@@ -1,32 +0,0 @@
package v1alpha1
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
TypeMeta `json:",inline" yaml:",inline"`
Metadata ObjectMeta `json:"metadata" yaml:"metadata"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
// PlatformSpec represents the platform build plan specification.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// holos.platform.v1alpha1.PlatformService.GetPlatform method and provides to
// CUE using a tag.
Model structpb.Struct `json:"model" yaml:"model"`
Components []PlatformSpecComponent `json:"components" yaml:"components"`
}
// PlatformSpecComponent represents a component to build or render with flags to
// pass, for example the cluster name.
type PlatformSpecComponent struct {
// Path is the path of the component relative to the platform root.
Path string `json:"path" yaml:"path"`
// Cluster is the cluster name to use when building the component.
Cluster string `json:"cluster" yaml:"cluster"`
}

View File

@@ -1,22 +0,0 @@
package v1alpha1
import (
"context"
"github.com/holos-run/holos"
)
type Renderer interface {
GetKind() string
Render(ctx context.Context, path holos.InstancePath) (*Result, error)
}
// Render produces a Result representing the kubernetes api objects to
// configure. Each of the various holos component types, e.g. Helm, Kustomize,
// et al, should implement the Renderer interface. This process is best
// conceptualized as a data pipeline, for example a component may render a
// result by first calling helm template, then passing the result through
// kustomize, then mixing in overlay api objects.
func Render(ctx context.Context, r Renderer, path holos.InstancePath) (*Result, error) {
return r.Render(ctx, path)
}

View File

@@ -1,165 +0,0 @@
package v1alpha1
import (
"context"
"fmt"
"os"
"path/filepath"
"slices"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
)
// Result is the build result for display or writing. Holos components Render the Result as a data pipeline.
type Result struct {
HolosComponent
// accumulatedOutput accumulates rendered api objects.
accumulatedOutput string
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
DeployFiles FileContentMap `json:"deployFiles,omitempty" yaml:"deployFiles,omitempty"`
}
// Continue returns true if Skip is true indicating the result is to be skipped over.
func (r *Result) Continue() bool {
if r == nil {
return false
}
return r.Skip
}
func (r *Result) Name() string {
return r.Metadata.Name
}
func (r *Result) Filename(writeTo string, cluster string) string {
name := r.Metadata.Name
return filepath.Join(writeTo, "clusters", cluster, "components", name, name+".gen.yaml")
}
func (r *Result) KustomizationFilename(writeTo string, cluster string) string {
return filepath.Join(writeTo, "clusters", cluster, "holos", "components", r.Metadata.Name+"-kustomization.gen.yaml")
}
// AccumulatedOutput returns the accumulated rendered output.
func (r *Result) AccumulatedOutput() string {
return r.accumulatedOutput
}
// addObjectMap renders the provided APIObjectMap into the accumulated output.
func (r *Result) addObjectMap(ctx context.Context, objectMap APIObjectMap) {
log := logger.FromContext(ctx)
b := []byte(r.AccumulatedOutput())
kinds := make([]Kind, 0, len(objectMap))
// Sort the keys
for kind := range objectMap {
kinds = append(kinds, kind)
}
slices.Sort(kinds)
for _, kind := range kinds {
v := objectMap[kind]
// Sort the keys
names := make([]Label, 0, len(v))
for name := range v {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
yamlString := v[name]
log.Debug(fmt.Sprintf("%s/%s", kind, name), "kind", kind, "name", name)
b = util.EnsureNewline(b)
header := fmt.Sprintf("---\n# Source: CUE apiObjects.%s.%s\n", kind, name)
b = append(b, []byte(header+yamlString)...)
b = util.EnsureNewline(b)
}
}
r.accumulatedOutput = string(b)
}
// kustomize replaces the accumulated output with the output of kustomize build
func (r *Result) kustomize(ctx context.Context) error {
log := logger.FromContext(ctx)
if r.ResourcesFile == "" {
log.DebugContext(ctx, "skipping kustomize: no resourcesFile")
return nil
}
if len(r.KustomizeFiles) < 1 {
log.DebugContext(ctx, "skipping kustomize: no kustomizeFiles")
return nil
}
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
// Write the main api object resources file for kustomize.
target := filepath.Join(tempDir, r.ResourcesFile)
b := []byte(r.AccumulatedOutput())
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write resources: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
// Write the kustomization tree, kustomization.yaml must be in this map for kustomize to work.
for file, content := range r.KustomizeFiles {
target := filepath.Join(tempDir, file)
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return errors.Wrap(err)
}
b := []byte(content)
b = util.EnsureNewline(b)
if err := os.WriteFile(target, b, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write: %w", err))
}
log.DebugContext(ctx, "wrote: "+target, "op", "write", "path", target, "bytes", len(b))
}
// Run kustomize.
kOut, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
log.ErrorContext(ctx, kOut.Stderr.String())
return errors.Wrap(err)
}
// Replace the accumulated output
r.accumulatedOutput = kOut.Stdout.String()
return nil
}
func (r *Result) WriteDeployFiles(ctx context.Context, path string) error {
log := logger.FromContext(ctx)
if len(r.DeployFiles) == 0 {
return nil
}
for k, content := range r.DeployFiles {
path := filepath.Join(path, k)
if err := r.Save(ctx, path, content); err != nil {
return errors.Wrap(err)
}
log.InfoContext(ctx, "wrote deploy file", "path", path, "bytes", len(content))
}
return nil
}
// Save writes the content to the filesystem for git ops.
func (r *Result) Save(ctx context.Context, path string, content string) error {
log := logger.FromContext(ctx)
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, os.FileMode(0775)); err != nil {
log.WarnContext(ctx, "could not mkdir", "path", dir, "err", err)
return errors.Wrap(err)
}
// Write the file content
if err := os.WriteFile(path, []byte(content), os.FileMode(0644)); err != nil {
log.WarnContext(ctx, "could not write", "path", path, "err", err)
return errors.Wrap(err)
}
log.DebugContext(ctx, "out: wrote "+path, "action", "write", "path", path, "status", "ok")
return nil
}

View File

@@ -1,20 +0,0 @@
package v1alpha1
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
func (tm *TypeMeta) GetKind() string {
return tm.Kind
}
func (tm *TypeMeta) GetAPIVersion() string {
return tm.APIVersion
}
// Discriminator is an interface to discriminate the kind api object.
type Discriminator interface {
GetKind() string
GetAPIVersion() string
}

View File

@@ -17,9 +17,6 @@ func TestMain(m *testing.M) {
}))
}
func TestGuides_v1alpha4(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha4", "guides")))
}
func TestGuides_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "guides")))
}

File diff suppressed because it is too large Load Diff

View File

@@ -6,7 +6,7 @@ chmod 755 bin/helm
# Initialize the platform
exec holos init platform v1alpha5 --force
# when helm update returns an error
! exec holos render platform ./platform
! exec holos render platform
# holos should log the helm error to stderr
stderr 'Error: chart "podinfo" matching 0.0.0 not found in podinfo index'
-- bin/helm --

View File

@@ -0,0 +1,461 @@
# https://github.com/holos-run/holos/issues/331
# ensure holos show components --labels selects correctly.
# ensure BuildPlan includes labels and annotations from the platform component.
# ensure holos render platform injects the holos_component_labels and
# holos_component_annotations tags.
env HOME=$WORK
exec holos init platform v1alpha5 --force
exec holos show platform
cmp stdout want/platform.yaml
# all buildplans are selected by default
exec holos show buildplans
cmp stdout want/all-buildplans.yaml
# one = works in the selector
exec holos show buildplans --selector app.holos.run/name=empty1-label
cmp stdout want/buildplans.1.yaml
# double == works in the selector
exec holos show buildplans --selector app.holos.run/name==empty2-label
cmp stdout want/buildplans.2.yaml
# not equal != negates the selection
exec holos show buildplans --selector app.holos.run/name!=empty3-label
cmp stdout want/buildplans.3.yaml
exec holos show buildplans --selector app.holos.run/name!=something-else
cmp stdout want/buildplans.4.yaml
-- platform/empty.cue --
package holos
Platform: Components: {
empty1: _
empty2: _
empty3: _
empty4: _
}
-- platform/metadata.cue --
package holos
Platform: Components: [NAME=string]: {
name: NAME
path: "components/empty"
labels: "app.holos.run/name": "\(name)-label"
annotations: "app.holos.run/description": "\(name)-annotation empty test case"
}
-- components/empty/empty.cue --
package holos
Component: #Kubernetes & {}
holos: Component.BuildPlan
-- want/platform.yaml --
apiVersion: v1alpha5
kind: Platform
metadata:
name: default
spec:
components:
- annotations:
app.holos.run/description: empty1-annotation empty test case
labels:
app.holos.run/name: empty1-label
name: empty1
path: components/empty
- annotations:
app.holos.run/description: empty2-annotation empty test case
labels:
app.holos.run/name: empty2-label
name: empty2
path: components/empty
- annotations:
app.holos.run/description: empty3-annotation empty test case
labels:
app.holos.run/name: empty3-label
name: empty3
path: components/empty
- annotations:
app.holos.run/description: empty4-annotation empty test case
labels:
app.holos.run/name: empty4-label
name: empty4
path: components/empty
-- want/empty.yaml --
-- want/all-buildplans.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.1.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.2.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.3.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
-- want/buildplans.4.yaml --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty1
labels:
app.holos.run/name: empty1-label
annotations:
app.holos.run/description: empty1-annotation empty test case
spec:
artifacts:
- artifact: components/empty1/empty1.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty1/empty1.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty2
labels:
app.holos.run/name: empty2-label
annotations:
app.holos.run/description: empty2-annotation empty test case
spec:
artifacts:
- artifact: components/empty2/empty2.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty2/empty2.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty3
labels:
app.holos.run/name: empty3-label
annotations:
app.holos.run/description: empty3-annotation empty test case
spec:
artifacts:
- artifact: components/empty3/empty3.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty3/empty3.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
---
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: empty4
labels:
app.holos.run/name: empty4-label
annotations:
app.holos.run/description: empty4-annotation empty test case
spec:
artifacts:
- artifact: components/empty4/empty4.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/empty4/empty4.gen.yaml
kustomize:
kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml

View File

@@ -0,0 +1,34 @@
# https://github.com/holos-run/holos/issues/348
# when the optional kustomize patch name field is omitted
exec holos init platform v1alpha5 --force
# want a buildplan shown
exec holos show buildplans
stdout 'kind: BuildPlan'
# want this error to go away
! stderr 'cannot convert non-concrete value string'
-- platform/example.cue --
package holos
Platform: Components: example: {
name: "example"
path: "components/example"
}
-- components/example/example.cue --
package holos
import "encoding/yaml"
holos: Component.BuildPlan
Component: #Kustomize & {
KustomizeConfig: Kustomization: patches: [
{
target: kind: "CustomResourceDefinition"
patch: yaml.Marshal([{
op: "add"
path: "/metadata/annotations/example"
value: "example-value"
}])
},
]
}

View File

@@ -4,10 +4,10 @@
cd $WORK
# Generate the directory structure we're going to work in.
exec holos generate platform v1alpha5 --force
exec holos init platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform ./platform
exec holos render platform
stderr -count=1 '^rendered platform'
# When author.#Kubernetes is empty
@@ -45,8 +45,3 @@ spec:
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
source:
component:
name: no-name
path: no-path
parameters: {}

View File

@@ -43,6 +43,11 @@ type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Labels represent the BuildPlan metadata.labels field.
Labels map[string]string
// Annotations represent the BuildPlan metadata.annotations field.
Annotations map[string]string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.

View File

@@ -56,10 +56,10 @@ Output fields are write\-once. It is an error for multiple Generators or Transfo
```go
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
Artifact FilePath `json:"artifact,omitempty" yaml:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty" yaml:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty" yaml:"transformers,omitempty"`
Skip bool `json:"skip,omitempty" yaml:"skip,omitempty"`
}
```
@@ -75,15 +75,13 @@ Holos uses CUE to construct a BuildPlan. A future enhancement will support user
```go
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
Kind string `json:"kind" yaml:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
// Source reflects the origin of the BuildPlan.
Source BuildPlanSource `json:"source,omitempty"`
Spec BuildPlanSpec `json:"spec" yaml:"spec"`
}
```
@@ -95,7 +93,7 @@ BuildPlanSource reflects the origin of a [BuildPlan](<#BuildPlan>). Useful to sa
```go
type BuildPlanSource struct {
// Component reflects the component that produced the build plan.
Component Component `json:"component,omitempty"`
Component Component `json:"component,omitempty" yaml:"component,omitempty"`
}
```
@@ -107,9 +105,9 @@ BuildPlanSpec represents the specification of the [BuildPlan](<#BuildPlan>).
```go
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
Artifacts []Artifact `json:"artifacts" yaml:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`
}
```
@@ -121,13 +119,13 @@ Chart represents a [Helm](<#Helm>) Chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Version represents the chart version.
Version string `json:"version"`
Version string `json:"version" yaml:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
Release string `json:"release" yaml:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
Repository Repository `json:"repository,omitempty" yaml:"repository,omitempty"`
}
```
@@ -140,19 +138,25 @@ Component represents the complete context necessary to produce a [BuildPlan](<#B
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path"`
Path string `json:"path" yaml:"path"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
WriteTo string `json:"writeTo,omitempty" yaml:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty"`
Parameters map[string]string `json:"parameters,omitempty" yaml:"parameters,omitempty"`
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
@@ -164,7 +168,7 @@ File represents a simple single file copy [Generator](<#Generator>). Useful with
```go
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
Source FilePath `json:"source" yaml:"source"`
}
```
@@ -209,19 +213,19 @@ Each Generator in an [Artifact](<#Artifact>) must have a distinct Output value f
```go
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
Resources Resources `json:"resources,omitempty" yaml:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
Helm Helm `json:"helm,omitempty" yaml:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
File File `json:"file,omitempty" yaml:"file,omitempty"`
}
```
@@ -233,18 +237,18 @@ Helm represents a [Chart](<#Chart>) manifest [Generator](<#Generator>).
```go
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
Chart Chart `json:"chart" yaml:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
Values Values `json:"values" yaml:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
EnableHooks bool `json:"enableHooks,omitempty" yaml:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty"`
APIVersions []string `json:"apiVersions,omitempty" yaml:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty"`
KubeVersion string `json:"kubeVersion,omitempty" yaml:"kubeVersion,omitempty"`
}
```
@@ -264,7 +268,7 @@ Join represents a [Transformer](<#Transformer>) using [bytes.Join](<https://pkg.
```go
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
Separator string `json:"separator,omitempty" yaml:"separator,omitempty"`
}
```
@@ -294,9 +298,9 @@ Kustomize represents a kustomization [Transformer](<#Transformer>).
```go
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
Kustomization Kustomization `json:"kustomization" yaml:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
Files FileContentMap `json:"files,omitempty" yaml:"files,omitempty"`
}
```
@@ -308,7 +312,13 @@ Metadata represents data about the resource such as the Name.
```go
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
Name string `json:"name" yaml:"name"`
// Labels represents a resource selector.
Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
Annotations map[string]string `json:"annotations,omitempty" yaml:"annotations,omitempty"`
}
```
@@ -326,14 +336,14 @@ cue export --out yaml ./platform
```go
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
APIVersion string `json:"apiVersion" yaml:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
Metadata Metadata `json:"metadata" yaml:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec"`
Spec PlatformSpec `json:"spec" yaml:"spec"`
}
```
@@ -345,7 +355,7 @@ PlatformSpec represents the platform specification.
```go
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components"`
Components []Component `json:"components" yaml:"components"`
}
```
@@ -356,8 +366,8 @@ Repository represents a [Helm](<#Helm>) [Chart](<#Chart>) repository.
```go
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
Name string `json:"name" yaml:"name"`
URL string `json:"url" yaml:"url"`
}
```
@@ -390,17 +400,17 @@ Transformer combines multiple inputs from prior [Generator](<#Generator>) or [Tr
```go
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
Kind string `json:"kind" yaml:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
Inputs []FilePath `json:"inputs" yaml:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
Output FilePath `json:"output" yaml:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
Kustomize Kustomize `json:"kustomize,omitempty" yaml:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
Join Join `json:"join,omitempty" yaml:"join,omitempty"`
}
```

View File

@@ -0,0 +1,16 @@
Integrate the `podinfo` component into the platform.
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```

View File

@@ -0,0 +1,34 @@
Create a directory for the example `podinfo` component we'll use to render
platform manifests.
```bash
mkdir -p components/podinfo
```
Create the CUE configuration for the example `podinfo` component.
```bash
cat <<EOF >components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Values: ui: {
message: string | *"Hello World" @tag(message, type=string)
}
}
```
```bash
EOF
```

View File

@@ -1,7 +1,7 @@
---
description: Architecture diagrams.
slug: architecture
sidebar_position: 90
sidebar_position: 100
---
import RenderPlatformDiagram from '@site/src/diagrams/render-platform-sequence.mdx';

View File

@@ -7,6 +7,8 @@ sidebar_position: 110
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
import CommonComponentIntegrate from '../../common/example-component-integrate.mdx';
# ArgoCD Application
@@ -30,62 +32,10 @@ mkdir holos-argocd-application && cd holos-argocd-application
holos init platform v1alpha5
```
### Creating a component
### Creating an example Component
Create a directory for the `podinfo` component. Create an empty file and then
add the following CUE configuration to it.
<Tabs groupId="1D2C6013-3D19-4516-8147-5A6EE214CAFF">
<TabItem value="components/podinfo/podinfo.cue" label="Podinfo Helm Chart">
```bash
mkdir -p components/podinfo
touch components/podinfo/podinfo.cue
```
```bash
cat <<EOF >components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
holos: Component.BuildPlan
Component: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
```bash
EOF
```
</TabItem>
</Tabs>
Integrate the `podinfo` component into the platform.
<Tabs groupId="tutorial-hello-register-podinfo-component">
<TabItem value="platform/podinfo.cue" label="Register Podinfo">
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
```
```bash
EOF
```
</TabItem>
</Tabs>
<CommonComponent />
<CommonComponentIntegrate />
## Adding ArgoCD Application
@@ -267,8 +217,8 @@ spec:
[podinfo]: https://github.com/stefanprodan/podinfo
[CUE Module]: https://cuelang.org/docs/reference/modules/
[CUE Tags]: https://cuelang.org/docs/howto/inject-value-into-evaluation-using-tag-attribute/
[Platform]: ../api/author.md#Platform
[Component Parameters]: ../topics/component-parameters.mdx
[Application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[ComponentConfig]: ../api/author.md#ComponentConfig
[Artifact]: ../api/core.md#Artifact
[Component Parameters]: ../component-parameters.mdx
[Platform]: ../../api/author.md#Platform
[ComponentConfig]: ../../api/author.md#ComponentConfig
[Artifact]: ../../api/core.md#Artifact

View File

@@ -0,0 +1,19 @@
---
slug: .
title: GitOps
description: Managing resources with GitOps.
sidebar_position: 120
---
import DocCardList from '@theme/DocCardList';
# GitOps
This section has self contained articles covering how to manage resources using
GitOps tooling like [ArgoCD] and [Flux].
---
<DocCardList />
[ArgoCD]: https://argo-cd.readthedocs.io/en/stable/
[Flux]: https://fluxcd.io/

View File

@@ -1,7 +1,7 @@
---
description: Build a local cluster for use with Holos.
slug: local-cluster
sidebar_position: 100
sidebar_position: 50
---
import Tabs from '@theme/Tabs';

View File

@@ -0,0 +1,424 @@
---
slug: clusters
title: Clusters
description: Managing clusters - management and workload sets.
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CommonComponent from '../../common/example-component.mdx';
# Clusters
## Overview
This topic covers one common method to manage multiple clusters with Holos. We'll
define two schemas to hold cluster attributes. First, a single `#Cluster` then
a `#Clusters` collection. We'll use a `Clusters: #Clusters` struct to look up
configuration data using a key. We'll use the cluster name as the lookup key
identifying the cluster.
We'll also organize sets of similar clusters by defining `#ClusterSet` and
`#ClusterSets`. We'll use a `ClusterSets:
#ClusterSets` struct to configure a management cluster and iterate over all
workload clusters.
## The Code
### Initializing the structure
Use `holos` to generate a minimal platform directory structure. Start by
creating a blank directory to hold the platform configuration.
```shell
mkdir holos-multiple-clusters && cd holos-multiple-clusters
```
```shell
holos init platform v1alpha5
```
### Example Component
<CommonComponent />
We'll integrate the component with the platform after we define clusters.
## Defining Clusters
We'll define a `#Cluster` schema and a `#Clusters` collection in this section.
We'll use these schemas to define a `Clusters` structure we use to manage
multiple clusters.
### Assumptions
We'll make the following assumptions, which hold true for many real world
environments.
1. There are two sets of clusters, workload clusters and management clusters.
2. There is one management cluster.
3. There are multiple workload clusters.
4. Each workload cluster is configured similarly, but not identically, to the
others.
### Prototyping the data
Before we define the schema, let's prototype the data structure we want to work
with. We want a structure that makes it easy to iterate over each cluster in
two distinct sets of clusters, management clusters and workload clusters. The
following `ClusterSets` struct accomplishes this goal.
```yaml showLineNumbers
management:
name: management
clusters:
management:
name: management
region: us-central1
set: management
workload:
name: workload
clusters:
e1:
name: e1
region: us-east1
set: workload
w1:
name: w1
region: us-west1
set: workload
```
:::tip
The `ClusterSets` data structure supports iterating over each cluster in each
cluster set.
:::
:::important
You're free to define your own fields and structures like we define `region` in
this topic.
:::
### Defining the schema
Armed with a concrete example of the structure, we can write a schema to define
and validate the data.
In CUE, schema definitions are usually defined at the root so they're accessible
in all subdirectories. The following is one example schema, you're free to
modify it to your situation. Holos is flexible, supporting schemas that match
your unique use case.
```bash
cat <<EOF > clusters.schema.cue
```
```cue showLineNumbers
package holos
import "strings"
// #Cluster represents one cluster
#Cluster: {
// name represents the cluster name.
name: string & =~"[a-z][a-z0-9]+" & strings.MinRunes(2) & strings.MaxRunes(63)
// Constrain the regions. No default, the region must be specified.
region: "us-east1" | "us-central1" | "us-west1"
// Each cluster must be in only one set of clusters. All but one cluster are
// workload clusters, so make it the default.
set: "management" | *"workload"
}
// #Clusters represents a cluster collection structure
#Clusters: {
// name is the lookup key for the collection.
[NAME=string]: #Cluster & {
// name must match the struct field name.
name: NAME
}
}
// #ClusterSet represents a set of clusters.
#ClusterSet: {
// name represents the cluster set name.
name: string & =~"[a-z][a-z0-9]+" & strings.MinRunes(2) & strings.MaxRunes(63)
clusters: #Clusters & {
// Constrain the cluster set to clusters having the same set. Ensures
// clusters are never mis-categorized.
[_]: set: name
}
}
// #ClusterSets represents a cluster set collection.
#ClusterSets: {
// name is the lookup key for the collection.
[NAME=string]: #ClusterSet & {
// name must match the struct field name.
name: NAME
}
}
```
```bash
EOF
```
### Defining the data
With a schema defined, we also define the data close to the root so it's
accessible through the unified configuration tree.
```bash
cat <<EOF > clusters.cue
```
```cue showLineNumbers
package holos
Clusters: #Clusters & {
// Management Cluster
management: region: "us-central1"
management: set: "management"
// Local Cluster
local: region: "us-west1"
// Some example clusters. Add new clusters to the Clusters struct like this.
e1: region: "us-east1"
e2: region: "us-east1"
e3: region: "us-east1"
w1: region: "us-west1"
w2: region: "us-west1"
w3: region: "us-west1"
}
// ClusterSets is dynamically built from the Clusters structure.
ClusterSets: #ClusterSets & {
// Map every cluster into the correct set.
for CLUSTER in Clusters {
(CLUSTER.set): clusters: (CLUSTER.name): CLUSTER
}
}
```
```bash
EOF
```
### Inspecting the data
We'll use the `holos cue` command to inspect the `ClusterSets` data structure we
just defined.
<Tabs groupId="9190BDAD-B4C5-4386-9C94-8E178AA6178A">
<TabItem value="command" label="Command">
```bash
holos cue export --expression ClusterSets --out=yaml ./
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
management:
name: management
clusters:
management:
name: management
region: us-central1
set: management
workload:
name: workload
clusters:
local:
name: local
region: us-west1
set: workload
e1:
name: e1
region: us-east1
set: workload
e2:
name: e2
region: us-east1
set: workload
e3:
name: e3
region: us-east1
set: workload
w1:
name: w1
region: us-west1
set: workload
w2:
name: w2
region: us-west1
set: workload
w3:
name: w3
region: us-west1
set: workload
```
</TabItem>
</Tabs>
This looks like our prototype, we're confident we can iterate over each cluster
in each set.
## Integrating Components
The `ClusterSets` data structure unlocks the capability to iterate over each
cluster in each cluster set. We'll use this capability to integrate the
`podinfo` component with each cluster in the platform.
### Configuring the Output directory
We need to configure `holos` to write output manifests into a cluster specific
output directory. We'll use the [ComponentConfig] `OutputBaseDir` field for
this purpose. We'll pass the value of this field as a component parameter.
```bash
cat <<EOF > componentconfig.cue
```
```cue showLineNumbers
package holos
#ComponentConfig: {
// Inject the output base directory from platform component parameters.
OutputBaseDir: string @tag(outputBaseDir, type=string)
}
```
```bash
EOF
```
### Integrating Podinfo
```bash
cat <<EOF >platform/podinfo.cue
```
```cue showLineNumbers
package holos
// Manage podinfo on all workload clusters.
for CLUSTER in ClusterSets.workload.clusters {
// We use the cluster name to disambiguate different podinfo build plans.
Platform: Components: "\(CLUSTER.name)-podinfo": {
name: "podinfo"
// Reuse the same component across multiple workload clusters.
path: "components/podinfo"
// Configure a cluster-unique message in the podinfo UI.
parameters: message: "Hello, I am cluster \(CLUSTER.name) in region \(CLUSTER.region)"
// Write to deploy/{outputBaseDir}/components/{name}/{name}.gen.yaml
parameters: outputBaseDir: "clusters/\(CLUSTER.name)"
}
}
```
```bash
EOF
```
## Rendering manifests
### Rendering the Platform
Render the platform to configure `podinfo` on each cluster.
<Tabs groupId="34A2D80B-0E86-4142-B65B-7DF70C47E1D2">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached podinfo 6.6.2
rendered podinfo in 164.278583ms
rendered podinfo in 165.48525ms
rendered podinfo in 165.186208ms
rendered podinfo in 165.831792ms
rendered podinfo in 166.845208ms
rendered podinfo in 167.000208ms
rendered podinfo in 167.012208ms
rendered platform in 167.06525ms
```
</TabItem>
</Tabs>
### Inspecting the Tree
Rendering the platform produces the following rendered manifests.
```bash
tree deploy
```
```txt showLineNumbers
deploy
└── clusters
├── e1
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── e2
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── e3
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── local
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── w1
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
├── w2
│   └── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── w3
└── components
└── podinfo
└── podinfo.gen.yaml
23 directories, 7 files
```
### Inspecting the Variation
Note how each component has slight variation using the component parameters.
```bash
diff -U2 deploy/clusters/{e,w}1/components/podinfo/podinfo.gen.yaml
```
```diff
--- deploy/clusters/e1/components/podinfo/podinfo.gen.yaml 2024-11-17 14:20:17
+++ deploy/clusters/w1/components/podinfo/podinfo.gen.yaml 2024-11-17 14:20:17
@@ -61,5 +61,5 @@
env:
- name: PODINFO_UI_MESSAGE
- value: Hello, I am cluster e1 in region us-east1
+ value: Hello, I am cluster w1 in region us-west1
- name: PODINFO_UI_COLOR
value: '#34577c'
```
## Concluding Remarks
In this topic we covered how to use CUE structures to organize multiple clusters
into various sets.
1. Clusters are defined in one place at the root of the configuration.
2. Clusters may be organized into sets by their purpose.
3. Most organizations have at least two sets, a set of workload clusters and a
set of management clusters.
4. Holos uses CUE, a super set of JSON. New clusters may be added by dropping a
JSON file into the root of the repository.
5. The pattern of defining a `#Cluster` and a `#Clusters` collection is a
general pattern. We'll see the same pattern for environments, projects, owners,
and more.
6. Component parameters are a flexible way to inject user defined configuration
from the platform level into a reusable component.
[ClusterSet]: https://multicluster.sigs.k8s.io/api-types/cluster-set/
[Environments]: ./environments.mdx
[Namespace Sameness - SIG Multicluster Position Statement]: https://github.com/kubernetes/community/blob/master/sig-multicluster/namespace-sameness-position-statement.md
[ComponentConfig]: ../../api/author.md#ComponentConfig

View File

@@ -0,0 +1,29 @@
---
slug: environments
title: Environments
description: Managing Environments - dev, test, stage, prod.
sidebar_position: 130
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Environments
## Overview
## The Code
### Generating the structure
### Using an example Component
## Defining Environments
### Defining one Environment
### Defining a collection of Environments
## Rendering manifests
## Reviewing the Manifests

View File

@@ -0,0 +1,25 @@
---
slug: .
title: Structures
description: Commonly used CUE structures.
sidebar_position: 120
---
import DocCardList from '@theme/DocCardList';
# Structures
This section has self contained articles covering commonly used CUE structures.
These structures are organized and presented as recipes you may adopt and adjust
to your unique organization.
:::important
Structures are defined by Holos Users, unlike the standardized [Core] and
[Author] schemas defined by the Holos Authors.
:::
---
<DocCardList />
[Core]: ../../api/core.md
[Author]: ../../api/author.md

View File

@@ -0,0 +1,29 @@
---
slug: owners
title: Owners
description: Managing and mapping projects to owners.
sidebar_position: 150
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Owners
## Overview
## The Code
### Generating the structure
### Using an example Component
## Defining Owners
### Defining one Owner
### Defining a collection of Owners
## Rendering manifests
## Reviewing the Manifests

View File

@@ -0,0 +1,29 @@
---
slug: projects
title: Projects
description: Managing components organizing them into projects.
sidebar_position: 140
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Projects
## Overview
## The Code
### Generating the structure
### Using an example Component
## Defining Projects
### Defining one Project
### Defining a collection of Projects
## Rendering manifests
## Reviewing the Manifests

View File

@@ -43,7 +43,9 @@ add the following CUE configuration to it.
```bash
mkdir -p components/podinfo
touch components/podinfo/podinfo.cue
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
@@ -66,11 +68,14 @@ Component: #Helm & {
}
}
```
```bash
EOF
```
Integrate the component with the platform.
```bash
touch platform/podinfo.cue
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
@@ -80,6 +85,9 @@ Platform: Components: podinfo: {
path: "components/podinfo"
}
```
```bash
EOF
```
Render the platform.
@@ -113,7 +121,7 @@ component kind. This field is a convenient wrapper around the core [BuildPlan]
Create the mixins.cue file.
```bash
touch components/podinfo/mixins.cue
cat <<EOF > components/podinfo/mixins.cue
```
```cue showLineNumbers
package holos
@@ -142,6 +150,9 @@ Component: {
}
}
```
```bash
EOF
```
:::important
Holos uses CUE to validate mixed in resources against a schema. The `Resources`

View File

@@ -109,11 +109,11 @@ initialization.
Start by creating a directory for the `podinfo` component. Create an empty file
and then add the following CUE configuration to it.
<Tabs groupId="tutorial-hello-podinfo-helm-cue-code">
<TabItem value="components/podinfo/podinfo.cue" label="Podinfo Helm Chart">
```bash
mkdir -p components/podinfo
touch components/podinfo/podinfo.cue
```
```bash
cat <<EOF > components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
@@ -140,8 +140,9 @@ HelmChart: #Helm & {
}
}
```
</TabItem>
</Tabs>
```bash
EOF
```
:::important
CUE loads all of `*.cue` files in the component directory to define component,
@@ -156,13 +157,11 @@ platform root directory. In this example, `#Helm` on line 6 is defined in
### Integrating the component
Integrate the `podinfo` component into the platform by creating a new cue file
Integrate the `podinfo` component into the platform by creating a new CUE file
in the `platform` directory with the following content.
<Tabs groupId="tutorial-hello-register-podinfo-component">
<TabItem value="platform/podinfo.cue" label="Register Podinfo">
```bash
touch platform/podinfo.cue
cat <<EOF > platform/podinfo.cue
```
```cue showLineNumbers
package holos
@@ -174,8 +173,9 @@ Platform: Components: podinfo: {
parameters: greeting: "Hello Holos!"
}
```
</TabItem>
</Tabs>
```bash
EOF
```
:::tip
Component parameters may have any name as long as they don't start with
@@ -348,7 +348,7 @@ grep -B2 Hello deploy/components/podinfo/podinfo.gen.yaml
## Breaking it down
We run `holos render platform ./platform` because the cue files in the platform
We run `holos render platform ./platform` because the CUE files in the platform
directory export a [Platform] resource to `holos`. The platform directory is
the entrypoint to the platform rendering process.
@@ -358,7 +358,7 @@ file integrates the `podinfo` Component with the Platform.
Holos requires two fields to integrate a component with the platform.
1. A unique name for the component.
2. The component path to the directory containing the cue files exporting a
2. The component path to the directory containing the CUE files exporting a
`BuildPlan` defining the component.
Component parameters are optional. They allow re-use of the same component.

View File

@@ -59,14 +59,12 @@ the following file contents.
```bash
mkdir -p components/prometheus components/blackbox
touch components/prometheus/prometheus.cue
touch components/blackbox/blackbox.cue
```
<Tabs groupId="D15A3008-1EFC-4D34-BED1-15BC0C736CC3">
<TabItem value="prometheus.cue" label="prometheus.cue">
```txt
components/prometheus/prometheus.cue
```bash
cat <<EOF > components/prometheus/prometheus.cue
```
```cue showLineNumbers
package holos
@@ -84,11 +82,14 @@ Helm: #Helm & {
}
}
}
```
```bash
EOF
```
</TabItem>
<TabItem value="blackbox.cue" label="blackbox.cue">
```txt
components/blackbox/blackbox.cue
```bash
cat <<EOF > components/blackbox/blackbox.cue
```
```cue showLineNumbers
package holos
@@ -106,6 +107,9 @@ Helm: #Helm & {
}
}
}
```
```bash
EOF
```
</TabItem>
</Tabs>
@@ -116,9 +120,8 @@ Integrate the components with the platform by adding the following file to the
platform directory.
```bash
touch platform/prometheus.cue
cat <<EOF > platform/prometheus.cue
```
```cue showLineNumbers
package holos
@@ -133,6 +136,9 @@ Platform: Components: {
}
}
```
```bash
EOF
```
Render the platform.
@@ -196,8 +202,8 @@ holos cue import \
components/blackbox/vendor/9.0.1/prometheus-blackbox-exporter/values.yaml
```
These command convert the YAML data into CUE code and nest the values under the
`Values` field of the `Holos` struct.
These commands convert the YAML data into CUE code and nest the values under the
`Values` field of the `Helm` struct.
:::important
CUE unifies `values.cue` with the other `*.cue` files in the same directory.
@@ -243,9 +249,8 @@ use. We add this configuration to the `components` directory so it's in scope
for all components.
```bash
touch components/blackbox.cue
cat <<EOF > components/blackbox.cue
```
```cue showLineNumbers
package holos
@@ -263,6 +268,9 @@ Blackbox: #Blackbox & {
port: 9115
}
```
```bash
EOF
```
:::important
1. CUE loads and unifies all `*.cue` files from the root directory containing

View File

@@ -61,11 +61,12 @@ Create the `httpbin` component directory and add the `httpbin.cue` and
<TabItem value="setup" label="Setup">
```bash
mkdir -p components/httpbin
touch components/httpbin/httpbin.cue
touch components/httpbin/httpbin.yaml
```
</TabItem>
<TabItem value="components/httpbin/httpbin.cue" label="httpbin.cue">
```bash
cat <<EOF > components/httpbin/httpbin.cue
```
```cue showLineNumbers
package holos
@@ -97,9 +98,15 @@ Kustomize: #Kustomize & {
}
}
}
```
```bash
EOF
```
</TabItem>
<TabItem value="components/httpbin/httpbin.yaml" label="httpbin.yaml">
```bash
cat <<EOF > components/httpbin/httpbin.yaml
```
```yaml showLineNumbers
# https://github.com/mccutchen/go-httpbin/blob/v2.15.0/kustomize/resources.yaml
apiVersion: apps/v1
@@ -137,6 +144,9 @@ spec:
protocol: TCP
name: http
appProtocol: http
```
```bash
EOF
```
</TabItem>
</Tabs>
@@ -150,9 +160,8 @@ Integrate `httpbin` with the platform by adding the following file to the
platform directory.
```bash
touch platform/httpbin.cue
cat <<EOF > platform/httpbin.cue
```
```cue showLineNumbers
package holos
@@ -163,6 +172,9 @@ Platform: Components: {
}
}
```
```bash
EOF
```
Render the platform.
@@ -275,13 +287,9 @@ makes this easier with CUE. We don't need to edit any yaml files.
Add a new `patches.cue` file to the `httpbin` component with the following
content.
<Tabs groupId="104D40FD-ED59-4F66-8B91-435436084743">
<TabItem value="touch" label="touch">
```bash
touch components/httpbin/patches.cue
cat <<EOF > components/httpbin/patches.cue
```
</TabItem>
<TabItem value="patches.cue" label="patches.cue">
```cue showLineNumbers
package holos
@@ -300,8 +308,9 @@ Kustomize: KustomizeConfig: Kustomization: _patches: {
}
}
```
</TabItem>
</Tabs>
```bash
EOF
```
:::note
We use a hidden `_patches` field to easily unify data into a struct, then

View File

@@ -1,13 +0,0 @@
---
slug: schema-definitions
title: Schema Definitions
description: Define your own custom data structures.
sidebar_position: 70
---
# Schema Definitions
- Work through defining a `#Cluster` schema and a `Clusters` struct.
- Direct the reader to [topics] for more recipes.
[topics]: ../topics.mdx

View File

@@ -17,22 +17,54 @@ This tutorial will guide you through the installation of Holos and its
dependencies, as well as the initialization of a minimal Platform that you can
extend to meet your specific needs.
## Installing Holos
## Installing
Holos is distributed as a single file executable that can be installed in a
couple of ways.
<Tabs groupId="FE2C74C8-B3A3-4AEA-BBD3-F57FAA654B6F">
<TabItem value="brew" label="Install with brew">
```bash
brew install holos-run/tap/holos
```
</TabItem>
<TabItem value="go" label="Go">
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
</TabItem>
</Tabs>
### Completion
<Tabs groupId="65F79D28-2E57-4A90-8EBA-3D8758C80233">
<TabItem value="zsh" label="zsh">
```bash
source <(holos completion zsh)
```
</TabItem>
<TabItem value="bash" label="bash">
```bash
source <(holos completion bash)
```
</TabItem>
<TabItem value="fish" label="fish">
```bash
source <(holos completion fish)
```
</TabItem>
<TabItem value="powershell" label="powershell">
```bash
holos completion powershell | Invoke-Expression
```
</TabItem>
</Tabs>
### Releases
Download `holos` from the [releases] page and place the executable into your
shell path.
### Go Install
```shell
go install github.com/holos-run/holos/cmd/holos@latest
```
### Dependencies
Holos integrates with the following tools that should be installed to enable
@@ -41,8 +73,12 @@ their functionality.
- [Helm] to fetch and render Helm chart Components.
- [Kubectl] to [kustomize] components.
Holos is tested with Helm version `v3.16.2`. Please try upgrading helm if you
encounter `Error: chart requires kubeVersion ...` errors.
:::note
Holos is tested with Helm version `v3.16.2`.
:::
Please try upgrading helm if you encounter `Error: chart requires kubeVersion
...` errors.
## Next Steps

15
hack/claude Normal file
View File

@@ -0,0 +1,15 @@
#! /bin/bash
TOPLEVEL="$(cd $(dirname "$0") && git rev-parse --show-toplevel)"
cd "${TOPLEVEL}"
# The text files use about 85% of the knowledge base.
mkdir -p tmp/claude
for x in $(git ls-files internal/\*.go api/author/v1alpha5/\*.go api/core/v1alpha5/\*.go doc/md/\*.md{,x}); do
y="${x//\//__}.txt"
[[ $y =~ "__ent__" ]] && continue
cp "$x" "tmp/claude/$y"
done

108
holos.go
View File

@@ -1,18 +1,6 @@
// Package holos defines types for the rest of the system.
package holos
import (
"bytes"
"context"
"encoding/json"
"fmt"
"log/slog"
"strings"
"cuelang.org/go/cue"
"github.com/holos-run/holos/internal/errors"
)
// A PathCueMod is a string representing the absolute filesystem path of a cue
// module. It is given a unique type so the API is clear.
type PathCueMod string
@@ -27,99 +15,3 @@ type FilePath string
// FileContent represents the contents of a file as a string.
type FileContent string
// TypeMeta represents the kind and version of a resource holos needs to
// process. Useful to discriminate generated resources.
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
// Builder builds file artifacts.
type Builder interface {
Build(context.Context, ArtifactMap) error
}
// ArtifactMap sets and gets data for file artifacts.
//
// Concrete values must ensure Set is write once, returning an error if a given
// FilePath was previously Set. Concrete values must be safe for concurrent
// reads and writes.
type ArtifactMap interface {
Get(path string) (data []byte, ok bool)
Set(path string, data []byte) error
Save(dir, path string) error
}
// Discriminator is useful to discriminate by type meta, the kind and api
// version of something.
type Discriminator interface {
Discriminate(ctx context.Context) (TypeMeta, error)
}
type Unifier interface {
Unify(ctx context.Context) (BuildData, error)
}
// BuildData represents the data necessary to produce a build plan. It is a
// convenience wrapper to store relevant fields to inform the user.
type BuildData struct {
Value cue.Value
ModuleRoot string
InstancePath InstancePath
Dir string
}
func (bd *BuildData) TypeMeta() (tm TypeMeta, err error) {
v, err := bd.value()
if err != nil {
return tm, errors.Wrap(err)
}
kind := v.LookupPath(cue.ParsePath("kind"))
if err := kind.Err(); err != nil {
return tm, errors.Wrap(err)
}
if tm.Kind, err = kind.String(); err != nil {
return tm, errors.Wrap(err)
}
version := v.LookupPath(cue.ParsePath("apiVersion"))
if err := version.Err(); err != nil {
return tm, errors.Wrap(err)
}
if tm.APIVersion, err = version.String(); err != nil {
return tm, errors.Wrap(err)
}
return
}
func (bd *BuildData) Decoder() (*json.Decoder, error) {
v, err := bd.value()
if err != nil {
return nil, errors.Wrap(err)
}
jsonBytes, err := v.MarshalJSON()
if err != nil {
return nil, errors.Wrap(err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
return decoder, nil
}
func (bd *BuildData) value() (v cue.Value, err error) {
v = bd.Value.LookupPath(cue.ParsePath("holos"))
if err := v.Err(); err != nil {
if strings.HasPrefix(err.Error(), "field not found") {
slog.Warn(fmt.Sprintf("%s: deprecated usage: nest output under holos: %s", err, bd.Dir), "err", err)
v = bd.Value
return v, nil
}
err = errors.Wrap(err)
return v, err
}
return
}

View File

@@ -9,21 +9,35 @@ import (
"github.com/holos-run/holos/internal/errors"
)
func New() *Artifact {
return &Artifact{m: make(map[string][]byte)}
// NewStore should provide a concrete Store.
var _ Store = NewStore()
// Store sets and gets data for file artifacts.
//
// Concrete values must ensure Set is write once, returning an error if a given
// FilePath was previously Set. Concrete values must be safe for concurrent
// reads and writes. Use [NewStore] to create a new concrete value.
type Store interface {
Get(path string) (data []byte, ok bool)
Set(path string, data []byte) error
Save(dir, path string) error
}
// Artifact represents the fully rendered manifests build from the holos
func NewStore() *MapStore {
return &MapStore{m: make(map[string][]byte)}
}
// MapStore represents the fully rendered manifests build from the holos
// rendering pipeline. Files are organized by keys representing paths relative
// to the current working directory. Values represent the file content.
type Artifact struct {
type MapStore struct {
mu sync.RWMutex
m map[string][]byte
}
// Set sets an artifact file with write locking. Set returns an error if the
// artifact was previously set.
func (a *Artifact) Set(path string, data []byte) error {
func (a *MapStore) Set(path string, data []byte) error {
a.mu.Lock()
defer a.mu.Unlock()
if _, ok := a.m[path]; ok {
@@ -34,7 +48,7 @@ func (a *Artifact) Set(path string, data []byte) error {
}
// Get gets the content of an artifact with read locking.
func (a *Artifact) Get(path string) (data []byte, ok bool) {
func (a *MapStore) Get(path string) (data []byte, ok bool) {
a.mu.RLock()
defer a.mu.RUnlock()
data, ok = a.m[path]
@@ -42,7 +56,7 @@ func (a *Artifact) Get(path string) (data []byte, ok bool) {
}
// Save writes a file to the filesystem.
func (a *Artifact) Save(dir, path string) error {
func (a *MapStore) Save(dir, path string) error {
fullPath := filepath.Join(dir, path)
msg := fmt.Sprintf("could not save %s", fullPath)
data, ok := a.Get(path)
@@ -58,7 +72,7 @@ func (a *Artifact) Save(dir, path string) error {
return nil
}
func (a *Artifact) Keys() []string {
func (a *MapStore) Keys() []string {
a.mu.RLock()
defer a.mu.RUnlock()
keys := make([]string, 0, len(a.m))

View File

@@ -1,504 +0,0 @@
// Package builder is responsible for building fully rendered kubernetes api
// objects from various input directories. A directory may contain a platform
// spec or a component spec.
package builder
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
"cuelang.org/go/cue/load"
"github.com/holos-run/holos"
core_v1alpha2 "github.com/holos-run/holos/api/core/v1alpha2"
core_v1alpha3 "github.com/holos-run/holos/api/core/v1alpha3"
meta_v1alpha2 "github.com/holos-run/holos/api/meta/v1alpha2"
"github.com/holos-run/holos/api/v1alpha1"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/render"
)
const (
KubernetesObjects = core_v1alpha3.KubernetesObjectsKind
// Helm is the value of the kind field of holos build output indicating helm
// values and helm command information.
Helm = core_v1alpha3.HelmChartKind
// Skip is the value when the instance should be skipped
Skip = "Skip"
// KustomizeBuild is the value of the kind field of cue output indicating
// holos should process the component using kustomize build to render output.
KustomizeBuild = v1alpha1.KustomizeBuildKind
)
// An Option configures a Builder
type Option func(*config)
type config struct {
args []string
cluster string
tags []string
}
type Builder struct {
cfg config
ctx *cue.Context
}
type buildPlanWrapper struct {
buildPlan *core_v1alpha3.BuildPlan
}
func (b *buildPlanWrapper) validate() error {
if b == nil {
return fmt.Errorf("invalid BuildPlan: is nil")
}
bp := b.buildPlan
if bp == nil {
return fmt.Errorf("invalid BuildPlan: is nil")
}
errs := make([]string, 0, 2)
if bp.Kind != core_v1alpha3.BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", v1alpha1.BuildPlanKind, bp.Kind))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}
return nil
}
func (b *buildPlanWrapper) resultCapacity() (count int) {
if b == nil {
return 0
}
bp := b.buildPlan
count = len(bp.Spec.Components.HelmChartList) +
len(bp.Spec.Components.KubernetesObjectsList) +
len(bp.Spec.Components.KustomizeBuildList) +
len(bp.Spec.Components.Resources)
return count
}
// New returns a new *Builder configured by opts Option.
func New(opts ...Option) *Builder {
var cfg config
for _, f := range opts {
f(&cfg)
}
b := &Builder{
cfg: cfg,
ctx: cuecontext.New(),
}
return b
}
// Entrypoints configures the leaf directories Builder builds.
func Entrypoints(args []string) Option {
return func(cfg *config) { cfg.args = args }
}
// Cluster configures the cluster name for the holos component instance.
func Cluster(name string) Option {
return func(cfg *config) { cfg.cluster = name }
}
// Tags configures tags to pass to cue when building the instance.
func Tags(tags []string) Option {
return func(cfg *config) { cfg.tags = tags }
}
// Cluster returns the cluster name of the component instance being built.
func (b *Builder) Cluster() string {
return b.cfg.cluster
}
func (b *Builder) Discriminate(ctx context.Context) (tm holos.TypeMeta, err error) {
cueModDir, err := b.findCueMod()
if err != nil {
err = errors.Wrap(err)
return
}
cueConfig := load.Config{
Dir: string(cueModDir),
ModuleRoot: string(cueModDir),
}
bd := &holos.BuildData{ModuleRoot: string(cueModDir)}
if len(b.cfg.args) > 1 {
return tm, errors.Wrap(errors.New("cannot provide more than one argument"))
}
// Make args relative to the module directory
args := make([]string, 0, len(b.cfg.args)+2)
for _, path := range b.cfg.args {
target, err := filepath.Abs(path)
if err != nil {
return tm, errors.Wrap(fmt.Errorf("could not find absolute path: %w", err))
}
relPath, err := filepath.Rel(bd.ModuleRoot, target)
if err != nil {
return tm, errors.Wrap(fmt.Errorf("invalid argument, must be relative to cue.mod: %w", err))
}
bd.InstancePath = holos.InstancePath(target)
bd.Dir = relPath
relPath = "./" + relPath
args = append(args, relPath)
}
instances := load.Instances(args, &cueConfig)
values, err := b.ctx.BuildInstances(instances)
if err != nil {
return tm, errors.Wrap(err)
}
bd.Value = values[0]
tm, err = bd.TypeMeta()
return
}
// Unify returns a cue.Value representing the kind of build holos is meant to
// execute. This function unifies a cue package entrypoint with
// platform.config.json and user data json files located recursively within the
// userdata directory at the cue module root.
//
// Deprecated: use Discriminate instead.
func (b *Builder) Unify(ctx context.Context, cfg *client.Config) (bd holos.BuildData, err error) {
// Ensure the value is from the same runtime, otherwise cue panics.
bd.Value = b.ctx.CompileString("")
cueModDir, err := b.findCueMod()
if err != nil {
err = errors.Wrap(err)
return
}
bd.ModuleRoot = string(cueModDir)
platformConfigData, err := os.ReadFile(filepath.Join(bd.ModuleRoot, client.PlatformConfigFile))
if err != nil {
return bd, errors.Wrap(fmt.Errorf("could not load platform model: %w", err))
}
// TODO(jeff): Changing these tag names breaks backwards compatibility. We
// need to refactor this unification into a versioned builder, at least at the
// component level. Right now it's executed when rendering the initial
// Platform spec, which should be backwards compatible but isn't because this
// package is shared by all versions.
tags := make([]string, 0, len(b.cfg.tags)+2)
// TODO: Use instance.FillPath to fill the platform config.
// Refer to https://pkg.go.dev/cuelang.org/go/cue#Value.FillPath
tags = append(tags, "holos_platform_config="+string(platformConfigData))
// TODO(jeff): This is hacky after I switched to reserved holos_ tag names in
// v1alpha4. Could use some serious clean up now that --cluster-name is
// deprecated for --inject holos_cluster=foo, but it was kind of nice to have
// a required argument.
if cluster := cfg.Holos().ClusterName(); cluster != "" {
tags = append(tags, "holos_cluster="+cluster)
}
tags = append(tags, b.cfg.tags...)
cueConfig := load.Config{
Dir: bd.ModuleRoot,
ModuleRoot: bd.ModuleRoot,
Tags: tags,
}
// Make args relative to the module directory
args := make([]string, 0, len(b.cfg.args)+2)
for _, path := range b.cfg.args {
target, err := filepath.Abs(path)
if err != nil {
return bd, errors.Wrap(fmt.Errorf("could not find absolute path: %w", err))
}
relPath, err := filepath.Rel(bd.ModuleRoot, target)
if err != nil {
return bd, errors.Wrap(fmt.Errorf("invalid argument, must be relative to cue.mod: %w", err))
}
// WATCH OUT: Assumes only one instance path is provided via args, which is
// true when I added this, but may be a poor assumption by the time you read
// this.
bd.InstancePath = holos.InstancePath(target)
bd.Dir = relPath
relPath = "./" + relPath
args = append(args, relPath)
}
instances := load.Instances(args, &cueConfig)
values, err := b.ctx.BuildInstances(instances)
if err != nil {
err = errors.Wrap(err)
return
}
// Unify into a single Value
for _, v := range values {
bd.Value = bd.Value.Unify(v)
}
// Fill in #UserData
userData, err := loadUserData(b.ctx, bd.ModuleRoot)
if err != nil {
err = errors.Wrap(err)
return
}
bd.Value = bd.Value.FillPath(cue.ParsePath("#UserData"), userData)
return
}
// loadUserData recursively unifies userdata/**/*.json files into cue.Value val.
func loadUserData(ctx *cue.Context, moduleRoot string) (val cue.Value, err error) {
// Ensure the value is from the same runtime, otherwise cue panics.
val = ctx.CompileString("")
userdataPath := filepath.Join(moduleRoot, "userdata")
if err = os.MkdirAll(userdataPath, 0755); err != nil {
return val, errors.Wrap(err)
}
err = filepath.Walk(userdataPath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return errors.Wrap(err)
}
if !info.IsDir() && filepath.Ext(info.Name()) == ".json" {
userData, err := os.ReadFile(path)
if err != nil {
return errors.Wrap(err)
}
val = val.Unify(ctx.CompileBytes(userData, cue.Filename(path)))
}
return nil
})
return val, errors.Wrap(err)
}
// Run builds the cue entrypoint into zero or more Results. Exactly one CUE
// package entrypoint is expected in the args slice. The platform config is
// provided to the entrypoint through a json encoded string tag named
// platform_config. The resulting cue.Value is unified with all user data files
// at the path "#UserData".
//
// Deprecated: Use holos.Builder instead
func (b *Builder) Run(ctx context.Context, cfg *client.Config) (results []*render.Result, err error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building instances")
bd, err := b.Unify(ctx, cfg)
if err != nil {
return nil, err
}
return b.build(ctx, bd)
}
func (b *Builder) build(ctx context.Context, bd holos.BuildData) (results []*render.Result, err error) {
log := logger.FromContext(ctx).With("dir", bd.InstancePath)
value := bd.Value
if err := value.Err(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not build %s: %w", bd.InstancePath, err))
}
log.DebugContext(ctx, "cue: validating instance")
if err := value.Validate(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not validate: %w", err))
}
log.DebugContext(ctx, "cue: decoding holos build plan")
jsonBytes, err := value.MarshalJSON()
if err != nil {
return nil, errors.Wrap(fmt.Errorf("could not marshal cue instance %s: %w", bd.Dir, err))
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
// Discriminate the type of build plan.
tm := &v1alpha1.TypeMeta{}
err = decoder.Decode(tm)
if err != nil {
return nil, errors.Wrap(fmt.Errorf("invalid BuildPlan: %s: %w", bd.Dir, err))
}
log.DebugContext(ctx, "cue: discriminated build kind: "+tm.Kind, "kind", tm.Kind, "apiVersion", tm.APIVersion)
// New decoder for the full object
decoder = json.NewDecoder(bytes.NewReader(jsonBytes))
// TODO: When we release v1, explicitly allow unknown fields so we can add
// fields without needing to bump the major version. Disallow until we reach
// v1 for clear error reporting.
decoder.DisallowUnknownFields()
switch tm.Kind {
case "BuildPlan":
var bp core_v1alpha3.BuildPlan
if err = decoder.Decode(&bp); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode BuildPlan %s: %w", bd.Dir, err))
return
}
results, err = b.buildPlan(ctx, &bp, bd.InstancePath)
if err != nil {
return results, err
}
default:
err = errors.Wrap(fmt.Errorf("unknown kind: %v", tm.Kind))
}
return results, err
}
func (b *Builder) buildPlan(ctx context.Context, buildPlan *core_v1alpha3.BuildPlan, path holos.InstancePath) (results []*render.Result, err error) {
log := logger.FromContext(ctx)
bpw := buildPlanWrapper{buildPlan: buildPlan}
if err := bpw.validate(); err != nil {
log.WarnContext(ctx, "could not validate", "skipped", true, "err", err)
return nil, errors.Wrap(fmt.Errorf("could not validate %w", err))
}
if buildPlan.Spec.Disabled {
log.DebugContext(ctx, "skipped: spec.disabled is true", "skipped", true)
return
}
results = make([]*render.Result, 0, bpw.resultCapacity())
log.DebugContext(ctx, "allocated results slice", "cap", bpw.resultCapacity())
for _, component := range buildPlan.Spec.Components.Resources {
ko := render.KubernetesObjects{Component: component}
if result, err := ko.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
for _, component := range buildPlan.Spec.Components.KubernetesObjectsList {
ko := render.KubernetesObjects{Component: component}
if result, err := ko.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
for _, component := range buildPlan.Spec.Components.HelmChartList {
hc := render.HelmChart{Component: component}
if result, err := hc.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
for _, component := range buildPlan.Spec.Components.KustomizeBuildList {
kb := render.KustomizeBuild{Component: component}
if result, err := kb.Render(ctx, path); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not render: %w", err))
} else {
results = append(results, result)
}
}
log.DebugContext(ctx, "returning results", "len", len(results))
return results, nil
}
// findCueMod returns the root module location containing the cue.mod file or
// directory or an error if the builder arguments do not share a common root
// module.
func (b *Builder) findCueMod() (dir holos.PathCueMod, err error) {
for _, origPath := range b.cfg.args {
absPath, err := filepath.Abs(origPath)
if err != nil {
return "", err
}
path := holos.PathCueMod(absPath)
for {
if _, err := os.Stat(filepath.Join(string(path), "cue.mod")); err == nil {
if dir != "" && dir != path {
return "", fmt.Errorf("multiple modules not supported: %v is not %v", dir, path)
}
dir = path
break
} else if !os.IsNotExist(err) {
return "", err
}
parentPath := holos.PathCueMod(filepath.Dir(string(path)))
if parentPath == path {
return "", fmt.Errorf("no cue.mod from root to leaf: %v", origPath)
}
path = parentPath
}
}
return dir, nil
}
// Platform builds a platform
// TODO: Refactor, lift up into NewPlatform RunE.
func (b *Builder) Platform(ctx context.Context, cfg *client.Config) (*core_v1alpha2.Platform, error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building platform instance")
bd, err := b.Unify(ctx, cfg)
if err != nil {
return nil, errors.Wrap(err)
}
return b.runPlatform(ctx, bd)
}
func (b *Builder) runPlatform(ctx context.Context, bd holos.BuildData) (*core_v1alpha2.Platform, error) {
path := holos.InstancePath(bd.Dir)
log := logger.FromContext(ctx).With("dir", path)
value := bd.Value
if err := bd.Value.Err(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not load: %w", err))
}
log.DebugContext(ctx, "cue: validating instance")
if err := value.Validate(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not validate: %w", err))
}
log.DebugContext(ctx, "cue: decoding holos platform")
jsonBytes, err := value.MarshalJSON()
if err != nil {
return nil, errors.Wrap(fmt.Errorf("could not marshal cue instance %s: %w", bd.Dir, err))
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
// Discriminate the type of build plan.
tm := &meta_v1alpha2.TypeMeta{}
err = decoder.Decode(tm)
if err != nil {
return nil, errors.Wrap(fmt.Errorf("invalid platform: %s: %w", bd.Dir, err))
}
log.DebugContext(ctx, "cue: discriminated build kind: "+tm.GetKind(), "kind", tm.GetKind(), "apiVersion", tm.GetAPIVersion())
decoder = json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
var pf core_v1alpha2.Platform
switch tm.GetKind() {
case "Platform":
if err = decoder.Decode(&pf); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode platform %s: %w", bd.Dir, err))
return nil, err
}
return &pf, nil
default:
err = errors.Wrap(fmt.Errorf("unknown kind: %v", tm.GetKind()))
}
return nil, err
}

View File

@@ -0,0 +1,44 @@
package builder
import (
"github.com/holos-run/holos/internal/builder/v1alpha5"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
)
type BuildPlan struct {
holos.BuildPlan
}
func LoadBuildPlan(i *Instance, opts holos.BuildOpts) (bp BuildPlan, err error) {
err = i.Discriminate(func(tm holos.TypeMeta) error {
if tm.Kind != "BuildPlan" {
return errors.Format("unsupported kind: %s, want BuildPlan", tm.Kind)
}
switch version := tm.APIVersion; version {
case "v1alpha5":
bp = BuildPlan{&v1alpha5.BuildPlan{Opts: opts}}
default:
return errors.Format("unsupported version: %s", version)
}
return nil
})
if err != nil {
return bp, errors.Wrap(err)
}
// Get the holos: field value from cue.
v, err := i.HolosValue()
if err != nil {
return bp, errors.Wrap(err)
}
// Load the platform from the cue value.
if err := bp.Load(v); err != nil {
return bp, errors.Wrap(err)
}
return bp, err
}

View File

@@ -0,0 +1,135 @@
package builder
import (
"bytes"
"encoding/json"
"fmt"
"log/slog"
"strings"
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
"cuelang.org/go/cue/load"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
"github.com/holos-run/holos/internal/util"
)
func LoadInstance(path string, tags []string) (*Instance, error) {
root, leaf, err := util.FindRootLeaf(path)
if err != nil {
return nil, errors.Wrap(err)
}
cfg := &load.Config{
Dir: root,
ModuleRoot: root,
Tags: tags,
}
ctx := cuecontext.New()
instances := load.Instances([]string{leaf}, cfg)
values, err := ctx.BuildInstances(instances)
if err != nil {
return nil, errors.Wrap(err)
}
inst := &Instance{
path: leaf,
ctx: ctx,
cfg: cfg,
value: values[0],
}
return inst, nil
}
// Instance represents a cue instance to build. Use LoadInstance to create a
// new Instance.
type Instance struct {
path string
ctx *cue.Context
cfg *load.Config
value cue.Value
}
// HolosValue returns the value of the holos field of the exported CUE instance.
func (i *Instance) HolosValue() (v cue.Value, err error) {
v = i.value.LookupPath(cue.ParsePath("holos"))
if err = v.Err(); err != nil {
if strings.HasPrefix(err.Error(), "field not found") {
slog.Warn(fmt.Sprintf("%s: deprecated usage: nest output under holos: %s", err, i.path), "err", err)
// Return the deprecated value at the root
return i.value, nil
}
err = errors.Wrap(err)
}
return
}
// Discriminate calls the discriminate func for side effects. Useful to switch
// over the instance kind and apiVersion.
func (i *Instance) Discriminate(discriminate func(tm holos.TypeMeta) error) error {
v, err := i.HolosValue()
if err != nil {
return errors.Wrap(err)
}
var tm holos.TypeMeta
kind := v.LookupPath(cue.ParsePath("kind"))
if err := kind.Err(); err != nil {
return errors.Wrap(err)
}
if tm.Kind, err = kind.String(); err != nil {
return errors.Wrap(err)
}
version := v.LookupPath(cue.ParsePath("apiVersion"))
if err := version.Err(); err != nil {
return errors.Wrap(err)
}
if tm.APIVersion, err = version.String(); err != nil {
return errors.Wrap(err)
}
if err := discriminate(tm); err != nil {
return errors.Wrap(err)
}
return nil
}
func (i *Instance) Decoder() (*json.Decoder, error) {
v, err := i.HolosValue()
if err != nil {
return nil, errors.Wrap(err)
}
jsonBytes, err := v.MarshalJSON()
if err != nil {
return nil, errors.Wrap(err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
return decoder, nil
}
func (i *Instance) Export(enc holos.Encoder) error {
v, err := i.HolosValue()
if err != nil {
return errors.Wrap(err)
}
var data interface{}
if err := v.Decode(&data); err != nil {
return errors.Wrap(err)
}
if err := enc.Encode(&data); err != nil {
return errors.Wrap(err)
}
return nil
}

View File

@@ -0,0 +1,126 @@
package builder
import (
"context"
"fmt"
"time"
"github.com/holos-run/holos/internal/builder/v1alpha5"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
"github.com/holos-run/holos/internal/logger"
"golang.org/x/sync/errgroup"
)
// PlatformOpts represents build options when processing the components in a
// platform.
type PlatformOpts struct {
Fn BuildFunc
Selector holos.Selector
Concurrency int
InfoEnabled bool
}
// Platform represents a common abstraction over different platform schema
// versions.
type Platform struct {
holos.Platform
}
// Build calls [PlatformOpts] Fn(ctx, component) concurrently.
func (p *Platform) Build(ctx context.Context, opts PlatformOpts) error {
limit := max(opts.Concurrency, 1)
parentStart := time.Now()
components := p.Select(opts.Selector)
total := len(components)
g, ctx := errgroup.WithContext(ctx)
// Limit the number of concurrent goroutines due to CUE memory usage concerns
// while rendering components. One more for the producer.
g.SetLimit(limit + 1)
// Spawn a producer because g.Go() blocks when the group limit is reached.
g.Go(func() error {
for idx := range components {
select {
case <-ctx.Done():
return errors.Wrap(ctx.Err())
default:
// Capture idx to avoid issues with closure. Fixed in Go 1.22.
idx := idx
component := components[idx]
// Worker go routine. Blocks if limit has been reached.
g.Go(func() error {
select {
case <-ctx.Done():
return errors.Wrap(ctx.Err())
default:
start := time.Now()
log := logger.FromContext(ctx).With("num", idx+1, "total", total)
if err := opts.Fn(ctx, idx, component); err != nil {
return errors.Wrap(err)
}
duration := time.Since(start)
msg := fmt.Sprintf("rendered %s in %s", component.Describe(), duration)
if opts.InfoEnabled {
log.InfoContext(ctx, msg, "duration", duration)
} else {
log.DebugContext(ctx, msg, "duration", duration)
}
return nil
}
})
}
}
return nil
})
// Wait for completion and return the first error (if any)
if err := g.Wait(); err != nil {
return err
}
duration := time.Since(parentStart)
msg := fmt.Sprintf("rendered platform in %s", duration)
if opts.InfoEnabled {
logger.FromContext(ctx).InfoContext(ctx, msg, "duration", duration)
} else {
logger.FromContext(ctx).DebugContext(ctx, msg, "duration", duration)
}
return nil
}
// BuildFunc is executed concurrently when processing platform components.
type BuildFunc func(context.Context, int, holos.Component) error
func LoadPlatform(i *Instance) (platform Platform, err error) {
err = i.Discriminate(func(tm holos.TypeMeta) error {
if tm.Kind != "Platform" {
return errors.Format("unsupported kind: %s, want Platform", tm.Kind)
}
switch version := tm.APIVersion; version {
case "v1alpha5":
platform = Platform{&v1alpha5.Platform{}}
default:
return errors.Format("unsupported version: %s", version)
}
return nil
})
if err != nil {
return platform, errors.Wrap(err)
}
// Get the holos: field value from cue.
v, err := i.HolosValue()
if err != nil {
return platform, errors.Wrap(err)
}
// Load the platform from the cue value.
if err := platform.Load(v); err != nil {
return platform, errors.Wrap(err)
}
return platform, err
}

View File

@@ -1,550 +0,0 @@
package v1alpha4
import (
"bytes"
"context"
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
"strings"
"syscall"
"time"
h "github.com/holos-run/holos"
"github.com/holos-run/holos/api/core/v1alpha4"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
"golang.org/x/sync/errgroup"
"gopkg.in/yaml.v3"
)
// Platform represents a platform builder.
type Platform struct {
Platform v1alpha4.Platform
Concurrency int
Stderr io.Writer
}
// Build builds a Platform by concurrently building a BuildPlan for each
// platform component. No artifact files are written directly, only indirectly
// by rendering each component.
func (p *Platform) Build(ctx context.Context, _ h.ArtifactMap) error {
parentStart := time.Now()
components := p.Platform.Spec.Components
total := len(components)
g, ctx := errgroup.WithContext(ctx)
// Limit the number of concurrent goroutines due to CUE memory usage concerns
// while rendering components. One more for the producer.
g.SetLimit(p.Concurrency + 1)
// Spawn a producer because g.Go() blocks when the group limit is reached.
g.Go(func() error {
for idx := range components {
select {
case <-ctx.Done():
return ctx.Err()
default:
// Capture idx to avoid issues with closure. Fixed in Go 1.22.
idx := idx
component := &components[idx]
// Worker go routine. Blocks if limit has been reached.
g.Go(func() error {
select {
case <-ctx.Done():
return ctx.Err()
default:
start := time.Now()
log := logger.FromContext(ctx).With(
"name", component.Name,
"path", component.Component,
"cluster", component.Cluster,
"num", idx+1,
"total", total,
)
log.DebugContext(ctx, "render component")
tags := make([]string, 0, 3+len(component.Tags))
tags = append(tags, "holos_name="+component.Name)
tags = append(tags, "holos_component="+component.Component)
tags = append(tags, "holos_cluster="+component.Cluster)
for key, value := range component.Tags {
tags = append(tags, fmt.Sprintf("%s=%s", key, value))
}
// Execute a sub-process to limit CUE memory usage.
args := make([]string, 0, 10)
args = append(args,
"render",
"component",
)
for _, tag := range tags {
args = append(args, "--inject", tag)
}
if component.WriteTo != "" {
args = append(args, "--write-to", component.WriteTo)
}
args = append(args, component.Component)
result, err := util.RunCmd(ctx, "holos", args...)
// I've lost an hour+ digging into why I couldn't see log output
// from sub-processes. Make sure to surface at least stderr from
// sub-processes.
_, _ = io.Copy(p.Stderr, result.Stderr)
if err != nil {
return errors.Wrap(fmt.Errorf("could not render component: %w", err))
}
duration := time.Since(start)
msg := fmt.Sprintf(
"rendered %s for cluster %s in %s",
component.Name,
component.Cluster,
duration,
)
log.InfoContext(ctx, msg, "duration", duration)
return nil
}
})
}
}
return nil
})
// Wait for completion and return the first error (if any)
if err := g.Wait(); err != nil {
return err
}
duration := time.Since(parentStart)
msg := fmt.Sprintf("rendered platform in %s", duration)
logger.FromContext(ctx).InfoContext(ctx, msg, "duration", duration, "version", p.Platform.APIVersion)
return nil
}
// BuildPlan represents a component builder.
type BuildPlan struct {
BuildPlan v1alpha4.BuildPlan
Concurrency int
Stderr io.Writer
// WriteTo --write-to=deploy flag
WriteTo string
// Path represents the path to the component
Path h.InstancePath
}
// Build builds a BuildPlan into Artifact files.
func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
name := b.BuildPlan.Metadata.Name
component := b.BuildPlan.Spec.Component
log := logger.FromContext(ctx).With("name", name, "component", component)
msg := fmt.Sprintf("could not build %s", name)
if b.BuildPlan.Spec.Disabled {
log.WarnContext(ctx, fmt.Sprintf("%s: disabled", msg))
return nil
}
g, ctx := errgroup.WithContext(ctx)
// One more for the producer
g.SetLimit(b.Concurrency + 1)
// Producer.
g.Go(func() error {
for _, a := range b.BuildPlan.Spec.Artifacts {
msg := fmt.Sprintf("%s artifact %s", msg, a.Artifact)
log := log.With("artifact", a.Artifact)
if a.Skip {
log.WarnContext(ctx, fmt.Sprintf("%s: skipped field is true", msg))
continue
}
select {
case <-ctx.Done():
return ctx.Err()
default:
// https://golang.org/doc/faq#closures_and_goroutines
a := a
// Worker. Blocks if limit has been reached.
g.Go(func() error {
for _, gen := range a.Generators {
switch gen.Kind {
case "Resources":
if err := b.resources(log, gen, am); err != nil {
return errors.Format("could not generate resources: %w", err)
}
case "Helm":
if err := b.helm(ctx, log, gen, am); err != nil {
return errors.Format("could not generate helm: %w", err)
}
case "File":
if err := b.file(log, gen, am); err != nil {
return errors.Format("could not generate file: %w", err)
}
default:
return errors.Format("%s: unsupported kind %s", msg, gen.Kind)
}
}
for _, t := range a.Transformers {
switch t.Kind {
case "Kustomize":
if err := b.kustomize(ctx, log, t, am); err != nil {
return errors.Wrap(err)
}
case "Join":
s := make([][]byte, 0, len(t.Inputs))
for _, input := range t.Inputs {
if data, ok := am.Get(string(input)); ok {
s = append(s, data)
} else {
return errors.Format("%s: missing %s", msg, input)
}
}
data := bytes.Join(s, []byte(t.Join.Separator))
if err := am.Set(string(t.Output), data); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact: " + string(t.Output))
default:
return errors.Format("%s: unsupported kind %s", msg, t.Kind)
}
}
// Write the final artifact
if err := am.Save(b.WriteTo, string(a.Artifact)); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+filepath.Join(b.WriteTo, string(a.Artifact)))
return nil
})
}
}
return nil
})
// Wait for completion and return the first error (if any)
return g.Wait()
}
func (b *BuildPlan) file(
log *slog.Logger,
g v1alpha4.Generator,
am h.ArtifactMap,
) error {
data, err := os.ReadFile(filepath.Join(string(b.Path), string(g.File.Source)))
if err != nil {
return errors.Wrap(err)
}
if err := am.Set(string(g.Output), data); err != nil {
return errors.Wrap(err)
}
log.Debug("set artifact: " + string(g.Output))
return nil
}
func (b *BuildPlan) helm(
ctx context.Context,
log *slog.Logger,
g v1alpha4.Generator,
am h.ArtifactMap,
) error {
chartName := g.Helm.Chart.Name
log = log.With("chart", chartName)
// Unnecessary? cargo cult copied from internal/cli/render/render.go
if chartName == "" {
return errors.New("missing chart name")
}
// Cache the chart by version to pull new versions. (#273)
cacheDir := filepath.Join(string(b.Path), "vendor", g.Helm.Chart.Version)
cachePath := filepath.Join(cacheDir, filepath.Base(chartName))
if _, err := os.Stat(cachePath); os.IsNotExist(err) {
timeout, cancel := context.WithTimeout(ctx, 5*time.Minute)
defer cancel()
err := onceWithLock(log, timeout, cachePath, func() error {
return b.cacheChart(ctx, log, cacheDir, g.Helm.Chart)
})
if err != nil {
return errors.Format("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos.helm")
if err != nil {
return errors.Format("could not make temp dir: %w", err)
}
defer util.Remove(ctx, tempDir)
data, err := yaml.Marshal(g.Helm.Values)
if err != nil {
return errors.Format("could not marshal values: %w", err)
}
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, data, 0666); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "wrote"+valuesPath)
// Run charts
args := []string{"template"}
if !g.Helm.EnableHooks {
args = append(args, "--no-hooks")
}
args = append(args,
"--include-crds",
"--values", valuesPath,
"--namespace", g.Helm.Namespace,
"--kubeconfig", "/dev/null",
"--version", g.Helm.Chart.Version,
g.Helm.Chart.Release,
cachePath,
)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Format("could not run helm template: %w", err)
}
// Set the artifact
if err := am.Set(string(g.Output), helmOut.Stdout.Bytes()); err != nil {
return errors.Format("could not store helm output: %w", err)
}
log.Debug("set artifact: " + string(g.Output))
return nil
}
func (b *BuildPlan) resources(
log *slog.Logger,
g v1alpha4.Generator,
am h.ArtifactMap,
) error {
var size int
for _, m := range g.Resources {
size += len(m)
}
list := make([]v1alpha4.Resource, 0, size)
for _, m := range g.Resources {
for _, r := range m {
list = append(list, r)
}
}
msg := fmt.Sprintf("could not generate %s for %s path %s", g.Output, b.BuildPlan.Metadata.Name, b.BuildPlan.Spec.Component)
buf, err := marshal(list)
if err != nil {
return errors.Format("%s: %w", msg, err)
}
if err := am.Set(string(g.Output), buf.Bytes()); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact " + string(g.Output))
return nil
}
func (b *BuildPlan) kustomize(
ctx context.Context,
log *slog.Logger,
t v1alpha4.Transformer,
am h.ArtifactMap,
) error {
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
msg := fmt.Sprintf("could not transform %s for %s path %s", t.Output, b.BuildPlan.Metadata.Name, b.BuildPlan.Spec.Component)
// Write the kustomization
data, err := yaml.Marshal(t.Kustomize.Kustomization)
if err != nil {
return errors.Format("%s: %w", msg, err)
}
path := filepath.Join(tempDir, "kustomization.yaml")
if err := os.WriteFile(path, data, 0666); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+path)
// Write the inputs
for _, input := range t.Inputs {
path := string(input)
if err := am.Save(tempDir, path); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+filepath.Join(tempDir, path))
}
// Execute kustomize
r, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
kErr := r.Stderr.String()
err = errors.Format("%s: could not run kustomize: %w", msg, err)
log.ErrorContext(ctx, fmt.Sprintf("%s: stderr:\n%s", err.Error(), kErr), "err", err, "stderr", kErr)
return err
}
// Store the artifact
if err := am.Set(string(t.Output), r.Stdout.Bytes()); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact " + string(t.Output))
return nil
}
func marshal(list []v1alpha4.Resource) (buf bytes.Buffer, err error) {
encoder := yaml.NewEncoder(&buf)
defer encoder.Close()
for _, item := range list {
if err = encoder.Encode(item); err != nil {
err = errors.Wrap(err)
return
}
}
return
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// We assume the only method responsible for writing to chartDir is cacheChart
// itself. cacheChart runs concurrently when rendering a platform.
//
// We rely on the atomicity of moving temporary directories into place on the
// same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is
// expected when this function is called concurrently.
//
// TODO(jeff): Break the dependency on v1alpha4, make it work across versions as
// a utility function.
func (b *BuildPlan) cacheChart(
ctx context.Context,
log *slog.Logger,
cacheDir string,
chart v1alpha4.Chart,
) error {
// Add repositories
repo := chart.Repository
if repo.URL == "" {
// repo update not needed for oci charts so this is debug instead of warn.
log.DebugContext(ctx, "skipped helm repo add and update: repo url is empty")
} else {
if r, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL); err != nil {
_, _ = io.Copy(b.Stderr, r.Stderr)
return errors.Format("could not run helm repo add: %w", err)
}
if r, err := util.RunCmd(ctx, "helm", "repo", "update", repo.Name); err != nil {
_, _ = io.Copy(b.Stderr, r.Stderr)
return errors.Format("could not run helm repo update: %w", err)
}
}
cacheTemp, err := os.MkdirTemp(cacheDir, chart.Name)
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, cacheTemp)
cn := chart.Name
if chart.Repository.Name != "" {
cn = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, cn)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
if len(items) != 1 {
return errors.Format("want: exactly one item, have: %+v", items)
}
item := items[0]
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cacheDir, chart.Name)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", dst)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
} else {
log.DebugContext(ctx, fmt.Sprintf("renamed %s to %s", src, dst), "src", src, "dst", dst)
}
log.InfoContext(ctx,
fmt.Sprintf("cached %s %s", chart.Name, chart.Version),
"chart", chart.Name,
"chart_version", chart.Version,
"path", dst,
)
return nil
}
// onceWithLock obtains a filesystem lock with mkdir, then executes fn. If the
// lock is already locked, onceWithLock waits for it to be released then returns
// without calling fn.
func onceWithLock(log *slog.Logger, ctx context.Context, path string, fn func() error) error {
if err := os.MkdirAll(filepath.Dir(path), 0777); err != nil {
return errors.Wrap(err)
}
// Obtain a lock with a timeout.
lockDir := path + ".lock"
log = log.With("lock", lockDir)
err := os.Mkdir(lockDir, 0777)
if err == nil {
defer os.RemoveAll(lockDir)
log.DebugContext(ctx, fmt.Sprintf("acquired %s", lockDir))
if err := fn(); err != nil {
return errors.Wrap(err)
}
log.DebugContext(ctx, fmt.Sprintf("released %s", lockDir))
return nil
}
// Wait until the lock is released then return.
if os.IsExist(err) {
log.DebugContext(ctx, fmt.Sprintf("blocked %s", lockDir))
stillBlocked := time.After(5 * time.Second)
deadLocked := time.After(10 * time.Second)
for {
select {
case <-stillBlocked:
log.WarnContext(ctx, fmt.Sprintf("waiting for %s to be released", lockDir))
case <-deadLocked:
log.WarnContext(ctx, fmt.Sprintf("still waiting for %s to be released (dead lock?)", lockDir))
case <-time.After(100 * time.Millisecond):
if _, err := os.Stat(lockDir); os.IsNotExist(err) {
log.DebugContext(ctx, fmt.Sprintf("unblocked %s", lockDir))
return nil
}
case <-ctx.Done():
return errors.Wrap(ctx.Err())
}
}
}
// Unexpected error
return errors.Wrap(err)
}

View File

@@ -3,8 +3,8 @@ package v1alpha5
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
@@ -13,170 +13,111 @@ import (
"time"
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
"cuelang.org/go/cue/load"
"github.com/holos-run/holos"
h "github.com/holos-run/holos"
core "github.com/holos-run/holos/api/core/v1alpha5"
"github.com/holos-run/holos/internal/artifact"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
"golang.org/x/sync/errgroup"
"gopkg.in/yaml.v3"
)
func Unify(cueCtx *cue.Context, path string, tags []string) (bd holos.BuildData, err error) {
if bd.ModuleRoot, bd.Dir, err = util.FindRootLeaf(path); err != nil {
return bd, errors.Wrap(err)
}
cueConfig := load.Config{
Dir: bd.ModuleRoot,
ModuleRoot: bd.ModuleRoot,
Tags: tags,
}
args := []string{bd.Dir}
instances := load.Instances(args, &cueConfig)
v := cueCtx.BuildInstance(instances[0])
if err = v.Err(); err != nil {
return bd, errors.Wrap(err)
}
bd.Value = v
return
}
func LoadPlatform(path string, tags []string) (*Platform, error) {
bd, err := Unify(cuecontext.New(), path, tags)
if err != nil {
return nil, errors.Wrap(err)
}
decoder, err := bd.Decoder()
if err != nil {
return nil, errors.Wrap(err)
}
var platform Platform
if err := decoder.Decode(&platform.Platform); err != nil {
return nil, errors.Wrap(err)
}
return &platform, nil
}
// Platform represents a platform builder.
type Platform struct {
Platform core.Platform
Concurrency int
Stderr io.Writer
Platform core.Platform
}
// Build builds a Platform by concurrently building a BuildPlan for each
// platform component. No artifact files are written directly, only indirectly
// by rendering each component.
func (p *Platform) Build(ctx context.Context, _ h.ArtifactMap) error {
parentStart := time.Now()
components := p.Platform.Spec.Components
total := len(components)
g, ctx := errgroup.WithContext(ctx)
// Limit the number of concurrent goroutines due to CUE memory usage concerns
// while rendering components. One more for the producer.
g.SetLimit(p.Concurrency + 1)
// Spawn a producer because g.Go() blocks when the group limit is reached.
g.Go(func() error {
for idx := range components {
select {
case <-ctx.Done():
return ctx.Err()
default:
// Capture idx to avoid issues with closure. Fixed in Go 1.22.
idx := idx
component := &components[idx]
// Worker go routine. Blocks if limit has been reached.
g.Go(func() error {
select {
case <-ctx.Done():
return ctx.Err()
default:
start := time.Now()
log := logger.FromContext(ctx).With(
"name", component.Name,
"path", component.Path,
"num", idx+1,
"total", total,
)
log.DebugContext(ctx, "render component")
// Load loads from a cue value.
func (p *Platform) Load(v cue.Value) error {
return errors.Wrap(v.Decode(&p.Platform))
}
tags := make([]string, 0, 2+len(component.Parameters))
tags = append(tags, "holos_component_name="+component.Name)
tags = append(tags, "holos_component_path="+component.Path)
for key, value := range component.Parameters {
tags = append(tags, fmt.Sprintf("%s=%s", key, value))
}
// Execute a sub-process to limit CUE memory usage.
args := make([]string, 0, 10)
args = append(args,
"render",
"component",
)
for _, tag := range tags {
args = append(args, "--inject", tag)
}
if component.WriteTo != "" {
args = append(args, "--write-to", component.WriteTo)
}
args = append(args, component.Path)
result, err := util.RunCmd(ctx, "holos", args...)
// I've lost an hour+ digging into why I couldn't see log output
// from sub-processes. Make sure to surface at least stderr from
// sub-processes.
_, _ = io.Copy(p.Stderr, result.Stderr)
if err != nil {
return errors.Wrap(fmt.Errorf("could not render component: %w", err))
}
duration := time.Since(start)
msg := fmt.Sprintf(
"rendered %s in %s",
component.Name,
duration,
)
log.InfoContext(ctx, msg, "duration", duration)
return nil
}
})
}
}
return nil
})
// Wait for completion and return the first error (if any)
if err := g.Wait(); err != nil {
return err
func (p *Platform) Export(encoder holos.Encoder) error {
if err := encoder.Encode(&p.Platform); err != nil {
return errors.Wrap(err)
}
duration := time.Since(parentStart)
msg := fmt.Sprintf("rendered platform in %s", duration)
logger.FromContext(ctx).InfoContext(ctx, msg, "duration", duration, "version", p.Platform.APIVersion)
return nil
}
func (p *Platform) Select(selectors ...holos.Selector) []holos.Component {
components := make([]holos.Component, 0, len(p.Platform.Spec.Components))
for _, component := range p.Platform.Spec.Components {
if holos.IsSelected(component.Labels, selectors...) {
components = append(components, &Component{component})
}
}
return components
}
type Component struct {
Component core.Component
}
func (c *Component) Describe() string {
if val, ok := c.Component.Annotations["app.holos.run/description"]; ok {
return val
}
return c.Component.Name
}
func (c *Component) Tags() ([]string, error) {
size := 2 +
len(c.Component.Parameters) +
len(c.Component.Labels) +
len(c.Component.Annotations)
tags := make([]string, 0, size)
for k, v := range c.Component.Parameters {
tags = append(tags, k+"="+v)
}
// Inject holos component metadata tags.
tags = append(tags, "holos_component_name="+c.Component.Name)
tags = append(tags, "holos_component_path="+c.Component.Path)
if len(c.Component.Labels) > 0 {
labels, err := json.Marshal(c.Component.Labels)
if err != nil {
return nil, err
}
tags = append(tags, "holos_component_labels="+string(labels))
}
if len(c.Component.Annotations) > 0 {
annotations, err := json.Marshal(c.Component.Annotations)
if err != nil {
return nil, err
}
tags = append(tags, "holos_component_annotations="+string(annotations))
}
return tags, nil
}
func (c *Component) WriteTo() string {
return c.Component.WriteTo
}
func (c *Component) Labels() holos.Labels {
return c.Component.Labels
}
func (c *Component) Path() string {
return util.DotSlash(c.Component.Path)
}
var _ holos.BuildPlan = &BuildPlan{}
// BuildPlan represents a component builder.
type BuildPlan struct {
BuildPlan core.BuildPlan
Concurrency int
Stderr io.Writer
// WriteTo --write-to=deploy flag
WriteTo string
// Path represents the path to the component
Path h.InstancePath
core.BuildPlan
Opts holos.BuildOpts
}
// Build builds a BuildPlan into Artifact files.
func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
func (b *BuildPlan) Build(ctx context.Context) error {
name := b.BuildPlan.Metadata.Name
path := b.BuildPlan.Source.Component.Path
path := b.Opts.Path
log := logger.FromContext(ctx).With("name", name, "path", path)
msg := fmt.Sprintf("could not build %s", name)
if b.BuildPlan.Spec.Disabled {
@@ -186,7 +127,7 @@ func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
g, ctx := errgroup.WithContext(ctx)
// One more for the producer
g.SetLimit(b.Concurrency + 1)
g.SetLimit(b.Opts.Concurrency + 1)
// Producer.
g.Go(func() error {
@@ -208,15 +149,15 @@ func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
for _, gen := range a.Generators {
switch gen.Kind {
case "Resources":
if err := b.resources(log, gen, am); err != nil {
if err := b.resources(log, gen, b.Opts.Store); err != nil {
return errors.Format("could not generate resources: %w", err)
}
case "Helm":
if err := b.helm(ctx, log, gen, am); err != nil {
if err := b.helm(ctx, log, gen, b.Opts.Store); err != nil {
return errors.Format("could not generate helm: %w", err)
}
case "File":
if err := b.file(log, gen, am); err != nil {
if err := b.file(log, gen, b.Opts.Store); err != nil {
return errors.Format("could not generate file: %w", err)
}
default:
@@ -227,20 +168,20 @@ func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
for _, t := range a.Transformers {
switch t.Kind {
case "Kustomize":
if err := b.kustomize(ctx, log, t, am); err != nil {
if err := b.kustomize(ctx, log, t, b.Opts.Store); err != nil {
return errors.Wrap(err)
}
case "Join":
s := make([][]byte, 0, len(t.Inputs))
for _, input := range t.Inputs {
if data, ok := am.Get(string(input)); ok {
if data, ok := b.Opts.Store.Get(string(input)); ok {
s = append(s, data)
} else {
return errors.Format("%s: missing %s", msg, input)
}
}
data := bytes.Join(s, []byte(t.Join.Separator))
if err := am.Set(string(t.Output), data); err != nil {
if err := b.Opts.Store.Set(string(t.Output), data); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact: " + string(t.Output))
@@ -250,10 +191,10 @@ func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
}
// Write the final artifact
if err := am.Save(b.WriteTo, string(a.Artifact)); err != nil {
if err := b.Opts.Store.Save(b.Opts.WriteTo, string(a.Artifact)); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+filepath.Join(b.WriteTo, string(a.Artifact)))
log.DebugContext(ctx, "wrote "+filepath.Join(b.Opts.WriteTo, string(a.Artifact)))
return nil
})
@@ -266,16 +207,27 @@ func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
return g.Wait()
}
func (b *BuildPlan) Export(idx int, encoder holos.OrderedEncoder) error {
if err := encoder.Encode(idx, &b.BuildPlan); err != nil {
return errors.Wrap(err)
}
return nil
}
func (b *BuildPlan) Load(v cue.Value) error {
return errors.Wrap(v.Decode(&b.BuildPlan))
}
func (b *BuildPlan) file(
log *slog.Logger,
g core.Generator,
am h.ArtifactMap,
store artifact.Store,
) error {
data, err := os.ReadFile(filepath.Join(string(b.Path), string(g.File.Source)))
data, err := os.ReadFile(filepath.Join(string(b.Opts.Path), string(g.File.Source)))
if err != nil {
return errors.Wrap(err)
}
if err := am.Set(string(g.Output), data); err != nil {
if err := store.Set(string(g.Output), data); err != nil {
return errors.Wrap(err)
}
log.Debug("set artifact: " + string(g.Output))
@@ -286,7 +238,7 @@ func (b *BuildPlan) helm(
ctx context.Context,
log *slog.Logger,
g core.Generator,
am h.ArtifactMap,
store artifact.Store,
) error {
chartName := g.Helm.Chart.Name
log = log.With("chart", chartName)
@@ -296,7 +248,7 @@ func (b *BuildPlan) helm(
}
// Cache the chart by version to pull new versions. (#273)
cacheDir := filepath.Join(string(b.Path), "vendor", g.Helm.Chart.Version)
cacheDir := filepath.Join(string(b.Opts.Path), "vendor", g.Helm.Chart.Version)
cachePath := filepath.Join(cacheDir, filepath.Base(chartName))
if _, err := os.Stat(cachePath); os.IsNotExist(err) {
@@ -362,7 +314,7 @@ func (b *BuildPlan) helm(
}
// Set the artifact
if err := am.Set(string(g.Output), helmOut.Stdout.Bytes()); err != nil {
if err := store.Set(string(g.Output), helmOut.Stdout.Bytes()); err != nil {
return errors.Format("could not store helm output: %w", err)
}
log.Debug("set artifact: " + string(g.Output))
@@ -373,7 +325,7 @@ func (b *BuildPlan) helm(
func (b *BuildPlan) resources(
log *slog.Logger,
g core.Generator,
am h.ArtifactMap,
store artifact.Store,
) error {
var size int
for _, m := range g.Resources {
@@ -391,7 +343,7 @@ func (b *BuildPlan) resources(
"could not generate %s for %s path %s",
g.Output,
b.BuildPlan.Metadata.Name,
b.BuildPlan.Source.Component.Path,
b.Opts.Path,
)
buf, err := marshal(list)
@@ -399,7 +351,7 @@ func (b *BuildPlan) resources(
return errors.Format("%s: %w", msg, err)
}
if err := am.Set(string(g.Output), buf.Bytes()); err != nil {
if err := store.Set(string(g.Output), buf.Bytes()); err != nil {
return errors.Format("%s: %w", msg, err)
}
@@ -411,7 +363,7 @@ func (b *BuildPlan) kustomize(
ctx context.Context,
log *slog.Logger,
t core.Transformer,
am h.ArtifactMap,
store artifact.Store,
) error {
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
@@ -422,7 +374,7 @@ func (b *BuildPlan) kustomize(
"could not transform %s for %s path %s",
t.Output,
b.BuildPlan.Metadata.Name,
b.BuildPlan.Source.Component.Path,
b.Opts.Path,
)
// Write the kustomization
@@ -439,7 +391,7 @@ func (b *BuildPlan) kustomize(
// Write the inputs
for _, input := range t.Inputs {
path := string(input)
if err := am.Save(tempDir, path); err != nil {
if err := store.Save(tempDir, path); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+filepath.Join(tempDir, path))
@@ -455,7 +407,7 @@ func (b *BuildPlan) kustomize(
}
// Store the artifact
if err := am.Set(string(t.Output), r.Stdout.Bytes()); err != nil {
if err := store.Set(string(t.Output), r.Stdout.Bytes()); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact " + string(t.Output))
@@ -495,16 +447,15 @@ func (b *BuildPlan) cacheChart(
) error {
// Add repositories
repo := chart.Repository
stderr := b.Opts.Stderr
if repo.URL == "" {
// repo update not needed for oci charts so this is debug instead of warn.
log.DebugContext(ctx, "skipped helm repo add and update: repo url is empty")
} else {
if r, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL); err != nil {
_, _ = io.Copy(b.Stderr, r.Stderr)
if _, err := util.RunCmdW(ctx, stderr, "helm", "repo", "add", repo.Name, repo.URL); err != nil {
return errors.Format("could not run helm repo add: %w", err)
}
if r, err := util.RunCmd(ctx, "helm", "repo", "update", repo.Name); err != nil {
_, _ = io.Copy(b.Stderr, r.Stderr)
if _, err := util.RunCmdW(ctx, stderr, "helm", "repo", "update", repo.Name); err != nil {
return errors.Format("could not run helm repo update: %w", err)
}
}
@@ -519,7 +470,7 @@ func (b *BuildPlan) cacheChart(
if chart.Repository.Name != "" {
cn = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, cn)
helmOut, err := util.RunCmdW(ctx, stderr, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, cn)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")

View File

@@ -1,60 +1 @@
package build
import (
"fmt"
"log/slog"
"strings"
"github.com/holos-run/holos/internal/builder"
"github.com/holos-run/holos/internal/cli/command"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
"github.com/holos-run/holos/internal/server/middleware/logger"
"github.com/spf13/cobra"
)
// makeBuildRunFunc returns the internal implementation of the build cli command
func makeBuildRunFunc(cfg *client.Config) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
logger.FromContext(ctx).DebugContext(ctx, "RunE", "args", args)
build := builder.New(builder.Entrypoints(args), builder.Cluster(cfg.Holos().ClusterName()))
//nolint:staticcheck
results, err := build.Run(ctx, cfg)
if err != nil {
return err
}
outs := make([]string, 0, len(results))
for idx, result := range results {
if result.Continue() {
slog.Debug("skip result", "idx", idx, "result", result)
continue
}
slog.Debug("append result", "idx", idx, "result.kind", result.Kind)
outs = append(outs, result.AccumulatedOutput())
}
out := strings.Join(outs, "---\n")
if _, err := fmt.Fprintln(cmd.OutOrStdout(), out); err != nil {
return errors.Wrap(err)
}
return nil
}
}
// New returns the build subcommand for the root command
func New(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
cmd := command.New("build DIRECTORY")
cmd.Hidden = !feature.Flag(holos.BuildFeature)
cmd.Args = cobra.ExactArgs(1)
cmd.Short = "write kubernetes manifests to standard output"
cmd.Example = " holos build components/argo/crds"
cmd.Flags().AddGoFlagSet(cfg.ClusterFlagSet())
config := client.NewConfig(cfg)
cmd.PersistentFlags().AddGoFlagSet(config.ClientFlagSet())
cmd.PersistentFlags().AddGoFlagSet(config.TokenFlagSet())
cmd.RunE = makeBuildRunFunc(config)
return cmd
}

View File

@@ -0,0 +1,4 @@
Show BuildPlans produced from Platform.spec.components
1. Selectors are applied to the Platform.spec.components list.
2. Results are output in the same order as listed in the Platform spec.

View File

@@ -32,6 +32,7 @@ func HandleError(ctx context.Context, err error, hc *holos.Config) (exitCode int
log := hc.NewTopLevelLogger().With("code", connect.CodeOf(err))
var cueErr cue.Error
var errAt *errors.ErrorAt
if errors.As(err, &errAt) {
loc := errAt.Source.Loc()
err2 := errAt.Unwrap()
@@ -39,10 +40,13 @@ func HandleError(ctx context.Context, err error, hc *holos.Config) (exitCode int
} else {
log.ErrorContext(ctx, fmt.Sprintf("could not run: %s", err), "err", err)
}
// cue errors are bundled up as a list and refer to multiple files / lines.
if errors.As(err, &cueErr) {
msg := cue.Details(cueErr, nil)
_, _ = fmt.Fprint(hc.Stderr(), msg)
if _, err := fmt.Fprint(hc.Stderr(), msg); err != nil {
log.ErrorContext(ctx, "could not write CUE error details: "+err.Error(), "err", err)
}
}
// connect errors have details and codes.
// Refer to https://connectrpc.com/docs/go/errors

View File

@@ -1,26 +1,18 @@
package render
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"io"
"runtime"
"strings"
"cuelang.org/go/cue/cuecontext"
h "github.com/holos-run/holos"
"github.com/holos-run/holos/internal/artifact"
"github.com/holos-run/holos/internal/builder"
"github.com/holos-run/holos/internal/builder/v1alpha4"
"github.com/holos-run/holos/internal/builder/v1alpha5"
"github.com/holos-run/holos/internal/cli/command"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/render"
"github.com/holos-run/holos/internal/util"
"github.com/spf13/cobra"
)
@@ -28,13 +20,74 @@ func New(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
cmd := command.New("render")
cmd.Args = cobra.NoArgs
cmd.Short = "render platforms and components to manifest files"
cmd.AddCommand(NewComponent(cfg))
cmd.AddCommand(NewPlatform(cfg))
cmd.AddCommand(newPlatform(cfg, feature))
cmd.AddCommand(newComponent(cfg, feature))
return cmd
}
func newPlatform(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
cmd := command.New("platform DIRECTORY")
cmd.Args = cobra.MaximumNArgs(1)
cmd.Example = "holos render platform"
cmd.Short = "render an entire platform"
config := client.NewConfig(cfg)
if feature.Flag(holos.ClientFeature) {
cmd.PersistentFlags().AddGoFlagSet(config.ClientFlagSet())
cmd.PersistentFlags().AddGoFlagSet(config.TokenFlagSet())
}
var concurrency int
cmd.Flags().IntVar(&concurrency, "concurrency", min(runtime.NumCPU(), 8), "number of components to render concurrently")
var platform string
cmd.Flags().StringVar(&platform, "platform", "./platform", "platform directory path")
var selector holos.Selector
cmd.Flags().VarP(&selector, "selector", "l", "label selector (e.g. label==string,label!=string)")
tagMap := make(holos.TagMap)
cmd.Flags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
log := logger.FromContext(ctx)
if len(args) > 0 {
platform = args[0]
msg := "deprecated: %s, use the --platform flag instead"
log.WarnContext(ctx, fmt.Sprintf(msg, platform))
}
inst, err := builder.LoadInstance(platform, tagMap.Tags())
if err != nil {
return errors.Wrap(err)
}
platform, err := builder.LoadPlatform(inst)
if err != nil {
return errors.Wrap(err)
}
prefixArgs := []string{
"--log-level", cfg.LogConfig().Level(),
"--log-format", cfg.LogConfig().Format(),
}
opts := builder.PlatformOpts{
Fn: makePlatformRenderFunc(cmd.ErrOrStderr(), prefixArgs),
Selector: selector,
Concurrency: concurrency,
InfoEnabled: true,
}
if err := platform.Build(ctx, opts); err != nil {
return errors.Wrap(err)
}
return nil
}
return cmd
}
// New returns the component subcommand for the render command
func NewComponent(cfg *holos.Config) *cobra.Command {
func newComponent(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
cmd := command.New("component DIRECTORY")
cmd.Args = cobra.ExactArgs(1)
cmd.Short = "render a platform component"
@@ -43,132 +96,37 @@ func NewComponent(cfg *holos.Config) *cobra.Command {
cmd.Flags().AddGoFlagSet(cfg.ClusterFlagSet())
config := client.NewConfig(cfg)
cmd.PersistentFlags().AddGoFlagSet(config.ClientFlagSet())
cmd.PersistentFlags().AddGoFlagSet(config.TokenFlagSet())
flagSet := flag.NewFlagSet("", flag.ContinueOnError)
tagMap := make(tags)
cmd.PersistentFlags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
if feature.Flag(holos.ClientFeature) {
cmd.PersistentFlags().AddGoFlagSet(config.ClientFlagSet())
cmd.PersistentFlags().AddGoFlagSet(config.TokenFlagSet())
}
tagMap := make(holos.TagMap)
cmd.Flags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
var concurrency int
flagSet.IntVar(&concurrency, "concurrency", min(runtime.NumCPU(), 8), "number of concurrent build steps")
cmd.Flags().AddGoFlagSet(flagSet)
cmd.Flags().IntVar(&concurrency, "concurrency", min(runtime.NumCPU(), 8), "number of concurrent build steps")
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
log := logger.FromContext(ctx)
build := builder.New(builder.Entrypoints(args))
tm, err := build.Discriminate(ctx)
if err != nil {
return errors.Wrap(err)
}
if tm.Kind != "BuildPlan" {
return errors.Format("invalid kind: want: BuildPlan have: %s", tm.Kind)
}
log.DebugContext(ctx, fmt.Sprintf("discriminated %s %s", tm.APIVersion, tm.Kind))
path := args[0]
switch tm.APIVersion {
case "v1alpha5":
builder := v1alpha5.BuildPlan{
Concurrency: concurrency,
Stderr: cmd.ErrOrStderr(),
WriteTo: cfg.WriteTo(),
Path: h.InstancePath(path),
}
bd, err := v1alpha5.Unify(cuecontext.New(), path, tagMap.Tags())
if err != nil {
return errors.Wrap(err)
}
decoder, err := bd.Decoder()
if err != nil {
return errors.Wrap(err)
}
if err := decoder.Decode(&builder.BuildPlan); err != nil {
return errors.Format("could not decode build plan %s: %w", bd.Dir, err)
}
// Process the BuildPlan.
return render.Component(ctx, &builder, artifact.New())
}
// This is the old way of doing it prior to v1alpha5 and should be removed
// before v1.
build = builder.New(
builder.Entrypoints(args),
builder.Cluster(cfg.ClusterName()),
builder.Tags(tagMap.Tags()),
)
log.DebugContext(ctx, "cue: building component instance")
//nolint:staticcheck
bd, err := build.Unify(ctx, config)
inst, err := builder.LoadInstance(path, tagMap.Tags())
if err != nil {
return errors.Wrap(err)
}
jsonBytes, err := bd.Value.MarshalJSON()
opts := holos.NewBuildOpts(path)
opts.Stderr = cmd.ErrOrStderr()
opts.Concurrency = concurrency
opts.WriteTo = cfg.WriteTo()
bp, err := builder.LoadBuildPlan(inst, opts)
if err != nil {
return errors.Format("could not marshal json %s: %w", bd.Dir, err)
return errors.Wrap(err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
switch tm.APIVersion {
case "v1alpha4":
builder := v1alpha4.BuildPlan{
WriteTo: cfg.WriteTo(),
Concurrency: concurrency,
Stderr: cmd.ErrOrStderr(),
Path: h.InstancePath(args[0]),
}
if err := decoder.Decode(&builder.BuildPlan); err != nil {
return errors.Format("could not decode build plan %s: %w", bd.Dir, err)
}
return render.Component(ctx, &builder, artifact.New())
// Legacy method.
case "v1alpha3", "v1alpha2", "v1alpha1":
//nolint:staticcheck
results, err := build.Run(ctx, config)
if err != nil {
return errors.Wrap(err)
}
// TODO: Avoid accidental over-writes if two or more holos component
// instances result in the same file path. Write files into a blank
// temporary directory, error if a file exists, then move the directory into
// place.
var result Result
for _, result = range results {
log := logger.FromContext(ctx).With(
"cluster", cfg.ClusterName(),
"name", result.Name(),
)
if result.Continue() {
continue
}
// DeployFiles from the BuildPlan
if err := result.WriteDeployFiles(ctx, cfg.WriteTo()); err != nil {
return errors.Wrap(err)
}
// API Objects
if result.SkipWriteAccumulatedOutput() {
log.DebugContext(ctx, "skipped writing k8s objects for "+result.Name())
} else {
path := result.Filename(cfg.WriteTo(), cfg.ClusterName())
if err := result.Save(ctx, path, result.AccumulatedOutput()); err != nil {
return errors.Wrap(err)
}
}
log.InfoContext(ctx, "rendered "+result.Name(), "status", "ok", "action", "rendered")
}
default:
return errors.Format("component version not supported: %s", tm.APIVersion)
if err := bp.Build(ctx); err != nil {
return errors.Wrap(err)
}
return nil
@@ -176,133 +134,27 @@ func NewComponent(cfg *holos.Config) *cobra.Command {
return cmd
}
func NewPlatform(cfg *holos.Config) *cobra.Command {
cmd := command.New("platform DIRECTORY")
cmd.Args = cobra.ExactArgs(1)
cmd.Example = " holos render platform ./platform"
cmd.Short = "render an entire platform"
config := client.NewConfig(cfg)
cmd.PersistentFlags().AddGoFlagSet(config.ClientFlagSet())
cmd.PersistentFlags().AddGoFlagSet(config.TokenFlagSet())
var concurrency int
cmd.Flags().IntVar(&concurrency, "concurrency", min(runtime.NumCPU(), 8), "number of components to render concurrently")
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: discriminating platform instance")
build := builder.New(builder.Entrypoints(args))
tm, err := build.Discriminate(ctx)
if err != nil {
return errors.Wrap(err)
}
if tm.Kind != "Platform" {
return errors.Format("invalid kind: want: Platform have: %s", tm.Kind)
}
log.DebugContext(ctx, fmt.Sprintf("discriminated %s %s", tm.APIVersion, tm.Kind))
switch version := tm.APIVersion; version {
case "v1alpha5":
builder, err := v1alpha5.LoadPlatform(args[0], nil)
if err != nil {
return errors.Wrap(err)
}
builder.Concurrency = concurrency
builder.Stderr = cmd.ErrOrStderr()
return render.Platform(ctx, builder)
}
// Prior to v1alpha5 we fully unified and injected tags, which was a bad
// idea because it assumed certain tags would always be passed, like
// cluster, which we made optional in v1alpha5.
log.DebugContext(ctx, "cue: building platform instance")
//nolint:staticcheck
bd, err := build.Unify(ctx, config)
if err != nil {
return errors.Wrap(err)
}
jsonBytes, err := bd.Value.MarshalJSON()
if err != nil {
return errors.Format("could not marshal json %s: %w", bd.Dir, err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
switch version := tm.APIVersion; version {
case "v1alpha4":
builder := v1alpha4.Platform{
Concurrency: concurrency,
Stderr: cmd.ErrOrStderr(),
}
if err := decoder.Decode(&builder.Platform); err != nil {
return errors.Format("could not decode platform %s: %w", bd.Dir, err)
}
return render.Platform(ctx, &builder)
// Legacy versions prior to the render.Builder interface.
case "v1alpha3", "v1alpha2", "v1alpha1":
platform, err := build.Platform(ctx, config)
if err != nil {
return errors.Wrap(err)
}
//nolint:staticcheck
return render.LegacyPlatform(ctx, concurrency, platform, cmd.ErrOrStderr())
func makePlatformRenderFunc(w io.Writer, prefixArgs []string) builder.BuildFunc {
return func(ctx context.Context, idx int, component holos.Component) error {
select {
case <-ctx.Done():
return errors.Wrap(ctx.Err())
default:
return errors.Format("platform version not supported: %s", version)
tags, err := component.Tags()
if err != nil {
return errors.Wrap(err)
}
args := make([]string, 0, 10+len(prefixArgs)+(len(tags)*2))
args = append(args, prefixArgs...)
args = append(args, "render", "component")
for _, tag := range tags {
args = append(args, "--inject", tag)
}
args = append(args, component.Path())
if _, err := util.RunCmdW(ctx, w, "holos", args...); err != nil {
return errors.Format("could not render component: %w", err)
}
}
return nil
}
return cmd
}
// tags represents a map of key values for CUE tags for flag parsing.
type tags map[string]string
func (t tags) Tags() []string {
parts := make([]string, 0, len(t))
for k, v := range t {
parts = append(parts, fmt.Sprintf("%s=%s", k, v))
}
return parts
}
func (t tags) String() string {
return strings.Join(t.Tags(), " ")
}
// Set sets a value. Only one value per flag is supported. For example
// --inject=foo=bar --inject=bar=baz. For JSON values, --inject=foo=bar,bar=baz
// is not supported.
func (t tags) Set(value string) error {
parts := strings.SplitN(value, "=", 2)
if len(parts) != 2 {
return errors.Format("invalid format, must be tag=value")
}
t[parts[0]] = parts[1]
return nil
}
func (t tags) Type() string {
return "strings"
}
// Deprecated: use render.Artifact instead.
type Result interface {
Continue() bool
Name() string
Filename(writeTo string, cluster string) string
KustomizationFilename(writeTo string, cluster string) string
Save(ctx context.Context, path string, content string) error
AccumulatedOutput() string
SkipWriteAccumulatedOutput() bool
WriteDeployFiles(ctx context.Context, writeTo string) error
GetKind() string
GetAPIVersion() string
}

View File

@@ -14,7 +14,6 @@ import (
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/server"
"github.com/holos-run/holos/internal/cli/build"
"github.com/holos-run/holos/internal/cli/command"
"github.com/holos-run/holos/internal/cli/create"
"github.com/holos-run/holos/internal/cli/destroy"
@@ -74,7 +73,6 @@ func New(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
rootCmd.PersistentFlags().Lookup("help").Hidden = true
// subcommands
rootCmd.AddCommand(build.New(cfg, feature))
rootCmd.AddCommand(render.New(cfg, feature))
rootCmd.AddCommand(get.New(cfg, feature))
rootCmd.AddCommand(create.New(cfg, feature))
@@ -101,6 +99,9 @@ func New(cfg *holos.Config, feature holos.Flagger) *cobra.Command {
// CUE
rootCmd.AddCommand(newCueCmd())
// Show
rootCmd.AddCommand(newShowCmd())
return rootCmd
}

138
internal/cli/show.go Normal file
View File

@@ -0,0 +1,138 @@
package cli
import (
"context"
_ "embed"
"runtime"
"github.com/holos-run/holos/internal/builder"
"github.com/holos-run/holos/internal/cli/command"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/holos"
"github.com/spf13/cobra"
)
//go:embed long-show-buildplans.txt
var longShowBuildPlansHelp string
func newShowCmd() (cmd *cobra.Command) {
cmd = command.New("show")
cmd.Short = "show a platform or build plans"
cmd.AddCommand(newShowPlatformCmd())
cmd.AddCommand(newShowBuildPlanCmd())
return cmd
}
func newShowPlatformCmd() (cmd *cobra.Command) {
cmd = command.New("platform")
cmd.Short = "show a platform"
cmd.Args = cobra.NoArgs
var platform string
cmd.Flags().StringVar(&platform, "platform", "./platform", "platform directory path")
var format string
cmd.Flags().StringVar(&format, "format", "yaml", "yaml or json format")
tagMap := make(holos.TagMap)
cmd.Flags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
cmd.RunE = func(c *cobra.Command, args []string) (err error) {
inst, err := builder.LoadInstance(platform, tagMap.Tags())
if err != nil {
return errors.Wrap(err)
}
encoder, err := holos.NewEncoder(format, cmd.OutOrStdout())
if err != nil {
return errors.Wrap(err)
}
defer encoder.Close()
if err := inst.Export(encoder); err != nil {
return errors.Wrap(err)
}
return nil
}
return cmd
}
func newShowBuildPlanCmd() (cmd *cobra.Command) {
cmd = command.New("buildplans")
cmd.Aliases = []string{"buildplan", "components", "component"}
cmd.Short = "show buildplans"
cmd.Long = longShowBuildPlansHelp
cmd.Args = cobra.MinimumNArgs(0)
var platform string
cmd.Flags().StringVar(&platform, "platform", "./platform", "platform directory path")
var format string
cmd.Flags().StringVar(&format, "format", "yaml", "yaml or json format")
var selector holos.Selector
cmd.Flags().VarP(&selector, "selector", "l", "label selector (e.g. label==string,label!=string)")
tagMap := make(holos.TagMap)
cmd.Flags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
var concurrency int
cmd.Flags().IntVar(&concurrency, "concurrency", min(runtime.NumCPU(), 8), "number of concurrent build steps")
cmd.RunE = func(c *cobra.Command, args []string) (err error) {
path := platform
inst, err := builder.LoadInstance(path, tagMap.Tags())
if err != nil {
return errors.Wrap(err)
}
platform, err := builder.LoadPlatform(inst)
if err != nil {
return errors.Wrap(err)
}
encoder, err := holos.NewSequentialEncoder(format, cmd.OutOrStdout())
if err != nil {
return errors.Wrap(err)
}
defer encoder.Close()
buildPlanOpts := holos.NewBuildOpts(path)
buildPlanOpts.Stderr = cmd.ErrOrStderr()
buildPlanOpts.Concurrency = concurrency
platformOpts := builder.PlatformOpts{
Fn: makeBuildFunc(encoder, buildPlanOpts),
Selector: selector,
Concurrency: concurrency,
}
if err := platform.Build(c.Context(), platformOpts); err != nil {
return errors.Wrap(err)
}
return nil
}
return cmd
}
func makeBuildFunc(encoder holos.OrderedEncoder, opts holos.BuildOpts) builder.BuildFunc {
return func(ctx context.Context, idx int, component holos.Component) error {
select {
case <-ctx.Done():
return errors.Wrap(ctx.Err())
default:
tags, err := component.Tags()
if err != nil {
return errors.Wrap(err)
}
inst, err := builder.LoadInstance(component.Path(), tags)
if err != nil {
return errors.Wrap(err)
}
bp, err := builder.LoadBuildPlan(inst, opts)
if err != nil {
return errors.Wrap(err)
}
if err := bp.Export(idx, encoder); err != nil {
return errors.Wrap(err)
}
}
return nil
}
}

View File

@@ -1,34 +0,0 @@
package holos
import "encoding/yaml"
import v1 "github.com/holos-run/holos/api/v1alpha1"
// #Helm represents a holos build plan composed of one or more helm charts.
#Helm: {
Name: string
Version: string
Namespace: string
Repo: {
name: string | *""
url: string | *""
}
Values: {...}
Chart: v1.#HelmChart & {
metadata: name: string | *Name
namespace: string | *Namespace
chart: name: string | *Name
chart: version: string | *Version
chart: repository: Repo
// Render the values to yaml for holos to provide to helm.
valuesContent: yaml.Marshal(Values)
}
// output represents the build plan provided to the holos cli.
Output: v1.#BuildPlan & {
spec: components: helmChartList: [Chart]
}
}

View File

@@ -1,4 +0,0 @@
package holos
// #ClusterName is the --cluster-name flag value provided by the holos cli.
#ClusterName: string @tag(cluster, type=string)

View File

@@ -1,30 +0,0 @@
package holos
import "encoding/yaml"
import v1 "github.com/holos-run/holos/api/v1alpha1"
// Provide a BuildPlan to the holos cli to render k8s api objects.
v1.#BuildPlan & {
spec: components: resources: platformConfigmap: {
metadata: name: "platform-configmap"
apiObjectMap: OBJECTS.apiObjectMap
}
}
// OBJECTS represents the kubernetes api objects to manage.
let OBJECTS = v1.#APIObjects & {
apiObjects: ConfigMap: platform: {
metadata: {
name: "platform"
namespace: "default"
}
// Output the platform model which is derived from the web app form the
// platform engineer provides and the form values the end user provides.
data: platform: yaml.Marshal(PLATFORM)
}
}
let PLATFORM = {
spec: model: _Platform.spec.model
}

View File

@@ -1,314 +0,0 @@
package forms
import v1 "github.com/holos-run/holos/api/v1alpha1"
// Provides a concrete v1.#Form
FormBuilder.Output
let FormBuilder = v1.#FormBuilder & {
Sections: org: {
displayName: "Organization"
description: "Organization config values are used to derive more specific configuration values throughout the platform."
fieldConfigs: {
// platform.spec.config.user.sections.org.fields.name
name: {
type: "input"
props: {
label: "Name"
// placeholder: "example" placeholder cannot be used with validation?
description: "DNS label, e.g. 'example'"
pattern: "^[a-z]([0-9a-z]|-){1,28}[0-9a-z]$"
minLength: 3
maxLength: 30
required: true
}
validation: messages: {
pattern: "It must be \(props.minLength) to \(props.maxLength) lowercase letters, digits, or hyphens. It must start with a letter. Trailing hyphens are prohibited."
minLength: "Must be at least \(props.minLength) characters"
maxLength: "Must be at most \(props.maxLength) characters"
}
}
// platform.spec.config.user.sections.org.fields.displayName
displayName: {
type: "input"
props: {
label: "Display Name"
placeholder: "Example Organization"
description: "Display name, e.g. 'Example Organization'"
maxLength: 100
required: true
}
}
}
}
Sections: cloud: {
displayName: "Cloud Providers"
description: "Select the services that provide resources for the platform."
fieldConfigs: {
providers: {
// https://formly.dev/docs/api/ui/material/select/
type: "select"
props: {
label: "Select Providers"
description: "Select the cloud providers the platform builds upon."
multiple: true
selectAllOption: "Select All"
options: [
{value: "aws", label: "Amazon Web Services"},
{value: "gcp", label: "Google Cloud Platform"},
{value: "azure", label: "Microsoft Azure"},
{value: "cloudflare", label: "Cloudflare"},
{value: "github", label: "GitHub"},
{value: "ois", label: "Open Infrastructure Services"},
{value: "onprem", label: "On Premises", disabled: true},
]
}
}
}
}
Sections: aws: {
displayName: "Amazon Web Services"
description: "Provide the information necessary for Holos to manage AWS resources to provide the platform."
expressions: hide: "!\(AWSSelected)"
fieldConfigs: {
primaryRoleARN: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Holos Admin Role ARN"
description: "Enter the AWS Role ARN Holos will use to bootstrap resources. For example, arn:aws:iam::123456789012:role/HolosAdminAccess"
pattern: "^arn:.*"
minLength: 4
required: true
}
validation: messages: {
pattern: "Must be a valid ARN. Refer to https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html"
}
}
regions: {
// https://formly.dev/docs/api/ui/material/select/
type: "select"
props: {
label: "Select Regions"
description: "Select the AWS regions this platform operates in."
multiple: true
required: true
selectAllOption: "Select All"
options: AWSRegions
}
}
}
}
Sections: gcp: {
displayName: "Google Cloud Platform"
description: "Use this form to configure platform level GCP settings."
expressions: hide: "!\(GCPSelected)"
fieldConfigs: {
regions: {
// https://formly.dev/docs/api/ui/material/select/
type: "select"
props: {
label: "Select Regions"
description: "Select the GCP regions this platform operates in."
multiple: true
selectAllOption: "Select All"
// gcloud compute regions list --format=json | jq '.[] | {value: .name, label: .description}' regions.json | jq -s | cue export --out cue
options: GCPRegions
}
}
gcpProjectID: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Project ID"
description: "Enter the project id where the provisioner cluster resides."
pattern: "^[a-z]([0-9a-z]|-){1,28}[0-9a-z]$"
minLength: 6
maxLength: 30
required: true
}
validation: messages: {
pattern: "It must be \(props.minLength) to \(props.maxLength) lowercase letters, digits, or hyphens. It must start with a letter. Trailing hyphens are prohibited."
minLength: "Must be at least \(props.minLength) characters."
maxLength: "Must be at most \(props.maxLength) characters."
}
}
gcpProjectNumber: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Project Number"
// note type number here
type: "number"
description: "Enter the project number where the provisioner cluster resides."
pattern: "^[0-9]+$"
required: true
}
validation: messages: {
pattern: "Must be a valid project number."
}
}
provisionerCABundle: {
type: "input"
props: {
label: "Provisioner CA Bundle"
description: "Enter the provisioner cluster ca bundle. kubectl config view --minify --flatten -ojsonpath='{.clusters[0].cluster.certificate-authority-data}'"
pattern: "^[0-9a-zA-Z]+=*$"
required: true
}
validation: messages: {
pattern: "Must be a base64 encoded pem encoded certificate bundle."
}
}
provisionerURL: {
type: "input"
props: {
label: "Provisioner URL"
description: "Enter the URL of the provisioner cluster API endpoint. kubectl config view --minify --flatten -ojsonpath='{.clusters[0].cluster.server}'"
pattern: "^https://.*$"
required: true
}
validation: messages: {
pattern: "Must be a https:// URL."
}
}
}
}
Sections: cloudflare: {
displayName: "Cloudflare"
description: "Cloudflare is primarily used for DNS automation."
expressions: hide: "!" + CloudflareSelected
fieldConfigs: {
email: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Account Email"
description: "Enter the Cloudflare email address to manage DNS"
minLength: 3
required: true
}
}
}
}
Sections: github: {
displayName: "GitHub"
description: "GitHub is primarily used to host Git repositories and execute Actions workflows."
expressions: hide: "!\(GitHubSelected)"
fieldConfigs: {
primaryOrg: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Organization"
description: "Enter the primary GitHub organization associed with the platform."
pattern: "^(?!-)(?!.*--)([a-zA-Z0-9]|-){1,39}$"
minLength: 1
maxLength: 39
required: true
}
validation: messages: {
pattern: "All characters must be either a hyphen or alphanumeric. Cannot start with a hyphen. Cannot include consecutive hyphens."
}
}
}
}
}
let GCPRegions = [
{value: "africa-south1", label: "africa-south1"},
{value: "asia-east1", label: "asia-east1"},
{value: "asia-east2", label: "asia-east2"},
{value: "asia-northeast1", label: "asia-northeast1"},
{value: "asia-northeast2", label: "asia-northeast2"},
{value: "asia-northeast3", label: "asia-northeast3"},
{value: "asia-south1", label: "asia-south1"},
{value: "asia-south2", label: "asia-south2"},
{value: "asia-southeast1", label: "asia-southeast1"},
{value: "asia-southeast2", label: "asia-southeast2"},
{value: "australia-southeast1", label: "australia-southeast1"},
{value: "australia-southeast2", label: "australia-southeast2"},
{value: "europe-central2", label: "europe-central2"},
{value: "europe-north1", label: "europe-north1"},
{value: "europe-southwest1", label: "europe-southwest1"},
{value: "europe-west1", label: "europe-west1"},
{value: "europe-west10", label: "europe-west10"},
{value: "europe-west12", label: "europe-west12"},
{value: "europe-west2", label: "europe-west2"},
{value: "europe-west3", label: "europe-west3"},
{value: "europe-west4", label: "europe-west4"},
{value: "europe-west6", label: "europe-west6"},
{value: "europe-west8", label: "europe-west8"},
{value: "europe-west9", label: "europe-west9"},
{value: "me-central1", label: "me-central1"},
{value: "me-central2", label: "me-central2"},
{value: "me-west1", label: "me-west1"},
{value: "northamerica-northeast1", label: "northamerica-northeast1"},
{value: "northamerica-northeast2", label: "northamerica-northeast2"},
{value: "southamerica-east1", label: "southamerica-east1"},
{value: "southamerica-west1", label: "southamerica-west1"},
{value: "us-central1", label: "us-central1"},
{value: "us-east1", label: "us-east1"},
{value: "us-east4", label: "us-east4"},
{value: "us-east5", label: "us-east5"},
{value: "us-south1", label: "us-south1"},
{value: "us-west1", label: "us-west1"},
{value: "us-west2", label: "us-west2"},
{value: "us-west3", label: "us-west3"},
{value: "us-west4", label: "us-west4"},
]
let AWSRegions = [
{value: "us-east-1", label: "N. Virginia (us-east-1)"},
{value: "us-east-2", label: "Ohio (us-east-2)"},
{value: "us-west-1", label: "N. California (us-west-1)"},
{value: "us-west-2", label: "Oregon (us-west-2)"},
{value: "us-gov-west1", label: "US GovCloud West (us-gov-west1)"},
{value: "us-gov-east1", label: "US GovCloud East (us-gov-east1)"},
{value: "ca-central-1", label: "Canada (ca-central-1)"},
{value: "eu-north-1", label: "Stockholm (eu-north-1)"},
{value: "eu-west-1", label: "Ireland (eu-west-1)"},
{value: "eu-west-2", label: "London (eu-west-2)"},
{value: "eu-west-3", label: "Paris (eu-west-3)"},
{value: "eu-central-1", label: "Frankfurt (eu-central-1)"},
{value: "eu-south-1", label: "Milan (eu-south-1)"},
{value: "af-south-1", label: "Cape Town (af-south-1)"},
{value: "ap-northeast-1", label: "Tokyo (ap-northeast-1)"},
{value: "ap-northeast-2", label: "Seoul (ap-northeast-2)"},
{value: "ap-northeast-3", label: "Osaka (ap-northeast-3)"},
{value: "ap-southeast-1", label: "Singapore (ap-southeast-1)"},
{value: "ap-southeast-2", label: "Sydney (ap-southeast-2)"},
{value: "ap-east-1", label: "Hong Kong (ap-east-1)"},
{value: "ap-south-1", label: "Mumbai (ap-south-1)"},
{value: "me-south-1", label: "Bahrain (me-south-1)"},
{value: "sa-east-1", label: "São Paulo (sa-east-1)"},
{value: "cn-north-1", label: "Bejing (cn-north-1)"},
{value: "cn-northwest-1", label: "Ningxia (cn-northwest-1)"},
{value: "ap-southeast-3", label: "Jakarta (ap-southeast-3)"},
]
let AWSSelected = "formState.model.cloud?.providers?.includes(\"aws\")"
let GCPSelected = "formState.model.cloud?.providers?.includes(\"gcp\")"
let GitHubSelected = "formState.model.cloud?.providers?.includes(\"github\")"
let CloudflareSelected = "formState.model.cloud?.providers?.includes(\"cloudflare\")"

View File

@@ -1,47 +0,0 @@
package holos
import "encoding/json"
import v1 "github.com/holos-run/holos/api/v1alpha1"
import dto "github.com/holos-run/holos/service/gen/holos/object/v1alpha1:object"
// _PlatformConfig represents all of the data passed from holos to cue.
// Intended to carry the platform model and project models.
_PlatformConfig: dto.#PlatformConfig & json.Unmarshal(_PlatformConfigJSON)
_PlatformConfigJSON: string | *"{}" @tag(platform_config, type=string)
// _Platform provides a platform resource to the holos cli for rendering. The
// field is hidden because most components need to refer to platform data,
// specifically the platform model and the project models. The platform
// resource itself is output once when rendering the entire platform, see the
// platform/ subdirectory.
_Platform: v1.#Platform & {
metadata: {
name: string | *"bare" @tag(platform_name, type=string)
}
// spec is the platform specification
spec: {
// model represents the web form values provided by the user.
model: _PlatformConfig.platform_model
components: [for c in _components {c}]
_components: [string]: v1.#PlatformSpecComponent
_components: {
for WorkloadCluster in _Clusters.Workload {
"\(WorkloadCluster)-configmap": {
path: "components/configmap"
cluster: WorkloadCluster
}
}
}
}
}
// _Clusters represents the clusters in the platform. The default values are
// intended to be provided by the user in a file which is not written over by
// `holos generate`.
_Clusters: {
Workload: [...string] | *["mycluster"]
}

View File

@@ -1,4 +0,0 @@
package holos
// Output the Platform resource for holos to render the entire platform.
{} & _Platform

View File

@@ -1,20 +0,0 @@
Bare Platform
| Folder | Description |
| - | - |
| forms | Contains Platform and Project form and model definitions |
| platform | Contains the Platform resource that defines how to render the configuration for all Platform Components |
| components | Contains BuildPlan resources which define how to render individual Platform Components |
## Forms
To populate the form, the platform must already be created in the Web UI:
```bash
platformId="018f36fb-e3ff-7f7f-a5d1-7ca2bf499e94"
cue export ./forms/platform/ --out json \
| jq '{platform_id: "'$platformId'", fields: .spec.fields}' \
| grpcurl -H "x-oidc-id-token: $(holos token)" -d @ \
app.dev.k2.holos.run:443 \
holos.v1alpha1.PlatformService.PutForm
```

View File

@@ -49,6 +49,12 @@ import "github.com/holos-run/holos/api/core/v1alpha5:core"
// fully rendered manifest file path.
Name: string
// Labels represent the BuildPlan metadata.labels field.
Labels: {[string]: string} @go(,map[string]string)
// Annotations represent the BuildPlan metadata.annotations field.
Annotations: {[string]: string} @go(,map[string]string)
// Path represents the path to the component producing the BuildPlan.
Path: string

View File

@@ -35,9 +35,6 @@ package core
// Spec specifies the desired state of the resource.
spec: #BuildPlanSpec @go(Spec)
// Source reflects the origin of the BuildPlan.
source?: #BuildPlanSource @go(Source)
}
// BuildPlanSpec represents the specification of the [BuildPlan].
@@ -214,7 +211,7 @@ package core
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
#Join: {
separator: string & (string | *"---\n") @go(Separator)
separator?: string @go(Separator)
}
// Kustomize represents a kustomization [Transformer].
@@ -254,6 +251,14 @@ package core
#Metadata: {
// Name represents the resource name.
name: string @go(Name)
// Labels represents a resource selector.
labels?: {[string]: string} @go(Labels,map[string]string)
// Annotations represents arbitrary non-identifying metadata. For example
// holos uses the `cli.holos.run/description` annotation to log resources in a
// user customized way.
annotations?: {[string]: string} @go(Annotations,map[string]string)
}
// Platform represents a platform to manage. A Platform specifies a [Component]
@@ -305,4 +310,12 @@ package core
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
parameters?: {[string]: string} @go(Parameters,map[string]string)
// Labels represent selector labels for the component. Copied to the
// resulting BuildPlan.
labels?: {[string]: string} @go(Labels,map[string]string)
// Annotations represents arbitrary non-identifying metadata. Use the
// `cli.holos.run/description` to customize the log message of each BuildPlan.
annotations?: {[string]: string} @go(Annotations,map[string]string)
}

View File

@@ -1,31 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// BuildPlan is the primary interface between CUE and the Holos cli.
#BuildPlan: {
#TypeMeta
// Metadata represents the holos component name
metadata?: #ObjectMeta @go(Metadata)
spec?: #BuildPlanSpec @go(Spec)
}
#BuildPlanSpec: {
disabled?: bool @go(Disabled)
components?: #BuildPlanComponents @go(Components)
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
deployFiles?: #FileContentMap @go(DeployFiles)
}
#BuildPlanComponents: {
helmChartList?: [...#HelmChart] @go(HelmChartList,[]HelmChart)
kubernetesObjectsList?: [...#KubernetesObjects] @go(KubernetesObjectsList,[]KubernetesObjects)
kustomizeBuildList?: [...#KustomizeBuild] @go(KustomizeBuildList,[]KustomizeBuild)
resources?: {[string]: #KubernetesObjects} @go(Resources,map[string]KubernetesObjects)
}

View File

@@ -1,24 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// HolosComponent defines the fields common to all holos component kinds including the Render Result.
#HolosComponent: {
#TypeMeta
// Metadata represents the holos component name
metadata?: #ObjectMeta @go(Metadata)
// APIObjectMap holds the marshalled representation of api objects. Think of
// these as resources overlaid at the back of the render pipeline.
apiObjectMap?: #APIObjectMap @go(APIObjectMap)
#Kustomization
#Kustomize
// Skip causes holos to take no action regarding the component.
Skip: bool
}

View File

@@ -1,15 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
#APIVersion: "holos.run/v1alpha1"
#BuildPlanKind: "BuildPlan"
#HelmChartKind: "HelmChart"
// ChartDir is the directory name created in the holos component directory to cache a chart.
#ChartDir: "vendor"
// ResourcesFile is the file name used to store component output when post-processing with kustomize.
#ResourcesFile: "resources.yaml"

View File

@@ -1,6 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
// Package v1alpha1 defines the api boundary between CUE and Holos.
package v1alpha1

View File

@@ -1,17 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
import "github.com/holos-run/holos/service/gen/holos/object/v1alpha1:object"
// Form represents a collection of Formly json powered form.
#Form: {
#TypeMeta
spec: #FormSpec @go(Spec)
}
#FormSpec: {
form: object.#Form @go(Form)
}

View File

@@ -1,28 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// A HelmChart represents a helm command to provide chart values in order to render kubernetes api objects.
#HelmChart: {
#HolosComponent
// Namespace is the namespace to install into. TODO: Use metadata.namespace instead.
namespace: string @go(Namespace)
chart: #Chart @go(Chart)
valuesContent: string @go(ValuesContent)
enableHooks: bool @go(EnableHooks)
}
#Chart: {
name: string @go(Name)
version: string @go(Version)
release: string @go(Release)
repository?: #Repository @go(Repository)
}
#Repository: {
name: string @go(Name)
url: string @go(URL)
}

View File

@@ -1,12 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
#KubernetesObjectsKind: "KubernetesObjects"
// KubernetesObjects represents CUE output which directly provides Kubernetes api objects to holos.
#KubernetesObjects: {
#HolosComponent
}

View File

@@ -1,11 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// Kustomization holds the rendered flux kustomization api object content for git ops.
#Kustomization: {
// KsContent is the yaml representation of the flux kustomization for gitops.
ksContent?: string @go(KsContent)
}

View File

@@ -1,25 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
#KustomizeBuildKind: "KustomizeBuild"
// Kustomize represents resources necessary to execute a kustomize build.
// Intended for at least two use cases:
//
// 1. Process raw yaml file resources in a holos component directory.
// 2. Post process a HelmChart to inject istio, add custom labels, etc...
#Kustomize: {
// KustomizeFiles holds file contents for kustomize, e.g. patch files.
kustomizeFiles?: #FileContentMap @go(KustomizeFiles)
// ResourcesFile is the file name used for api objects in kustomization.yaml
resourcesFile?: string @go(ResourcesFile)
}
// KustomizeBuild renders plain yaml files in the holos component directory using kubectl kustomize build.
#KustomizeBuild: {
#HolosComponent
}

View File

@@ -1,18 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// Label is an arbitrary unique identifier. Defined as a type for clarity and type checking.
#Label: string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type checking.
#Kind: string
// APIObjectMap is the shape of marshalled api objects returned from cue to the
// holos cli. A map is used to improve the clarity of error messages from cue.
#APIObjectMap: {[string]: [string]: string}
// FileContentMap is a map of file names to file contents.
#FileContentMap: {[string]: string}

View File

@@ -1,22 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// ObjectMeta represents metadata of a holos component object. The fields are a
// copy of upstream kubernetes api machinery but are by holos objects distinct
// from kubernetes api objects.
#ObjectMeta: {
// Name uniquely identifies the holos component instance and must be suitable as a file name.
name?: string @go(Name)
// Namespace confines a holos component to a single namespace via kustomize if set.
namespace?: string @go(Namespace)
// Labels are not used but are copied from api machinery ObjectMeta for completeness.
labels?: {[string]: string} @go(Labels,map[string]string)
// Annotations are not used but are copied from api machinery ObjectMeta for completeness.
annotations?: {[string]: string} @go(Annotations,map[string]string)
}

View File

@@ -1,37 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
import "google.golang.org/protobuf/types/known/structpb"
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
#Platform: {
#TypeMeta
metadata: #ObjectMeta @go(Metadata)
spec: #PlatformSpec @go(Spec)
}
// PlatformSpec represents the platform build plan specification.
#PlatformSpec: {
// Model represents the platform model holos gets from from the
// holos.platform.v1alpha1.PlatformService.GetPlatform method and provides to
// CUE using a tag.
model: structpb.#Struct @go(Model)
components: [...#PlatformSpecComponent] @go(Components,[]PlatformSpecComponent)
}
// PlatformSpecComponent represents a component to build or render with flags to
// pass, for example the cluster name.
#PlatformSpecComponent: {
// Path is the path of the component relative to the platform root.
path: string @go(Path)
// Cluster is the cluster name to use when building the component.
cluster: string @go(Cluster)
}

View File

@@ -1,7 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
#Renderer: _

View File

@@ -1,15 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
// Result is the build result for display or writing. Holos components Render the Result as a data pipeline.
#Result: {
HolosComponent: #HolosComponent
// DeployFiles keys represent file paths relative to the cluster deploy
// directory. Map values represent the string encoded file contents. Used to
// write the argocd Application, but may be used to render any file from CUE.
deployFiles?: #FileContentMap @go(DeployFiles)
}

View File

@@ -1,13 +0,0 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/v1alpha1
package v1alpha1
#TypeMeta: {
kind?: string @go(Kind)
apiVersion?: string @go(APIVersion)
}
// Discriminator is an interface to discriminate the kind api object.
#Discriminator: _

View File

@@ -35,6 +35,8 @@ import (
// https://holos.run/docs/next/api/author/#Kubernetes
#Kubernetes: {
Name: _
Labels: _
Annotations: _
Path: _
Parameters: _
Resources: _
@@ -84,18 +86,21 @@ import (
BuildPlan: {
metadata: name: Name
spec: artifacts: [for x in Artifacts {x}]
source: component: {
name: Name
path: Path
parameters: Parameters
if len(Labels) != 0 {
metadata: labels: Labels
}
if len(Annotations) != 0 {
metadata: annotations: Annotations
}
spec: artifacts: [for x in Artifacts {x}]
}
}
// https://holos.run/docs/next/api/author/#Helm
#Helm: {
Name: _
Labels: _
Annotations: _
Path: _
Parameters: _
Resources: _
@@ -175,11 +180,12 @@ import (
BuildPlan: {
metadata: name: Name
spec: artifacts: [for x in Artifacts {x}]
source: component: {
name: Name
path: Path
parameters: Parameters
if len(Labels) != 0 {
metadata: labels: Labels
}
if len(Annotations) != 0 {
metadata: annotations: Annotations
}
spec: artifacts: [for x in Artifacts {x}]
}
}

View File

@@ -0,0 +1,7 @@
package types
#Target: {
// Ensure name has a concrete value so json.Marshal works. See
// https://github.com/holos-run/holos/issues/348
name: string | *""
}

View File

@@ -1,4 +0,0 @@
package holos
// #ClusterName is the --cluster-name flag value provided by the holos cli.
#ClusterName: string @tag(cluster, type=string)

View File

@@ -1,26 +0,0 @@
package holos
import "encoding/yaml"
import v1 "github.com/holos-run/holos/api/v1alpha1"
let PLATFORM = {message: "TODO: Load the platform from the API."}
// Provide a BuildPlan to the holos cli to render k8s api objects.
v1.#BuildPlan & {
spec: components: resources: platformConfigmap: {
metadata: name: "platform-configmap"
apiObjectMap: OBJECTS.apiObjectMap
}
}
// OBJECTS represents the kubernetes api objects to manage.
let OBJECTS = v1.#APIObjects & {
apiObjects: ConfigMap: platform: {
metadata: {
name: "platform"
namespace: "default"
}
data: platform: yaml.Marshal(PLATFORM)
}
}

View File

@@ -1,358 +0,0 @@
package forms
import v1 "github.com/holos-run/holos/v1alpha1"
// Provides a concrete v1.#Form
FormBuilder.Output
let FormBuilder = v1.#FormBuilder & {
Sections: org: {
displayName: "Organization"
description: "Organization config values are used to derive more specific configuration values throughout the platform."
fieldConfigs: {
// platform.spec.config.user.sections.org.fields.name
name: {
type: "input"
props: {
label: "Name"
// placeholder: "example" placeholder cannot be used with validation?
description: "DNS label, e.g. 'example'"
pattern: "^[a-z]([0-9a-z]|-){1,28}[0-9a-z]$"
minLength: 3
maxLength: 30
required: true
}
validation: messages: {
pattern: "It must be 3 to 30 lowercase letters, digits, or hyphens. It must start with a letter. Trailing hyphens are prohibited."
}
}
// platform.spec.config.user.sections.org.fields.domain
domain: {
type: "input"
props: {
label: "Domain"
placeholder: "example.com"
minLength: 3
maxLength: 100
description: "DNS domain, e.g. 'example.com'"
required: true
}
}
// platform.spec.config.user.sections.org.fields.displayName
displayName: {
type: "input"
props: {
label: "Display Name"
placeholder: "Example Organization"
description: "Display name, e.g. 'Example Organization'"
maxLength: 100
required: true
}
}
// platform.spec.config.user.sections.org.fields.contactEmail
contactEmail: {
type: "input"
props: {
label: "Contact Email"
placeholder: "platform-team@example.com"
description: "Technical contact email address"
required: true
}
}
}
}
Sections: cloud: {
displayName: "Cloud Providers"
description: "Select the services that provide resources for the platform."
fieldConfigs: {
providers: {
// https://formly.dev/docs/api/ui/material/select/
type: "select"
props: {
label: "Select Providers"
description: "Select the cloud providers the platform builds upon."
multiple: true
selectAllOption: "Select All"
options: [
{value: "aws", label: "Amazon Web Services"},
{value: "gcp", label: "Google Cloud Platform"},
{value: "azure", label: "Microsoft Azure"},
{value: "cloudflare", label: "Cloudflare"},
{value: "github", label: "GitHub"},
{value: "ois", label: "Open Infrastructure Services"},
{value: "onprem", label: "On Premises", disabled: true},
]
}
}
}
}
Sections: aws: {
displayName: "Amazon Web Services"
description: "Provide the information necessary for Holos to manage AWS resources to provide the platform."
expressions: hide: "!\(AWSSelected)"
fieldConfigs: {
primaryRoleARN: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Holos Admin Role ARN"
description: "Enter the AWS Role ARN Holos will use to bootstrap resources. For example, arn:aws:iam::123456789012:role/HolosAdminAccess"
pattern: "^arn:.*"
minLength: 4
required: true
}
validation: messages: {
pattern: "Must be a valid ARN. Refer to https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html"
}
}
regions: {
// https://formly.dev/docs/api/ui/material/select/
type: "select"
props: {
label: "Select Regions"
description: "Select the AWS regions this platform operates in."
multiple: true
required: true
selectAllOption: "Select All"
options: AWSRegions
}
}
}
}
Sections: gcp: {
displayName: "Google Cloud Platform"
description: "Use this form to configure platform level GCP settings."
expressions: hide: "!\(GCPSelected)"
fieldConfigs: {
regions: {
// https://formly.dev/docs/api/ui/material/select/
type: "select"
props: {
label: "Select Regions"
description: "Select the GCP regions this platform operates in."
multiple: true
selectAllOption: "Select All"
// gcloud compute regions list --format=json | jq '.[] | {value: .name, label: .description}' regions.json | jq -s | cue export --out cue
options: GCPRegions
}
}
gcpProjectID: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Project ID"
description: "Enter the project id where the provisioner cluster resides."
pattern: "^[a-z]([0-9a-z]|-){1,28}[0-9a-z]$"
minLength: 6
maxLength: 30
required: true
}
validation: messages: {
pattern: "It must be 3 to 30 lowercase letters, digits, or hyphens. It must start with a letter. Trailing hyphens are prohibited."
}
}
gcpProjectNumber: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Project Number"
// note type number here
type: "number"
description: "Enter the project number where the provisioner cluster resides."
pattern: "^[0-9]+$"
required: true
}
validation: messages: {
pattern: "Must be a valid project number."
}
}
provisionerCABundle: {
type: "input"
props: {
label: "Provisioner CA Bundle"
description: "Enter the provisioner cluster ca bundle. kubectl config view --minify --flatten -ojsonpath='{.clusters[0].cluster.certificate-authority-data}'"
pattern: "^[0-9a-zA-Z]+=*$"
required: true
}
validation: messages: {
pattern: "Must be a base64 encoded pem encoded certificate bundle."
}
}
provisionerURL: {
type: "input"
props: {
label: "Provisioner URL"
description: "Enter the URL of the provisioner cluster API endpoint. kubectl config view --minify --flatten -ojsonpath='{.clusters[0].cluster.server}'"
pattern: "^https://.*$"
required: true
}
validation: messages: {
pattern: "Must be a https:// URL."
}
}
}
}
Sections: cloudflare: {
displayName: "Cloudflare"
description: "Cloudflare is primarily used for DNS automation."
expressions: hide: "!" + CloudflareSelected
fieldConfigs: {
email: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Account Email"
description: "Enter the Cloudflare email address to manage DNS"
minLength: 3
required: true
}
}
}
}
Sections: github: {
displayName: "GitHub"
description: "GitHub is primarily used to host Git repositories and execute Actions workflows."
expressions: hide: "!\(GitHubSelected)"
fieldConfigs: {
primaryOrg: {
// https://formly.dev/docs/api/ui/material/input
type: "input"
props: {
label: "Organization"
description: "Enter the primary GitHub organization associed with the platform."
pattern: "^(?!-)(?!.*--)([a-zA-Z0-9]|-){1,39}$"
minLength: 1
maxLength: 39
required: true
}
validation: messages: {
pattern: "All characters must be either a hyphen or alphanumeric. Cannot start with a hyphen. Cannot include consecutive hyphens."
}
}
}
}
Sections: backups: {
displayName: "Backups"
description: "Configure platform level data backup settings. Requires AWS."
fieldConfigs: {
s3bucket: {
// https://formly.dev/docs/api/ui/material/input
type: "select"
props: {
label: "S3 Bucket Region"
description: "Select the S3 Bucket Region."
multiple: true
options: AWSRegions
}
expressions: {
// Disable the control if AWS is not selected.
"props.disabled": "!" + AWSSelected
// Required if AWS is selected.
"props.required": AWSSelected
// Change the label depending on AWS
"props.description": AWSSelected + " ? '\(props.description)' : 'Enable AWS in the Cloud Provider section to configure backups.'"
}
}
}
}
}
let GCPRegions = [
{value: "africa-south1", label: "africa-south1"},
{value: "asia-east1", label: "asia-east1"},
{value: "asia-east2", label: "asia-east2"},
{value: "asia-northeast1", label: "asia-northeast1"},
{value: "asia-northeast2", label: "asia-northeast2"},
{value: "asia-northeast3", label: "asia-northeast3"},
{value: "asia-south1", label: "asia-south1"},
{value: "asia-south2", label: "asia-south2"},
{value: "asia-southeast1", label: "asia-southeast1"},
{value: "asia-southeast2", label: "asia-southeast2"},
{value: "australia-southeast1", label: "australia-southeast1"},
{value: "australia-southeast2", label: "australia-southeast2"},
{value: "europe-central2", label: "europe-central2"},
{value: "europe-north1", label: "europe-north1"},
{value: "europe-southwest1", label: "europe-southwest1"},
{value: "europe-west1", label: "europe-west1"},
{value: "europe-west10", label: "europe-west10"},
{value: "europe-west12", label: "europe-west12"},
{value: "europe-west2", label: "europe-west2"},
{value: "europe-west3", label: "europe-west3"},
{value: "europe-west4", label: "europe-west4"},
{value: "europe-west6", label: "europe-west6"},
{value: "europe-west8", label: "europe-west8"},
{value: "europe-west9", label: "europe-west9"},
{value: "me-central1", label: "me-central1"},
{value: "me-central2", label: "me-central2"},
{value: "me-west1", label: "me-west1"},
{value: "northamerica-northeast1", label: "northamerica-northeast1"},
{value: "northamerica-northeast2", label: "northamerica-northeast2"},
{value: "southamerica-east1", label: "southamerica-east1"},
{value: "southamerica-west1", label: "southamerica-west1"},
{value: "us-central1", label: "us-central1"},
{value: "us-east1", label: "us-east1"},
{value: "us-east4", label: "us-east4"},
{value: "us-east5", label: "us-east5"},
{value: "us-south1", label: "us-south1"},
{value: "us-west1", label: "us-west1"},
{value: "us-west2", label: "us-west2"},
{value: "us-west3", label: "us-west3"},
{value: "us-west4", label: "us-west4"},
]
let AWSRegions = [
{value: "us-east-1", label: "N. Virginia (us-east-1)"},
{value: "us-east-2", label: "Ohio (us-east-2)"},
{value: "us-west-1", label: "N. California (us-west-1)"},
{value: "us-west-2", label: "Oregon (us-west-2)"},
{value: "us-gov-west1", label: "US GovCloud West (us-gov-west1)"},
{value: "us-gov-east1", label: "US GovCloud East (us-gov-east1)"},
{value: "ca-central-1", label: "Canada (ca-central-1)"},
{value: "eu-north-1", label: "Stockholm (eu-north-1)"},
{value: "eu-west-1", label: "Ireland (eu-west-1)"},
{value: "eu-west-2", label: "London (eu-west-2)"},
{value: "eu-west-3", label: "Paris (eu-west-3)"},
{value: "eu-central-1", label: "Frankfurt (eu-central-1)"},
{value: "eu-south-1", label: "Milan (eu-south-1)"},
{value: "af-south-1", label: "Cape Town (af-south-1)"},
{value: "ap-northeast-1", label: "Tokyo (ap-northeast-1)"},
{value: "ap-northeast-2", label: "Seoul (ap-northeast-2)"},
{value: "ap-northeast-3", label: "Osaka (ap-northeast-3)"},
{value: "ap-southeast-1", label: "Singapore (ap-southeast-1)"},
{value: "ap-southeast-2", label: "Sydney (ap-southeast-2)"},
{value: "ap-east-1", label: "Hong Kong (ap-east-1)"},
{value: "ap-south-1", label: "Mumbai (ap-south-1)"},
{value: "me-south-1", label: "Bahrain (me-south-1)"},
{value: "sa-east-1", label: "São Paulo (sa-east-1)"},
{value: "cn-north-1", label: "Bejing (cn-north-1)"},
{value: "cn-northwest-1", label: "Ningxia (cn-northwest-1)"},
{value: "ap-southeast-3", label: "Jakarta (ap-southeast-3)"},
]
let AWSSelected = "formState.model.cloud?.providers?.includes(\"aws\")"
let GCPSelected = "formState.model.cloud?.providers?.includes(\"gcp\")"
let GitHubSelected = "formState.model.cloud?.providers?.includes(\"github\")"
let CloudflareSelected = "formState.model.cloud?.providers?.includes(\"cloudflare\")"

View File

@@ -1,7 +0,0 @@
Example Platform
| Folder | Description |
| - | - |
| forms | Contains Platform and Project form and model definitions |
| platform | Contains the Platform resource that defines how to render the configuration for all Platform Components |
| components | Contains BuildPlan resources which define how to render individual Platform Components |

View File

@@ -3,9 +3,6 @@ package platforms
// TODO: Remove env GODEBUG=gotypesalias=0 when cue 0.11 is released and used.
// See: https://github.com/cue-lang/cue/issues/3539
//go:generate rm -rf cue.mod/gen/github.com/holos-run/holos/api/v1alpha1
//go:generate env GODEBUG=gotypesalias=0 cue get go github.com/holos-run/holos/api/v1alpha1/...
//go generate rm -rf cue.mod/gen/github.com/holos-run/holos/api/core
//go:generate env GODEBUG=gotypesalias=0 cue get go github.com/holos-run/holos/api/core/...

View File

@@ -1,9 +0,0 @@
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
vendor/
node_modules/

View File

@@ -1,5 +0,0 @@
# Holos
Generated for use with [Holos Guides][guides].
[guides]: https://holos.run/docs/guides/

View File

@@ -1,3 +0,0 @@
package holos
#Platform: Name: "guide"

View File

@@ -1,3 +0,0 @@
package holos
#Platform.Output

Some files were not shown because too many files have changed in this diff Show More