Compare commits

..

96 Commits

Author SHA1 Message Date
Jeff McCune
6bf0cb8d8e v1alpha4: v0.97.0 2024-10-17 07:50:49 -07:00
Jeff McCune
766c8912b7 Merge pull request #281 from holos-run/jeff/280-authorapi-v1alpha4
v1alpha4 Author API
2024-10-17 07:17:14 -07:00
Jeff McCune
be1dee5f1c v1alpha4: update technical overview guide (#280)
Update the guide and the bank of holos repository to use v1alpha4
instead of v1alpha3.
2024-10-17 07:13:51 -07:00
Jeff McCune
6ad56525ac v1alpha4: refactor --tag to --inject and remove environment (#276)
Cue uses --inject, -t as the flags to set variables for fields tagged
using @tag(var,type=string).

We used --tag, which is different and requires a mental mapping.  Let's
use the same flag and also pass it multiple times like they require so
we can copy and paste the command line output from the debug logs into a
cue export command to see what's going on.

This patch deprecates the --cluster-name flag, use --inject
holos_cluster=mycluster instead.

This patch also removes the environment field from the Component core
API, leaving this to the user namespace to define via tags.  We don't
want to be too opinionated on how users manage their platform, baking
environment into the schema is a slippery slope toward those kinds of
opinions.

Closes: #276
2024-10-16 22:07:47 -07:00
Jeff McCune
791ec5ee71 v1alpha4: refactor core.Component Tags to map[string]string (#280)
Previously it was a []string slice that must be formatted as key=value.
This is more difficult to work with than a map[string]string.
2024-10-16 20:10:14 -07:00
Jeff McCune
638ac7473c fixup 2024-10-16 20:05:05 -07:00
Jeff McCune
ee24b5ce13 fixup 2024-10-16 20:04:18 -07:00
Jeff McCune
fa2fdbe4e8 fixup 2024-10-16 20:00:53 -07:00
Jeff McCune
63e1df1d4c v1alpha4: add common labels to projects schema (#280)
Now that we have CommonLabels as part of the ComponentConfig for all
components, it makes sense to also mix in CommonLabels for a Project.

Common labes are key aspect of the Technical Overview document.
2024-10-16 17:40:40 -07:00
Jeff McCune
2ad0c2a93e render: refactor tm to typeMeta
Gary and I noted tm wasn't clear when I was showing him code.
2024-10-16 17:11:09 -07:00
Jeff McCune
3a6a04f318 v1alpha4: add projects to author api (#280)
Projects are a key element of the Technical Overview guide, so we need
the schema for them in the Author API.
2024-10-16 12:29:35 -07:00
Jeff McCune
8afeece890 v1alpha4: embed ComponentConfig in Helm, Kustomize, Kubernetes (#280)
For the Author API, it would be nice to define a schema for the fields
common to all component kinds.  Users could then configure all kinds by
unifying the schema into their own platform tree.

This makes a clear use case to extract the common fields back into an
embedded struct like we did in v1alpha3.  I removed the embedded struct
in v1alpha4 because it wasn't clear why it should be separate, but now
the use case is clear, to configure all component kinds.
2024-10-16 12:16:48 -07:00
Jeff McCune
bc9c43a0b9 fix argocd application project 2024-10-15 20:52:56 -07:00
Jeff McCune
5a98c77e4c add argocd.argoproj.io/instance label to resources
But not the ArgoCD Application resource.
2024-10-15 20:41:39 -07:00
Jeff McCune
b3f7de39ec v1alpha4: feedback in case of chart cache dead lock (#280)
Without this patch holos render platform may hang until the overall
context timeout is reached.  This is a problem because the user has no
idea why it's hung.

This patch adds a warning at the 5 second and another at the 10 second
mark indicating the lock may be deadlocked.  The user can then remove
the directory.
2024-10-15 16:43:36 -07:00
Jeff McCune
ca4ecf1b28 v1alpha4: KustomizeConfig Resources and Files (#280)
The Kustomize build plan kind needs to support both copying files from
the component directory and pulling resources from https URL's.  Without
this patch this support is missing from the Author API

With this patch the Kustomize build plan kind has a KustomizeConfig
field with two structs, Files and Resources.  The kustomization
resources list is built up from both of these.

Two transformers are used so we don't affect the GitOps transfomer which
really only needs CommonLabels.

I decided to keep this field exclusive to the Kustomize kind, but it
could replace the Kustomization field of the other kinds as well.
2024-10-15 16:11:05 -07:00
Jeff McCune
9ce28660ce v1alpha4: intermediate kustomization (#280)
Without this patch the user facing API doesn't have a way to kustomize
the output of all the build plan kinds.  This patch ensures the
Kustomization field is present on all of Helm, Kustomize, and
Kubernetes.

This field is inteded for patches and transforms.  The second
kustomization in the transformer sequence is intended for common labels
and annotations, managed by a corresponding field instead of a full on
Kustomization resource.
2024-10-15 15:02:15 -07:00
Jeff McCune
728e8ba06e v1alpha4: default helm chart release to chart name (#280) 2024-10-15 14:47:42 -07:00
Jeff McCune
e4b07dad6d v1alpha4: helm enable hooks default false (#280)
Fix:

could not run: could not marshal json projects/platform/components/cert-manager: cue: marshal error: spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON at internal/builder/builder.go:63
spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON:
    /Users/jeff/Holos/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha4/types_go_gen.cue:235:16
could not run: could not render component: exit status 1 at builder/v1alpha4/builder.go:94
2024-10-15 14:43:05 -07:00
Jeff McCune
b7c0bba2b9 v1alpha4: add resources to schema.cue (#280)
Without this patch kustomize cannot execute because resources from CUE
are incomplete.
2024-10-15 14:41:49 -07:00
Jeff McCune
847ab8441c v1alpha4: add default chart name (#280)
Defaults to the build plan name.
2024-10-15 14:40:36 -07:00
Jeff McCune
5f72af3d53 v1alpha4: link helm to the chart struct (#280) 2024-10-15 14:32:16 -07:00
Jeff McCune
33eed43fd1 v1alpha4: surface kustomize stderr output (#280)
Without this patch kustomize errors aren't surfaced when executing holos
render platform.

This patch gives a fighting chance to the user to figure out what's
going on.  The stderr is copied, logged, and surfaced up to the parent
holos render platform command.
2024-10-15 14:32:11 -07:00
Jeff McCune
d2fbbdd1cc logger: log the process pid (#280)
Useful when troubleshooting concurrent processes.
2024-10-15 14:31:42 -07:00
Jeff McCune
e42da118dc v1alpha4: add Helm and Kustomize to author api (#280)
Previously the #Helm and #Kustomize build plan helpers were not defined
in the v1alpha4 Author API.  We need this definition to update the
Quickstart guide for v1alpha4 from v1alpha3.

This patch defines the #Helm and #Kustomize helpers in the Author API
similar to how #Kubernetes is defined.
2024-10-15 10:32:16 -07:00
Jeff McCune
7d36567dcf v1alpha4: define Kubernetes in author api (#280)
Previously #Kubernetes was defined in the platform code.  This is a
problem because every platform engineer would need to copy and paste
this code.

This patch moves the #Kubernetes helper into the cue.mod directory so it
can be imported and used ergonomically.
2024-10-14 20:45:04 -07:00
Jeff McCune
bee698bebe v1alpha4: add platform to the author api (#280)
This patch gets the Author API rendering the namespaces component in the
Bank of Holos guide.  It's not the final form of the API yet, we still
need to decide how best to expose the Kubernetes, Helm, and Kustomize
definitions.

I'm thinking we abstract away the transformers and generators within the
Author API Kubernetes definition.
2024-10-14 17:19:49 -07:00
Jeff McCune
58df0626d0 v1alpha4: plumb --write-to flag from platform (#280)
Without this patch the --write-to flag can't be controlled from the
PlatformSpec in the CoreAPI.  We need to surface this for the ArgoConfig
struct in the AuthorAPI.

That is to say, in v1alpha3 the --write-to flag was previously assumed
to be deploy/ in ArgoConfig using the DeployFiles functionality.  We no
longer have DeployFiles in Core API v1alpha4, all artifacts are instead
written relative to the --write-to flag.  Still, we need to expose this
flag in the PlatformSpec so users can use something other than the
deploy directory.
2024-10-14 15:16:36 -07:00
Jeff McCune
c817a24704 v1alpha4: improve core api documentation 2024-10-10 20:37:38 -07:00
Jeff McCune
bd56f3118c v0.96.0 - v1alpha4 Core API
Release v0.96.0 with the new Core API design.
2024-10-10 13:55:06 -07:00
Jeff McCune
d3aa748e92 v1alpha4: add file generator (#277)
Previously the file generator was unimplemented.  This patch implements
it as a simple file read into the ArtifactMap for use by the Kustomize
or Join transformers.

With this patch all v1alpha4 Core API features are implemented.
Resources, Helm, and File generators.  Kustomize and Join transformers.
2024-10-10 13:52:08 -07:00
Jeff McCune
6738248756 v1alpha4: fix blank log lines
Blank lines show up in the output which is confusing. This patch fixes
the only source location identified with the following command.

    export HOLOS_LOG_LEVEL=debug
    export HOLOS_LOG_FORMAT=json
    holos render platform ./platform 2>&1 | jq -r 'select (.msg == "")'
2024-10-10 13:44:08 -07:00
Jeff McCune
011b488775 v1alpha4: cache helm charts by version (#273)
Previously helm charts were cached only by name, which is a problem
because the wrong version would be used when previously cached.

This patch caches charts by name and version to ensure changes in the
version results in pulling the new cached version.  It is the user's
responsibility to remove old versions.

This patch also ensures only a single go routine can run cacheChart() at
a time across processes.  This is necessary when rendering a platform
because multiple processes will run the Helm generator concurrently, for
example when the same chart is used for multiple environments or
customers.

The mkdir system call serves as the locking mechanism, which is robust
and atomic on all commonly used file systems.
2024-10-10 12:15:20 -07:00
Jeff McCune
c8d89f3291 v1alpha4: add helm chart generator
Previously the helm generator was not implemented and returned an error.
This patch is a first pass copying the helm method from
internal/render/helm.go

Basic testing performed with a podinfo chart.  It works as the previous
versions in v1alpha3 and before works.  This patch does not address the
cached version issue in #273
2024-10-10 08:25:13 -07:00
Jeff McCune
a44ebe5171 refactor Artifact to use strings instead of FilePath
holos.FilePath is intended for paths relative to the platform root
directory.  We use the Artifact to store lots of stuff not related to
the platform root directory, for example kustomization.yaml in a temp
dir.  Most entries are not relative to the platform root directory given
the implicit cfg.WriteTo prefix.
2024-10-09 20:16:13 -07:00
Jeff McCune
66a3b6a874 improve error of missing key
Previously:

  could not run: could not build dev-join: could not get foo.yaml: not set at builder/v1alpha4/builder.go:180

This is confusing because set has nothing to do with the missing input
from the cue code the user writes.

Result:

  could not run: could not build test-join: missing foo.yaml at builder/v1alpha4/builder.go:180

This is better because it at doesn't distract the user from the fact
they're missing a foo.yaml generator output to align with the
transformer input.
2024-10-09 19:39:33 -07:00
Jeff McCune
0886788238 Merge pull request #272 from holos-run/jeff/268-v1alpha4
v1alpha4: add join transformer
2024-10-09 17:45:28 -07:00
Jeff McCune
4fd6785a10 v1alpha4: add Join transformer
The Join transformer was not implemented.  This patch completes the
transformers we support, the Join and Kustomize transformers.
2024-10-09 17:42:56 -07:00
Jeff McCune
e4695fa204 artifact: add Save() method to save artifacts
The code was inlined in a number of places, it makes sense to move it to
the interface.  It'll also make it easier to test, we can provide a null
writer concrete value.
2024-10-09 16:53:31 -07:00
Jeff McCune
4cd9395e6c v1alpha4: add concurrent build plan artifacts
Previously the Artifact collection was processed sequentially.  This
patch provides a modest performance improvement, about 16% faster for
our simple 2 artifact use case, by processing each artifact
concurrently.
2024-10-09 16:15:46 -07:00
Jeff McCune
6f4f355ee0 Merge pull request #270 from holos-run/jeff/268-v1alpha4
v1alpha4 - Resources and Kustomize
2024-10-09 15:28:13 -07:00
Jeff McCune
0fde16f477 v1alpha4: fix lint 2024-10-09 14:43:03 -07:00
Jeff McCune
426b4323f7 v1alpha4: inject component build plan name from platform
Platform rendering provides poor user feedback:

```
❯ holos render platform ./platforms/minimal
rendered namespaces for cluster local in 143.068583ms
rendered namespaces for cluster local in 143.861834ms
rendered namespaces for cluster local in 144.072666ms
rendered namespaces for cluster local in 144.219417ms
rendered platform in 144.326625ms
```

We want to see the metadata.name field of each BuildPlan.  This patch
injects the build plan name from the platform spec to make the name
available through the end to end platform rendering process.

Result:

```
❯ holos render platform ./platforms/minimal
rendered stage-namespaces for cluster local in 146.078375ms
rendered prod-namespaces for cluster local in 146.544583ms
rendered test-namespaces for cluster local in 147.0535ms
rendered dev-namespaces for cluster local in 147.499166ms
rendered platform in 147.553875ms
```
2024-10-09 14:06:42 -07:00
Jeff McCune
ee1e4988a6 v1alpha4: write fully rendered build plan artifacts
With this patch the first use case of CUE Resources + Kustomize is fully
working, artifacts are written into the deploy directory.

❯ holos render platform ./platforms/minimal
rendered namespaces for cluster local in 143.068583ms
rendered namespaces for cluster local in 143.861834ms
rendered namespaces for cluster local in 144.072666ms
rendered namespaces for cluster local in 144.219417ms
rendered platform in 144.326625ms

The output indicates we need to plumb the BuildPlan metadata.name from
the PlatfromSpec through to the render component command.  This is
necessary so we can report the correct name instead of just the base
path.
2024-10-09 14:06:42 -07:00
Jeff McCune
5c391e8444 v1alpha4: refactor error handing for resources and kustomize
Consistently use errors.Format("%s: %w", msg, err)
2024-10-09 14:06:41 -07:00
Jeff McCune
aba1b44f4d v1alpha4: fix resources manifest generation
Without this patch holos writes a single yaml document that is a list.
It needs to write a file that contains multiple documents, each document
a map[string]any representing the kubernetes resource.

This patch fixes the problem.  With this patch kustomize fully executes.
2024-10-09 14:06:41 -07:00
Jeff McCune
44f9615a93 v1alpha4: implement kustomize transformer 2024-10-09 14:06:41 -07:00
Jeff McCune
69a6d2acad v1alpha4: implement resources generator 2024-10-09 14:06:40 -07:00
Jeff McCune
d2c94dc8df refactor artifact manifest field to inputs and output
The manifest field isn't clear.

Much more clear to have generators produce one Output.  Transformers
take multiple Inputs and produce one Output.

The final Transformer, or a single Generator, must produce the final
Artifact.

The Inputs and Output naming to produce an Artifact makes clear the
rendering pipeline we're implementing.

This also makes clear that multiple generators must have at least one
transformer to produce the final output artifact.  We model a simple
Join transformer for this case, which is what `holos` was implicitly
doing previously.
2024-10-09 14:06:40 -07:00
Jeff McCune
01720b38fc refactor BuildContext back to Component
Component makes much more sense, that's the domain terminology we use.
BuildContext was meant to be re-used elsewhere, but we never did so the
name serves no purpose.
2024-10-09 14:06:40 -07:00
Jeff McCune
632e3c2725 refactor BuildStep to Artifact
Step doesn't make sense, they're not sequential.  They're meant to be
concurrently produced, and the whole point is to produce one artifact so
we might as well call it an Artifact.

```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
  name: prod-namespaces
spec:
  component: projects/platform/components/namespaces
  artifacts:
    - artifact: clusters/no-cluster/components/prod-namespaces/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: resources.gen.yaml
          resources:
            Namespace:
              prod-jeff:
                metadata:
                  name: prod-jeff
                  labels:
                    kubernetes.io/metadata.name: prod-jeff
                kind: Namespace
                apiVersion: v1
              prod-gary:
                metadata:
                  name: prod-gary
                  labels:
                    kubernetes.io/metadata.name: prod-gary
                kind: Namespace
                apiVersion: v1
              prod-nate:
                metadata:
                  name: prod-nate
                  labels:
                    kubernetes.io/metadata.name: prod-nate
                kind: Namespace
                apiVersion: v1
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
    - artifact: clusters/no-cluster/gitops/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: application.gen.yaml
          resources:
            Application:
              argocd:
                apiVersion: argoproj.io/v1alpha1
                kind: Application
                metadata:
                  name: prod-namespaces
                  namespace: argocd
                spec:
                  destination:
                    server: https://kubernetes.default.svc
                  project: default
                  source:
                    path: examples/v1alpha4/deploy/clusters/no-cluster/components/prod-namespaces
                    repoURL: https://github.com/holos-run/bank-of-holos
                    targetRevision: main
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
```
2024-10-09 14:06:39 -07:00
Jeff McCune
18e0b48012 refactor generators and transformers
The repeated enabled booleans and file fields are awkward.  It's clear
it's three separate things smashed into one.

kustomize isn't really a generator.  It's useless because there is no
way to reference a plain file in a component directory.

This patch replaces the kustomize generator with a file generator which
simply reads one single file.  Multiple of these generators may be used
to read one or more files.

Then, kustomize may transform these generated files, which are generated
by simply reading from the filesystem.

This API is much improved over the previous.

```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
  name: prod-namespaces
spec:
  component: projects/platform/components/namespaces
  steps:
    - artifact: clusters/no-cluster/components/prod-namespaces/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: resources.gen.yaml
          resources:
            Namespace:
              prod-jeff:
                metadata:
                  name: prod-jeff
                  labels:
                    kubernetes.io/metadata.name: prod-jeff
                kind: Namespace
                apiVersion: v1
              prod-gary:
                metadata:
                  name: prod-gary
                  labels:
                    kubernetes.io/metadata.name: prod-gary
                kind: Namespace
                apiVersion: v1
              prod-nate:
                metadata:
                  name: prod-nate
                  labels:
                    kubernetes.io/metadata.name: prod-nate
                kind: Namespace
                apiVersion: v1
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
    - artifact: clusters/no-cluster/gitops/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: application.gen.yaml
          resources:
            Application:
              argocd:
                apiVersion: argoproj.io/v1alpha1
                kind: Application
                metadata:
                  name: prod-namespaces
                  namespace: argocd
                spec:
                  destination:
                    server: https://kubernetes.default.svc
                  project: default
                  source:
                    path: examples/v1alpha4/deploy/clusters/no-cluster/components/prod-namespaces
                    repoURL: https://github.com/holos-run/bank-of-holos
                    targetRevision: main
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
```
2024-10-09 14:06:39 -07:00
Jeff McCune
7dfa9dcb93 remove unnecessary buildstep name
The manifest field is the critical piece of information, not an
arbitrary name.
2024-10-09 14:06:17 -07:00
Jeff McCune
3dedd857ea add manifest field to each build step
A build step either produces kubernetes objects or a gitops manifest.
Both are effectively the same, they're just kubernetes resources.

For the use case of applying common labels to both, we'll have the
Author API pass the same Kustomization to two separate build steps.  One
step to produce the resources, a second to produce the argocd
application or flux kustomization.
2024-10-09 14:06:17 -07:00
Jeff McCune
cd379167cc don't provide a default value for files
Leave this for the author api to avoid always including them in the
exported output.
2024-10-09 14:06:16 -07:00
Jeff McCune
52a5348f82 add build plan build step name
Each step produces a manifest and a gitops file, so we need a unique
name for each step.  The most common case will be a single build step
matching the name of the build plan itself.
2024-10-09 14:06:16 -07:00
Jeff McCune
7c5c8fe692 add fields to store generator output and transformer input
The kustomize transformer needs a filename to store the output from
generators so it has an input for the transformer.  This patch adds
fields for each kind of generator so the kustomize.#Kustomization can be
configured with the files `holos` will write generated output to.
2024-10-09 14:06:15 -07:00
Jeff McCune
ec371ed688 inject missing component tag from render platform to render component 2024-10-09 14:06:15 -07:00
Jeff McCune
1984410577 lift up component path to build plan spec
Hard to use when it's deep inside each build step.
2024-10-09 14:06:15 -07:00
Jeff McCune
438e01fbad replace structpb.Struct with map[string]any
So we can decode using a plain json decoder without needing to marshal
the api objects into yaml from CUE.
2024-10-09 14:06:14 -07:00
Jeff McCune
d122cadae6 render: draft 1 of v1alpha4 core api (#268)
First draft, now to try and wire it up to a platform and namespaces
component.
2024-10-09 14:06:14 -07:00
Jeff McCune
53fcdf307b render: build plan builder (#268)
This patch implements the v1alpha4 component rendering builder for a
component BuildPlan.  We don't yet have the CUE definitions, so this
hasn't been end to end tested yet, the next step is defining the
generators and transforms in the core API BuildPlan.
2024-10-08 06:15:15 -07:00
Jeff McCune
5e6bb96147 render: plumb v1alpha4 component (#268)
This patch plumbs the switch statement to branch on a v1alpha4
BuildPlan.  Tags need to be passed from the render platform subcommand
to the render component subcommand via the --tags argument.
2024-10-07 17:23:21 -07:00
Jeff McCune
94d03f9c59 render: render v1alpha4 platform (#268)
This patch implements minimal rendering of a v1alpha4 platform using the
new render.Builder interface.

Tags aren't wired up yet, but this patch does cleanly separate Builder
interface from the Artifacts.  Platform rendering doesn't have an
artifact itself, all artifacts are produced by rendering each component,
so we'll see how that works when we make the same changes to component
rendering, breaking it down to a render.Builder interface that sets
values in an Artifact.
2024-10-07 16:09:06 -07:00
Jeff McCune
0c01c9177d render: switch platform on api version (#268)
The holos cli does not use an interface to handle different Platform api
versions.  This makes it difficult to evolve the API in a backwards
compatible way.

This patch adds a top level switch statement to the `holos render
platform` command.  The switch discriminates on the Platform API
version.  v1alpha3 and earlier are classified as legacy versions and
will use the existing strict types.  v1alpha4 and later versions will
use an interface to render the platform, allowing for multiple types to
implement the platform rendering interface.
2024-10-07 16:08:59 -07:00
Gary Larizza
36f542da7a Merge pull request #269 from holos-run/gl/landing-page-condense
Make the landing page more concise
2024-10-07 14:17:08 -07:00
Gary Larizza
e41f0aa70c Make the landing page more concise
PROBLEM:

The landing page contains a lot of text, and much of that text was
written before we refined our messaging within the guides and technical
overview pages.

SOLUTION:

* Whittle down landing page text to only the key messages we want to convey.
* Provide messaging bullets for the features.
* Steer folks (via links) to the quickstart guide or technical overview document.

OUTCOME:

Visitors don't need to wade through a lot of text to receive key
messaging talking points or links to the pages they should read.
2024-10-07 11:17:21 -07:00
Jeff McCune
351c8ba74a docs: landing page learn more button to technical overview
Instead of the introduction.
2024-10-04 13:15:28 -07:00
Gary Larizza
a0e0b5bb75 Merge pull request #267 from holos-run/gl/diagram-landing-page
Add Holos diagram to landing page
2024-10-04 12:44:23 -07:00
Gary Larizza
7bc94e314c Add Holos diagram to landing page
PROBLEM:

There's a lot of text to grok on the landing page. A diagram would help
to visually convey what Holos does.

SOLUTION:

* Create a diagram
* Add to landing page

OUTCOME:

A visual aide is present on the landing page that helps explain where
Holos sits.
2024-10-04 12:25:02 -07:00
Jeff McCune
9681ce02e7 docs: clean up tech overview pass 3 (#263) 2024-10-04 08:56:44 -07:00
Jeff McCune
07e8e9f5f3 docs: clean up tech overview pass 2 (#263) 2024-10-03 16:09:13 -07:00
Jeff McCune
437d8a7824 docs: clean up tech overview pass 1 (#263) 2024-10-03 09:56:43 -07:00
Jeff McCune
6cc8214636 docs: fix spelling in technical overview (#263) 2024-10-02 11:29:14 -07:00
Jeff McCune
7d8f324014 blog: add holos technical overview article (#263) 2024-10-02 11:22:08 -07:00
Jeff McCune
8555d56f8c website: enable gtag 2024-09-26 11:55:23 -07:00
Jeff McCune
5884d720f2 website: add mailing lists to footer 2024-09-26 11:38:58 -07:00
Jeff McCune
92d274ced1 website: add support page 2024-09-26 11:30:50 -07:00
Jeff McCune
4dd78bd826 guides: add rendering pipeline diagram to quickstart 2024-09-24 15:24:34 -07:00
Jeff McCune
aeb8fd8e72 blog: add launch article 2024-09-24 15:05:16 -07:00
Gary Larizza
1e9744f748 website: update quickstart narrative (#260)
PROBLEM:

The Quickstart is lacking narrative tying the changes we're asking
people to make to the underlying organizational problems.

SOLUTION:

Improve the narrative to surface the problems we are solving and how
this affects the different teams at the Bank of Holos

OUTCOME:

Clarity on the problems the quickstart is solving.

Closes: #259
2024-09-24 11:37:24 -07:00
Jeff McCune
a95abe65f6 website: refine introduction to be more concise 2024-09-24 07:10:41 -07:00
Jeff McCune
c58510b92c website: refine landing to be more concise 2024-09-24 06:59:14 -07:00
Jeff McCune
302a7bfcf0 guides: fix grammar and tweak the change a service guide 2024-09-23 20:48:35 -07:00
Jeff McCune
decbbaab0f guides: fix typos in change a service guide 2024-09-23 20:04:58 -07:00
Jeff McCune
822f599202 guides: move reset cluster step in change a service
Move to the correct place, after forking the repo.
2024-09-23 20:02:03 -07:00
Jeff McCune
6a47edbc3d guides: Add Change Service Guide (#253)
This patch also adds Organization to the Author API as an example of a
regular expression constraint.
2024-09-23 19:54:46 -07:00
Jeff McCune
67d00f1dd4 website: get rid of What is Holos? on landing page 2024-09-23 14:08:09 -07:00
Jeff McCune
bfa02cd6ed website: introduction (#258) 2024-09-23 13:45:28 -07:00
Jeff McCune
2d8ca474f3 website: Landing Page 3 (#256) 2024-09-23 10:08:25 -07:00
Jeff McCune
c7cd6f5190 website: Landing Page 2 (#256) 2024-09-20 17:14:34 -07:00
Jeff McCune
21e3e6f5e4 website: Landing Page 1 (#256) 2024-09-20 14:57:40 -07:00
Jeff McCune
16a4f89c2f doc: clarify story in the deploy a service guide pass 2
Focus on the migration team and platform team.
2024-09-20 11:12:41 -07:00
Jeff McCune
f15dea5ee7 doc: clarify story in the deploy a service guide
Focus on the migration team and platform team.
2024-09-20 11:02:41 -07:00
Jeff McCune
a3bbadd1f5 doc: apply manifests for deploy a service guide
This gets us through to the end with podinfo deployed.  Need to tell the
story of the migration team a bit better though, working with the
platform team to expose the service.
2024-09-20 09:42:00 -07:00
Jeff McCune
30cbb0d537 doc: add deploy a service guide
Covers wrapping a helm chart with a Holos Component.
2024-09-19 21:31:45 -07:00
71 changed files with 7633 additions and 537 deletions

View File

@@ -5,25 +5,35 @@
"mdx"
],
"words": [
"acraccesstokens",
"admissionregistration",
"anthos",
"apiextensions",
"applicationset",
"applicationsets",
"appproject",
"appprojects",
"argoproj",
"authcode",
"authorizationpolicies",
"authpolicy",
"authproxy",
"authroutes",
"balancereader",
"buildplan",
"cainjector",
"CAROOT",
"certificaterequests",
"certificatesigningrequests",
"clsx",
"clusterexternalsecrets",
"clusterissuer",
"clusterissuers",
"clusterrole",
"clusterrolebinding",
"clustersecretstores",
"CNCF",
"CODEOWNERS",
"configmap",
"cookiesecret",
"coredns",
@@ -40,16 +50,22 @@
"devicecode",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtokens",
"entgo",
"envoyfilters",
"errgroup",
"etcdsnapshotfiles",
"externalsecret",
"externalsecrets",
"fctr",
"fieldmaskpb",
"flushcache",
"gatewayclasses",
"gcraccesstokens",
"gendoc",
"ggnpl",
"ghaction",
"githubaccesstokens",
"gitops",
"godoc",
"golangci",
@@ -57,9 +73,14 @@
"grpcreflect",
"grpcroutes",
"grpcurl",
"healthz",
"helmchartconfigs",
"helmcharts",
"Hiera",
"holos",
"holoslogger",
"horizontalpodautoscaler",
"Hostnames",
"httpbin",
"httproute",
"httproutes",
@@ -70,6 +91,7 @@
"jeffmccune",
"jetstack",
"Jsonnet",
"kfbh",
"killall",
"kubeadm",
"kubeconfig",
@@ -79,6 +101,7 @@
"kustomize",
"ldflags",
"leaderelection",
"ledgerwriter",
"libnss",
"loadbalancer",
"mattn",
@@ -92,6 +115,7 @@
"mxcl",
"myhostname",
"nameserver",
"nolint",
"organizationconnect",
"orgid",
"otelconnect",
@@ -102,6 +126,7 @@
"pipefail",
"PKCE",
"platformconnect",
"podcli",
"poddisruptionbudget",
"podinfo",
"portmapping",
@@ -110,9 +135,12 @@
"protojson",
"proxyconfigs",
"Pulumi",
"pushsecrets",
"putenv",
"qjbp",
"quickstart",
"QVRFLS",
"readyz",
"referencegrant",
"referencegrants",
"requestauthentications",
@@ -121,15 +149,19 @@
"ropc",
"seccomp",
"SECRETKEY",
"secretstore",
"secretstores",
"serverlb",
"serverside",
"serviceaccount",
"servicebindings",
"serviceentries",
"spanid",
"spiffe",
"startupapicheck",
"statefulset",
"stefanprodan",
"struct",
"structpb",
"subjectaccessreviews",
"svclb",
@@ -143,7 +175,9 @@
"Tokener",
"Traceid",
"traefik",
"transactionhistory",
"uibutton",
"unstage",
"untar",
"Upsert",
"urandom",
@@ -152,6 +186,7 @@
"userdata",
"userservice",
"validatingwebhookconfiguration",
"vaultdynamicsecrets",
"virtualservices",
"wasmplugins",
"workloadentries",

View File

@@ -161,3 +161,66 @@ type Platform struct {
// platform operators to define.
Domain string `cue:"string | *\"holos.localhost\""`
}
// Organization represents organizational metadata useful across the platform.
type Organization struct {
Name string
DisplayName string
Domain string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
type Projects map[core.NameLabel]Project
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[core.NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[core.NameLabel]Hostname
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
type Owner struct {
Name string
Email string
}
// Namespace represents a Kubernetes namespace.
type Namespace struct {
Name string
}
// Hostname represents the left most dns label of a domain name.
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}

View File

@@ -0,0 +1,290 @@
// # Author API
//
// Package v1alpha4 contains ergonomic CUE definitions for Holos component
// authors. These definitions serve as adapters to produce [Core API] resources
// for the holos command line tool.
//
// [Core API]: https://holos.run/docs/api/core/v1alpha4/
package v1alpha4
import core "github.com/holos-run/holos/api/core/v1alpha4"
//go:generate ../../../hack/gendoc
// Platform assembles a Core API [Platform] in the Resource field for the holos
// render platform command. Use the Components field to register components
// with the platform using a struct. This struct is converted into a list for
// final output to holos.
//
// See related:
//
// - [Component] collection of components composing the platform.
// - [Platform] resource assembled for holos to process.
//
// [Platform]: https://holos.run/docs/api/core/v1alpha4/#Platform
// [Component]: https://holos.run/docs/api/core/v1alpha4/#Component
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// Cluster represents a cluster managed by the Platform.
type Cluster struct {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
Name string `json:"name"`
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
Primary bool `json:"primary" cue:"true | *false"`
}
// Fleet represents a named collection of similarly configured Clusters. Useful
// to segregate workload clusters from their management cluster.
type Fleet struct {
Name string `json:"name"`
// Clusters represents a mapping of Clusters by their name.
Clusters map[string]Cluster `json:"clusters" cue:"{[Name=_]: name: Name}"`
}
// StandardFleets represents the standard set of Clusters in a Platform
// segmented into Fleets by their purpose. The management Fleet contains a
// single Cluster, for example a GKE autopilot cluster with no workloads
// deployed for reliability and cost efficiency. The workload Fleet contains
// all other Clusters which contain workloads and sync Secrets from the
// management cluster.
type StandardFleets struct {
// Workload represents a Fleet of zero or more workload Clusters.
Workload Fleet `json:"workload" cue:"{name: \"workload\"}"`
// Management represents a Fleet with one Cluster named management.
Management Fleet `json:"management" cue:"{name: \"management\"}"`
}
// ArgoConfig represents the ArgoCD GitOps configuration associated with a
// [BuildPlan]. Useful to define once at the root of the Platform configuration
// and reuse across all components.
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
type ArgoConfig struct {
// Enabled causes holos to render an Application resource when true.
Enabled bool `cue:"true | *false"`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// Root represents the path from the git repository root to the WriteTo output
// directory, the behavior of the holos render component --write-to flag and
// the Core API Component WriteTo field. Used as a prefix for the
// Application.spec.source.path field.
Root string `cue:"string | *\"deploy\""`
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
// AppProject represents the ArgoCD Project to associate the Application with.
AppProject string `cue:"string | *\"default\""`
}
// Organization represents organizational metadata useful across the platform.
type Organization struct {
Name string
DisplayName string
Domain string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
// Kubernetes provides a [BuildPlan] via the Output field which contains inline
// API Objects provided directly from CUE in the Resources field of
// [ComponentConfig].
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Helm provides a [BuildPlan] via the Output field which generates manifests
// from a helm chart with optional mix-in resources provided directly from CUE
// in the Resources field.
//
// This definition is a convenient way to produce a [BuildPlan] composed of
// three [Resources] generators with one [Kustomize] transformer.
//
// See related:
//
// - [ComponentConfig]
// - [Chart]
// - [Values]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Chart]: https://holos.run/docs/api/core/v1alpha4/#Chart
// [Values]: https://holos.run/docs/api/core/v1alpha4/#Values
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize provides a [BuildPlan] via the Output field which generates
// manifests from a kustomize kustomization with optional mix-in resources
// provided directly from CUE in the Resources field.
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// ComponentConfig represents the configuration common to all kinds of
// component.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// See the following resources for additional details:
//
// - [Resources]
// - [ArgoConfig]
// - [KustomizeConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Resources]: https://holos.run/docs/api/core/v1alpha4/#Resources
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Component represents the path to the component producing the BuildPlan.
Component string
// Cluster represents the name of the cluster this BuildPlan is for.
Cluster string
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// ArgoConfig represents the ArgoCD GitOps configuration for this BuildPlan.
ArgoConfig ArgoConfig
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
// Namespace manages the metadata.namespace field on all resources except the
// ArgoCD Application.
Namespace string `json:",omitempty"`
// KustomizeConfig represents the configuration for kustomize.
KustomizeConfig KustomizeConfig
}
// KustomizeConfig represents the configuration for kustomize post processing.
// The Files field is used to mixing in static manifest files from the component
// directory. The Resources field is used for mixing in manifests from network
// locations urls.
//
// See related:
//
// - [ComponentConfig]
// - [Kustomization]
//
// [Kustomization]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
type Projects map[NameLabel]Project
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[NameLabel]Hostname
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
type Owner struct {
Name string
Email string
}
// Namespace represents a Kubernetes namespace.
type Namespace struct {
Name string
}
// Hostname represents the left most dns label of a domain name.
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}
// NameLabel signals the common use case of converting a struct to a list where
// the name field of each value unifies with the field name of the outer struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -14,6 +14,12 @@ import "google.golang.org/protobuf/types/known/structpb"
// output.
type InternalLabel string
// NameLabel is a unique identifier useful to convert a CUE struct to a list
// when the values have a Name field with a default value. This type is
// intended to indicate the common use case of converting a struct to a list
// where the Name field of the value aligns with the struct field name.
type NameLabel string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type
// checking.
type Kind string

402
api/core/v1alpha4/types.go Normal file
View File

@@ -0,0 +1,402 @@
// # Core API
//
// Package v1alpha4 contains the core API contract between the holos cli and CUE
// configuration code. Platform designers, operators, and software developers
// use this API to write configuration in CUE which holos loads. The Core API
// is declarative. Each resource represents a desired state necessary for holos
// to fully render Kubernetes manifests into plain files.
//
// The following resources provide important context for the Core API. The
// [Author API] is intended for component authors as a convenient adapter for
// the Core API resources Holos expects.
//
// 1. [Technical Overview]
// 2. [Quickstart]
// 3. [Author API]
//
// # Platform
//
// [Platform] defines the complete configuration of a platform. A platform
// represents a [Component] collection.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
//
// # Component
//
// A [Component] is the combination of CUE code along one path relative to the
// platform root directory plus data injected from the [PlatformSpec] via CUE tags.
// The platform configuration root is the directory containing cue.mod.
//
// A [Component] always produces exactly one [BuildPlan].
//
// # BuildPlan
//
// A [BuildPlan] contains an [Artifact] collection. A BuildPlan often produces
// two artifacts, one containing the fully rendered Kubernetes API resources,
// the other containing an additional resource to manage the former with GitOps.
// For example, a BuildPlan for a podinfo component produces a manifest
// containing a Deployment and a Service, along with a second manifest
// containing an ArgoCD Application.
//
// Inspect a BuildPlan resource holos render component would process by executing:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// # Artifact
//
// An [Artifact] is one fully rendered manifest file produced from the final
// [Transformer] in a sequence of transformers. An Artifact may also be
// produced directly from a [Generator], but this use case is uncommon.
//
// # Transformer
//
// A [Transformer] takes multiple inputs from prior [Generator] or [Transformer]
// outputs, then transforms the data into one output. [Kustomize] is the most
// commonly used transformer, though a simple [Join] is also supported.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// # Generators
//
// A [Generator] generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
// [Author API]: https://holos.run/docs/api/author/
// [Quickstart]: https://holos.run/docs/quickstart/
// [Technical Overview]: https://holos.run/docs/technical-overview/
package v1alpha4
//go:generate ../../../hack/gendoc
// BuildPlan represents a build plan for holos to execute. Each [Platform]
// component produces exactly one BuildPlan.
//
// One or more [Artifact] files are produced by a BuildPlan, representing the
// fully rendered manifests for the Kubernetes API Server.
//
// # Example BuildPlan
//
// Command:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// Output:
//
// kind: BuildPlan
// apiVersion: v1alpha4
// metadata:
// name: dev-namespaces
// spec:
// component: projects/platform/components/namespaces
// artifacts:
// - artifact: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// generators:
// - kind: Resources
// output: resources.gen.yaml
// resources:
// Namespace:
// dev-jeff:
// metadata:
// name: dev-jeff
// labels:
// kubernetes.io/metadata.name: dev-jeff
// kind: Namespace
// apiVersion: v1
// transformers:
// - kind: Kustomize
// inputs:
// - resources.gen.yaml
// output: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// kustomize:
// kustomization:
// commonLabels:
// holos.run/component.name: dev-namespaces
// resources:
// - resources.gen.yaml
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha4\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Component represents the component that produced the build plan.
// Represented as a path relative to the platform root.
Component string `json:"component"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
}
// Generator generates an intermediate manifest for a [Artifact].
//
// Each Generator in a [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// Refer to [Resources], [Helm], and [File].
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents a kubernetes resources [Generator] from CUE.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Transformer transforms [Generator] manifests within a [Artifact].
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContent represents file contents.
type FileContent string
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// NameLabel is a unique identifier useful to convert a CUE struct to a list
// when the values have a Name field with a default value. NameLabel indicates
// the common use case of converting a struct to a list where the Name field of
// the value aligns with the outer struct field name.
//
// For example:
//
// Outer: [NAME=_]: Name: NAME
type NameLabel string
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha4\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
}
// PlatformSpec represents the specification of a [Platform]. Think of a
// platform spec as a [Component] collection for multiple kubernetes clusters
// combined with the user-specified Platform Model.
type PlatformSpec struct {
// Components represents a list of holos components to manage.
Components []Component `json:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan].
// Component carries information injected from holos render platform to holos
// render component to produce each [BuildPlan].
//
// All of these fields are passed to the holos render component command using
// flags, which in turn are injected to CUE using tags. For clarity, CUE field
// and tag names should match the struct json tag names below.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_name" to set the BuildPlan metadata.name field. Necessary for clear
// user feedback during platform rendering.
Name string `json:"name"`
// Component represents the path of the component relative to the platform
// root. Injected as the tag variable "holos_component".
Component string `json:"component"`
// Cluster is the cluster name to provide when rendering the component.
// Injected as the tag variable "holos_cluster".
Cluster string `json:"cluster"`
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
// Injected as the tag "holos_model".
Model map[string]any `json:"model"`
// Tags represents cue @tag variables injected into the holos render component
// command from the holos render platform command. Tags with a "holos_"
// prefix are reserved for use by the Holos Authors.
Tags map[string]string `json:"tags,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
}
// Tags represents standardized fields injected into the component [BuildPlan]
// from the [Platform].
//
// Note, tags should have a reasonable default value to easily use cue eval and
// cue export without needing to make a bunch of decisions about tag values.
//
// Example:
//
// import core "github.com/holos-run/holos/api/core/v1alpha4"
// _Tags: core.#Tags & {
// cluster: _ @tag(cluster, type=string)
// environment: _ @tag(environment, type=string)
// component: _ @tag(component, type=string)
// name: _ @tag(name, type=string)
// }
type Tags struct {
// Name represents the BuildPlan metadata.name field injected from the Platform.
Name string `json:"name" cue:"string | *\"no-name\""`
// Cluster represents the cluster name injected from
Cluster string `json:"cluster" cue:"string | *\"no-cluster\""`
// Environment represents the build plan environment.
Environment string `json:"environment" cue:"string | *\"no-environment\""`
// Component represents the path of the component relative to the platform root.
Component string `json:"component" cue:"string | *\"no-component\""`
}

View File

@@ -15,9 +15,16 @@ Package v1alpha3 contains CUE definitions intended as convenience wrappers aroun
- [type ComponentFields](<#ComponentFields>)
- [type Fleet](<#Fleet>)
- [type Helm](<#Helm>)
- [type Hostname](<#Hostname>)
- [type Kubernetes](<#Kubernetes>)
- [type Kustomize](<#Kustomize>)
- [type Namespace](<#Namespace>)
- [type Organization](<#Organization>)
- [type OrganizationStrict](<#OrganizationStrict>)
- [type Owner](<#Owner>)
- [type Platform](<#Platform>)
- [type Project](<#Project>)
- [type Projects](<#Projects>)
- [type StandardFleets](<#StandardFleets>)
@@ -146,6 +153,26 @@ type Helm struct {
}
```
<a name="Hostname"></a>
## type Hostname {#Hostname}
Hostname represents the left most dns label of a domain name.
```go
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}
```
<a name="Kubernetes"></a>
## type Kubernetes {#Kubernetes}
@@ -172,6 +199,59 @@ type Kustomize struct {
}
```
<a name="Namespace"></a>
## type Namespace {#Namespace}
Namespace represents a Kubernetes namespace.
```go
type Namespace struct {
Name string
}
```
<a name="Organization"></a>
## type Organization {#Organization}
Organization represents organizational metadata useful across the platform.
```go
type Organization struct {
Name string
DisplayName string
Domain string
}
```
<a name="OrganizationStrict"></a>
## type OrganizationStrict {#OrganizationStrict}
OrganizationStrict represents organizational metadata useful across the platform. This is an example of using CUE regular expressions to constrain and validate configuration.
```go
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
```
<a name="Owner"></a>
## type Owner {#Owner}
Owner represents the owner of a resource. For example, the name and email address of an engineering team.
```go
type Owner struct {
Name string
Email string
}
```
<a name="Platform"></a>
## type Platform {#Platform}
@@ -196,6 +276,33 @@ type Platform struct {
}
```
<a name="Project"></a>
## type Project {#Project}
Project represents logical grouping of components owned by one or more teams. Useful for the platform team to manage resources for project teams to use.
```go
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[core.NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[core.NameLabel]Hostname
}
```
<a name="Projects"></a>
## type Projects {#Projects}
Projects represents projects managed by the platform team for use by other teams using the platform.
```go
type Projects map[core.NameLabel]Project
```
<a name="StandardFleets"></a>
## type StandardFleets {#StandardFleets}

View File

@@ -0,0 +1,377 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# v1alpha4
```go
import "github.com/holos-run/holos/api/author/v1alpha4"
```
### Author API
Package v1alpha4 contains ergonomic CUE definitions for Holos component authors. These definitions serve as adapters to produce [Core API](<https://holos.run/docs/api/core/v1alpha4/>) resources for the holos command line tool.
## Index
- [type ArgoConfig](<#ArgoConfig>)
- [type Cluster](<#Cluster>)
- [type ComponentConfig](<#ComponentConfig>)
- [type Fleet](<#Fleet>)
- [type Helm](<#Helm>)
- [type Hostname](<#Hostname>)
- [type Kubernetes](<#Kubernetes>)
- [type Kustomize](<#Kustomize>)
- [type KustomizeConfig](<#KustomizeConfig>)
- [type NameLabel](<#NameLabel>)
- [type Namespace](<#Namespace>)
- [type Organization](<#Organization>)
- [type OrganizationStrict](<#OrganizationStrict>)
- [type Owner](<#Owner>)
- [type Platform](<#Platform>)
- [type Project](<#Project>)
- [type Projects](<#Projects>)
- [type StandardFleets](<#StandardFleets>)
<a name="ArgoConfig"></a>
## type ArgoConfig {#ArgoConfig}
ArgoConfig represents the ArgoCD GitOps configuration associated with a [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#buildplan>). Useful to define once at the root of the Platform configuration and reuse across all components.
```go
type ArgoConfig struct {
// Enabled causes holos to render an Application resource when true.
Enabled bool `cue:"true | *false"`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// Root represents the path from the git repository root to the WriteTo output
// directory, the behavior of the holos render component --write-to flag and
// the Core API Component WriteTo field. Used as a prefix for the
// Application.spec.source.path field.
Root string `cue:"string | *\"deploy\""`
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
// AppProject represents the ArgoCD Project to associate the Application with.
AppProject string `cue:"string | *\"default\""`
}
```
<a name="Cluster"></a>
## type Cluster {#Cluster}
Cluster represents a cluster managed by the Platform.
```go
type Cluster struct {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
Name string `json:"name"`
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
Primary bool `json:"primary" cue:"true | *false"`
}
```
<a name="ComponentConfig"></a>
## type ComponentConfig {#ComponentConfig}
ComponentConfig represents the configuration common to all kinds of component.
- [Helm](<#Helm>) charts.
- [Kubernetes](<#Kubernetes>) resources generated from CUE.
- [Kustomize](<#Kustomize>) bases.
See the following resources for additional details:
- [Resources](<https://holos.run/docs/api/core/v1alpha4/#Resources>)
- [ArgoConfig](<#ArgoConfig>)
- [KustomizeConfig](<#KustomizeConfig>)
- [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#BuildPlan>)
```go
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Component represents the path to the component producing the BuildPlan.
Component string
// Cluster represents the name of the cluster this BuildPlan is for.
Cluster string
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// ArgoConfig represents the ArgoCD GitOps configuration for this BuildPlan.
ArgoConfig ArgoConfig
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
// Namespace manages the metadata.namespace field on all resources except the
// ArgoCD Application.
Namespace string `json:",omitempty"`
// KustomizeConfig represents the configuration for kustomize.
KustomizeConfig KustomizeConfig
}
```
<a name="Fleet"></a>
## type Fleet {#Fleet}
Fleet represents a named collection of similarly configured Clusters. Useful to segregate workload clusters from their management cluster.
```go
type Fleet struct {
Name string `json:"name"`
// Clusters represents a mapping of Clusters by their name.
Clusters map[string]Cluster `json:"clusters" cue:"{[Name=_]: name: Name}"`
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm provides a [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#BuildPlan>) via the Output field which generates manifests from a helm chart with optional mix\-in resources provided directly from CUE in the Resources field.
This definition is a convenient way to produce a [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#BuildPlan>) composed of three \[Resources\] generators with one [Kustomize](<#Kustomize>) transformer.
See related:
- [ComponentConfig](<#ComponentConfig>)
- [Chart](<https://holos.run/docs/api/core/v1alpha4/#Chart>)
- [Values](<https://holos.run/docs/api/core/v1alpha4/#Values>)
- [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#BuildPlan>)
```go
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="Hostname"></a>
## type Hostname {#Hostname}
Hostname represents the left most dns label of a domain name.
```go
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}
```
<a name="Kubernetes"></a>
## type Kubernetes {#Kubernetes}
Kubernetes provides a [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#BuildPlan>) via the Output field which contains inline API Objects provided directly from CUE in the Resources field of [ComponentConfig](<#ComponentConfig>).
See related:
- [ComponentConfig](<#ComponentConfig>)
- [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#BuildPlan>)
```go
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize provides a [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#buildplan>) via the Output field which generates manifests from a kustomize kustomization with optional mix\-in resources provided directly from CUE in the Resources field.
See related:
- [ComponentConfig](<#ComponentConfig>)
- [BuildPlan](<https://holos.run/docs/api/core/v1alpha4/#buildplan>)
```go
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
```
<a name="KustomizeConfig"></a>
## type KustomizeConfig {#KustomizeConfig}
KustomizeConfig represents the configuration for kustomize post processing. The Files field is used to mixing in static manifest files from the component directory. The Resources field is used for mixing in manifests from network locations urls.
See related:
- [ComponentConfig](<#ComponentConfig>)
- [Kustomization](<https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/>)
```go
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
}
```
<a name="NameLabel"></a>
## type NameLabel {#NameLabel}
NameLabel signals the common use case of converting a struct to a list where the name field of each value unifies with the field name of the outer struct.
For example:
```
S: [NameLabel=string]: name: NameLabel
S: jeff: _
S: gary: _
S: nate: _
L: [for x in S {x}]
// L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
```
```go
type NameLabel string
```
<a name="Namespace"></a>
## type Namespace {#Namespace}
Namespace represents a Kubernetes namespace.
```go
type Namespace struct {
Name string
}
```
<a name="Organization"></a>
## type Organization {#Organization}
Organization represents organizational metadata useful across the platform.
```go
type Organization struct {
Name string
DisplayName string
Domain string
}
```
<a name="OrganizationStrict"></a>
## type OrganizationStrict {#OrganizationStrict}
OrganizationStrict represents organizational metadata useful across the platform. This is an example of using CUE regular expressions to constrain and validate configuration.
```go
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
```
<a name="Owner"></a>
## type Owner {#Owner}
Owner represents the owner of a resource. For example, the name and email address of an engineering team.
```go
type Owner struct {
Name string
Email string
}
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform assembles a Core API [Platform](<https://holos.run/docs/api/core/v1alpha4/#Platform>) in the Resource field for the holos render platform command. Use the Components field to register components with the platform using a struct. This struct is converted into a list for final output to holos.
See related:
- [Component](<https://holos.run/docs/api/core/v1alpha4/#Component>) collection of components composing the platform.
- [Platform](<https://holos.run/docs/api/core/v1alpha4/#Platform>) resource assembled for holos to process.
```go
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
```
<a name="Project"></a>
## type Project {#Project}
Project represents logical grouping of components owned by one or more teams. Useful for the platform team to manage resources for project teams to use.
```go
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[NameLabel]Hostname
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
}
```
<a name="Projects"></a>
## type Projects {#Projects}
Projects represents projects managed by the platform team for use by other teams using the platform.
```go
type Projects map[NameLabel]Project
```
<a name="StandardFleets"></a>
## type StandardFleets {#StandardFleets}
StandardFleets represents the standard set of Clusters in a Platform segmented into Fleets by their purpose. The management Fleet contains a single Cluster, for example a GKE autopilot cluster with no workloads deployed for reliability and cost efficiency. The workload Fleet contains all other Clusters which contain workloads and sync Secrets from the management cluster.
```go
type StandardFleets struct {
// Workload represents a Fleet of zero or more workload Clusters.
Workload Fleet `json:"workload" cue:"{name: \"workload\"}"`
// Management represents a Fleet with one Cluster named management.
Management Fleet `json:"management" cue:"{name: \"management\"}"`
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -41,6 +41,7 @@ Note that Holos operates as a data pipeline, so the output of a [HelmChart](<#He
- [type Kustomize](<#Kustomize>)
- [type KustomizeBuild](<#KustomizeBuild>)
- [type Metadata](<#Metadata>)
- [type NameLabel](<#NameLabel>)
- [type Platform](<#Platform>)
- [type PlatformMetadata](<#PlatformMetadata>)
- [type PlatformSpec](<#PlatformSpec>)
@@ -328,6 +329,15 @@ type Metadata struct {
}
```
<a name="NameLabel"></a>
## type NameLabel {#NameLabel}
NameLabel is a unique identifier useful to convert a CUE struct to a list when the values have a Name field with a default value. This type is intended to indicate the common use case of converting a struct to a list where the Name field of the value aligns with the struct field name.
```go
type NameLabel string
```
<a name="Platform"></a>
## type Platform {#Platform}

537
doc/md/api/core/v1alpha4.md Normal file
View File

@@ -0,0 +1,537 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# v1alpha4
```go
import "github.com/holos-run/holos/api/core/v1alpha4"
```
### Core API
Package v1alpha4 contains the core API contract between the holos cli and CUE configuration code. Platform designers, operators, and software developers use this API to write configuration in CUE which holos loads. The Core API is declarative. Each resource represents a desired state necessary for holos to fully render Kubernetes manifests into plain files.
The following resources provide important context for the Core API. The [Author API](<https://holos.run/docs/api/author/>) is intended for component authors as a convenient adapter for the Core API resources Holos expects.
1. [Technical Overview](<https://holos.run/docs/technical-overview/>)
2. [Quickstart](<https://holos.run/docs/quickstart/>)
3. [Author API](<https://holos.run/docs/api/author/>)
### Platform
[Platform](<#Platform>) defines the complete configuration of a platform. A platform represents a [Component](<#Component>) collection.
Inspect a Platform resource holos would process by executing:
```
cue export --out yaml ./platform
```
### Component
A [Component](<#Component>) is the combination of CUE code along one path relative to the platform root directory plus data injected from the [PlatformSpec](<#PlatformSpec>) via CUE tags. The platform configuration root is the directory containing cue.mod.
A [Component](<#Component>) always produces exactly one [BuildPlan](<#BuildPlan>).
### BuildPlan
A [BuildPlan](<#BuildPlan>) contains an [Artifact](<#Artifact>) collection. A BuildPlan often produces two artifacts, one containing the fully rendered Kubernetes API resources, the other containing an additional resource to manage the former with GitOps. For example, a BuildPlan for a podinfo component produces a manifest containing a Deployment and a Service, along with a second manifest containing an ArgoCD Application.
Inspect a BuildPlan resource holos render component would process by executing:
```
cue export --out yaml ./projects/platform/components/namespaces
```
### Artifact
An [Artifact](<#Artifact>) is one fully rendered manifest file produced from the final [Transformer](<#Transformer>) in a sequence of transformers. An Artifact may also be produced directly from a [Generator](<#Generator>), but this use case is uncommon.
### Transformer
A [Transformer](<#Transformer>) takes multiple inputs from prior [Generator](<#Generator>) or [Transformer](<#Transformer>) outputs, then transforms the data into one output. [Kustomize](<#Kustomize>) is the most commonly used transformer, though a simple [Join](<#Join>) is also supported.
1. [Kustomize](<#Kustomize>) \- Patch and transform the output from prior generators or transformers. See [Introduction to Kustomize](<https://kubectl.docs.kubernetes.io/guides/config_management/introduction/>).
2. [Join](<#Join>) \- Concatenate multiple prior outputs into one output.
### Generators
A [Generator](<#Generator>) generates Kubernetes resources. [Helm](<#Helm>) and [Resources](<#Resources>) are the most commonly used, often paired together to mix\-in resources to an unmodified Helm chart. A simple [File](<#File>) generator is also available for use with the [Kustomize](<#Kustomize>) transformer.
1. [Resources](<#Resources>) \- Generates resources from CUE code.
2. [Helm](<#Helm>) \- Generates rendered yaml from a [Chart](<#Chart>).
3. [File](<#File>) \- Generates data by reading a file from the component directory.
## Index
- [type Artifact](<#Artifact>)
- [type BuildPlan](<#BuildPlan>)
- [type BuildPlanSpec](<#BuildPlanSpec>)
- [type Chart](<#Chart>)
- [type Component](<#Component>)
- [type File](<#File>)
- [type FileContent](<#FileContent>)
- [type FileContentMap](<#FileContentMap>)
- [type FilePath](<#FilePath>)
- [type Generator](<#Generator>)
- [type Helm](<#Helm>)
- [type InternalLabel](<#InternalLabel>)
- [type Join](<#Join>)
- [type Kind](<#Kind>)
- [type Kustomization](<#Kustomization>)
- [type Kustomize](<#Kustomize>)
- [type Metadata](<#Metadata>)
- [type NameLabel](<#NameLabel>)
- [type Platform](<#Platform>)
- [type PlatformSpec](<#PlatformSpec>)
- [type Repository](<#Repository>)
- [type Resource](<#Resource>)
- [type Resources](<#Resources>)
- [type Tags](<#Tags>)
- [type Transformer](<#Transformer>)
- [type Values](<#Values>)
<a name="Artifact"></a>
## type Artifact {#Artifact}
Artifact represents one fully rendered manifest produced by a [Transformer](<#Transformer>) sequence, which transforms a [Generator](<#Generator>) collection. A [BuildPlan](<#BuildPlan>) produces an [Artifact](<#Artifact>) collection.
Each Artifact produces one manifest file artifact. Generator Output values are used as Transformer Inputs. The Output field of the final [Transformer](<#Transformer>) should have the same value as the Artifact field.
When there is more than one [Generator](<#Generator>) there must be at least one [Transformer](<#Transformer>) to combine outputs into one Artifact. If there is a single Generator, it may directly produce the Artifact output.
An Artifact is processed concurrently with other artifacts in the same [BuildPlan](<#BuildPlan>). An Artifact should not use an output from another Artifact as an input. Each [Generator](<#Generator>) may also run concurrently. Each [Transformer](<#Transformer>) is executed sequentially starting after all generators have completed.
Output fields are write\-once. It is an error for multiple Generators or Transformers to produce the same Output value within the context of a [BuildPlan](<#BuildPlan>).
```go
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
}
```
<a name="BuildPlan"></a>
## type BuildPlan {#BuildPlan}
BuildPlan represents a build plan for holos to execute. Each [Platform](<#Platform>) component produces exactly one BuildPlan.
One or more [Artifact](<#Artifact>) files are produced by a BuildPlan, representing the fully rendered manifests for the Kubernetes API Server.
### Example BuildPlan
Command:
```
cue export --out yaml ./projects/platform/components/namespaces
```
Output:
```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
name: dev-namespaces
spec:
component: projects/platform/components/namespaces
artifacts:
- artifact: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources:
Namespace:
dev-jeff:
metadata:
name: dev-jeff
labels:
kubernetes.io/metadata.name: dev-jeff
kind: Namespace
apiVersion: v1
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
kustomize:
kustomization:
commonLabels:
holos.run/component.name: dev-namespaces
resources:
- resources.gen.yaml
```
```go
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha4\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
}
```
<a name="BuildPlanSpec"></a>
## type BuildPlanSpec {#BuildPlanSpec}
BuildPlanSpec represents the specification of the [BuildPlan](<#BuildPlan>).
```go
type BuildPlanSpec struct {
// Component represents the component that produced the build plan.
// Represented as a path relative to the platform root.
Component string `json:"component"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
}
```
<a name="Chart"></a>
## type Chart {#Chart}
Chart represents a [Helm](<#Helm>) Chart.
```go
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
```
<a name="Component"></a>
## type Component {#Component}
Component represents the complete context necessary to produce a [BuildPlan](<#BuildPlan>). Component carries information injected from holos render platform to holos render component to produce each [BuildPlan](<#BuildPlan>).
All of these fields are passed to the holos render component command using flags, which in turn are injected to CUE using tags. For clarity, CUE field and tag names should match the struct json tag names below.
```go
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_name" to set the BuildPlan metadata.name field. Necessary for clear
// user feedback during platform rendering.
Name string `json:"name"`
// Component represents the path of the component relative to the platform
// root. Injected as the tag variable "holos_component".
Component string `json:"component"`
// Cluster is the cluster name to provide when rendering the component.
// Injected as the tag variable "holos_cluster".
Cluster string `json:"cluster"`
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
// Injected as the tag "holos_model".
Model map[string]any `json:"model"`
// Tags represents cue @tag variables injected into the holos render component
// command from the holos render platform command. Tags with a "holos_"
// prefix are reserved for use by the Holos Authors.
Tags map[string]string `json:"tags,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
}
```
<a name="File"></a>
## type File {#File}
File represents a simple single file copy [Generator](<#Generator>). Useful with a [Kustomize](<#Kustomize>) [Transformer](<#Transformer>) to process plain manifest files stored in the component directory. Multiple File generators may be used to transform multiple resources.
```go
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
}
```
<a name="FileContent"></a>
## type FileContent {#FileContent}
FileContent represents file contents.
```go
type FileContent string
```
<a name="FileContentMap"></a>
## type FileContentMap {#FileContentMap}
FileContentMap represents a mapping of file paths to file contents.
```go
type FileContentMap map[FilePath]FileContent
```
<a name="FilePath"></a>
## type FilePath {#FilePath}
FilePath represents a file path.
```go
type FilePath string
```
<a name="Generator"></a>
## type Generator {#Generator}
Generator generates an intermediate manifest for a [Artifact](<#Artifact>).
Each Generator in a [Artifact](<#Artifact>) must have a distinct Output value for a [Transformer](<#Transformer>) to reference.
Refer to [Resources](<#Resources>), [Helm](<#Helm>), and [File](<#File>).
```go
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm represents a [Chart](<#Chart>) manifest [Generator](<#Generator>).
```go
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
}
```
<a name="InternalLabel"></a>
## type InternalLabel {#InternalLabel}
InternalLabel is an arbitrary unique identifier internal to holos itself. The holos cli is expected to never write a InternalLabel value to rendered output files, therefore use a InternalLabel when the identifier must be unique and internal. Defined as a type for clarity and type checking.
```go
type InternalLabel string
```
<a name="Join"></a>
## type Join {#Join}
Join represents a [Transformer](<#Transformer>) using [bytes.Join](<https://pkg.go.dev/bytes#Join>) to concatenate multiple inputs into one output with a separator. Useful for combining output from [Helm](<#Helm>) and [Resources](<#Resources>) together into one [Artifact](<#Artifact>) when [Kustomize](<#Kustomize>) is otherwise unnecessary.
```go
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
}
```
<a name="Kind"></a>
## type Kind {#Kind}
Kind is a discriminator. Defined as a type for clarity and type checking.
```go
type Kind string
```
<a name="Kustomization"></a>
## type Kustomization {#Kustomization}
Kustomization represents a kustomization.yaml file for use with the [Kustomize](<#Kustomize>) [Transformer](<#Transformer>). Untyped to avoid tightly coupling holos to kubectl versions which was a problem for the Flux maintainers. Type checking is expected to happen in CUE against the kubectl version the user prefers.
```go
type Kustomization map[string]any
```
<a name="Kustomize"></a>
## type Kustomize {#Kustomize}
Kustomize represents a kustomization [Transformer](<#Transformer>).
```go
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
}
```
<a name="Metadata"></a>
## type Metadata {#Metadata}
Metadata represents data about the resource such as the Name.
```go
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
}
```
<a name="NameLabel"></a>
## type NameLabel {#NameLabel}
NameLabel is a unique identifier useful to convert a CUE struct to a list when the values have a Name field with a default value. NameLabel indicates the common use case of converting a struct to a list where the Name field of the value aligns with the outer struct field name.
For example:
```
Outer: [NAME=_]: Name: NAME
```
```go
type NameLabel string
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform represents a platform to manage. A Platform resource informs holos which components to build. The platform resource also acts as a container for the platform model form values provided by the PlatformService. The primary use case is to collect the cluster names, cluster types, platform model, and holos components to build into one resource.
```go
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha4\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
```
<a name="PlatformSpec"></a>
## type PlatformSpec {#PlatformSpec}
PlatformSpec represents the specification of a [Platform](<#Platform>). Think of a platform spec as a [Component](<#Component>) collection for multiple kubernetes clusters combined with the user\-specified Platform Model.
```go
type PlatformSpec struct {
// Components represents a list of holos components to manage.
Components []Component `json:"components"`
}
```
<a name="Repository"></a>
## type Repository {#Repository}
Repository represents a [Helm](<#Helm>) [Chart](<#Chart>) repository.
```go
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
```
<a name="Resource"></a>
## type Resource {#Resource}
Resource represents one kubernetes api object.
```go
type Resource map[string]any
```
<a name="Resources"></a>
## type Resources {#Resources}
Resources represents a kubernetes resources [Generator](<#Generator>) from CUE.
```go
type Resources map[Kind]map[InternalLabel]Resource
```
<a name="Tags"></a>
## type Tags {#Tags}
Tags represents standardized fields injected into the component [BuildPlan](<#BuildPlan>) from the [Platform](<#Platform>).
Note, tags should have a reasonable default value to easily use cue eval and cue export without needing to make a bunch of decisions about tag values.
Example:
```
import core "github.com/holos-run/holos/api/core/v1alpha4"
_Tags: core.#Tags & {
cluster: _ @tag(cluster, type=string)
environment: _ @tag(environment, type=string)
component: _ @tag(component, type=string)
name: _ @tag(name, type=string)
}
```
```go
type Tags struct {
// Name represents the BuildPlan metadata.name field injected from the Platform.
Name string `json:"name" cue:"string | *\"no-name\""`
// Cluster represents the cluster name injected from
Cluster string `json:"cluster" cue:"string | *\"no-cluster\""`
// Environment represents the build plan environment.
Environment string `json:"environment" cue:"string | *\"no-environment\""`
// Component represents the path of the component relative to the platform root.
Component string `json:"component" cue:"string | *\"no-component\""`
}
```
<a name="Transformer"></a>
## type Transformer {#Transformer}
Transformer transforms [Generator](<#Generator>) manifests within a [Artifact](<#Artifact>).
```go
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
}
```
<a name="Values"></a>
## type Values {#Values}
Values represents [Helm](<#Helm>) Chart values generated from CUE.
```go
type Values map[string]any
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -2,8 +2,12 @@ import DocCardList from '@theme/DocCardList';
# Guides
## Start with the [Quickstart] guide
## Technical Overview
Please see the [Technical Overview] to learn about Holos. If you're ready to
drive in and try Holos, please work through the following guides.
## Bank of Holos
The guides are organized as a progression. We'll use Holos to manage a
fictional bank's platform, the Bank of Holos in each of the guides. In doing so
we'll take the time to explain the foundational concepts of Holos.
@@ -22,3 +26,4 @@ deployed services safely and consistently.
[Quickstart]: /docs/quickstart/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Change a Service]: /docs/guides/change-a-service/
[Technical Overview]: /docs/technical-overview/

View File

@@ -4,8 +4,711 @@ slug: /guides/change-a-service
sidebar_position: 300
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Change a Service
:::warning
Under construction.
In this guide, we'll explore how Holos supports the frontend development team at [Bank of Holos] in reconfiguring an already deployed service. Along the way, we'll demonstrate how simple configuration changes are made safer with type checking, and how rendering the complete platform provides clear visibility into those changes.
This guide builds on the concepts covered in the [Quickstart] and [Deploy a Service] guides.
## What you'll need {#requirements}
Like our other guides, this guide is intended to be useful without needing to
run each command. If you'd like to apply the manifests to a real Cluster,
complete the [Local Cluster Guide](/docs/guides/local-cluster) before this
guide.
You'll need the following tools installed to run the commands in this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap Helm charts.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Holos
Components that render with Kustomize.
## Fork the Git Repository
If you haven't already done so, [fork the Bank of
Holos](https://github.com/holos-run/bank-of-holos/fork) then clone the
repository to your local machine.
<Tabs groupId="git-clone">
<TabItem value="command" label="Command">
```bash
# Change YourName
git clone https://github.com/YourName/bank-of-holos
cd bank-of-holos
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Cloning into 'bank-of-holos'...
remote: Enumerating objects: 1177, done.
remote: Counting objects: 100% (1177/1177), done.
remote: Compressing objects: 100% (558/558), done.
remote: Total 1177 (delta 394), reused 1084 (delta 303), pack-reused 0 (from 0)
Receiving objects: 100% (1177/1177), 2.89 MiB | 6.07 MiB/s, done.
Resolving deltas: 100% (394/394), done.
```
</TabItem>
</Tabs>
Run the rest of the commands in this guide from the root of the repository.
If you plan to apply the changes we make, you can delete and re-create your
local platform synced to the start of this guide.
```bash
./scripts/reset-cluster
./scripts/apply
```
## Rename the Bank
Let's imagine the bank recently re-branded from The Bank of Holos to The
Holistic Bank. The software development team responsible for the front end
website needs to update the branding accordingly.
Let's explore how Holos catches errors early, before they land in production,
then guides the team to the best place to make a change.
The bank front end web service is managed by the
`projects/bank-of-holos/frontend/components/bank-frontend/` component which
refers to the organization display name in `schema.gen.cue`.
<Tabs groupId="F5B546EB-566F-4B83-84C3-C55B40F55555">
<TabItem value="schema.gen.cue" label="schema.gen.cue">
```cue showLineNumbers
package holos
import api "github.com/holos-run/holos/api/author/v1alpha3"
// Define the default organization name
// highlight-next-line
#Organization: DisplayName: string | *"Bank of Holos"
#Organization: Name: string | *"bank-of-holos"
#Organization: api.#OrganizationStrict
#Platform: api.#Platform
#Fleets: api.#StandardFleets
_ComponentConfig: {
Resources: #Resources
ArgoConfig: #ArgoConfig
}
#Helm: api.#Helm & _ComponentConfig
#Kustomize: api.#Kustomize & _ComponentConfig
#Kubernetes: api.#Kubernetes & _ComponentConfig
#ArgoConfig: api.#ArgoConfig & {
ClusterName: _ClusterName
}
```
</TabItem>
<TabItem value="projects/bank-of-holos/frontend/components/bank-frontend/bank-frontend.cue" label="projects/bank-of-holos/frontend/components/bank-frontend/bank-frontend.cue">
```cue showLineNumbers
package holos
// Produce a kubernetes objects build plan.
(#Kubernetes & Objects).BuildPlan
let Objects = {
Name: "bank-frontend"
Namespace: #BankOfHolos.Frontend.Namespace
// Ensure resources go in the correct namespace
Resources: [_]: [_]: metadata: namespace: Namespace
// https://github.com/GoogleCloudPlatform/bank-of-anthos/blob/release/v0.6.5/kubernetes-manifests/frontend.yaml
Resources: {
Service: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
_ports: http: {
name: "http"
port: 80
targetPort: 8080
protocol: "TCP"
}
ports: [for x in _ports {x}]
}
}
Deployment: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: matchLabels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
template: {
metadata: labels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
securityContext: {
seccompProfile: type: "RuntimeDefault"
fsGroup: 1000
runAsGroup: 1000
runAsNonRoot: true
runAsUser: 1000
}
serviceAccountName: "bank-of-holos"
terminationGracePeriodSeconds: 5
containers: [{
env: [{
name: "BANK_NAME"
// highlight-next-line
value: #Organization.DisplayName
}, {
name: "ENV_PLATFORM"
value: "local"
}, {
name: "VERSION"
value: "v0.6.5"
}, {
name: "PORT"
value: "8080"
}, {
name: "ENABLE_TRACING"
value: "false"
}, {
name: "SCHEME"
value: "https"
}, {
name: "LOG_LEVEL"
value: "info"
}, {
name: "DEFAULT_USERNAME"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_USERNAME"
name: "demo-data-config"
}
}, {
name: "DEFAULT_PASSWORD"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_PASSWORD"
name: "demo-data-config"
}
}, {
name: "REGISTERED_OAUTH_CLIENT_ID"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_CLIENT_ID"
name: "oauth-config"
optional: true
}
}, {
name: "ALLOWED_OAUTH_REDIRECT_URI"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_REDIRECT_URI"
name: "oauth-config"
optional: true
}
}]
envFrom: [{
configMapRef: name: "environment-config"
}, {
configMapRef: name: "service-api-config"
}]
image: "us-central1-docker.pkg.dev/bank-of-anthos-ci/bank-of-anthos/frontend:v0.6.5@sha256:d72050f70d12383e4434ad04d189b681dc625f696087ddf0b5df641645c9dafa"
livenessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 60
periodSeconds: 15
timeoutSeconds: 30
}
name: "front"
readinessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 10
}
resources: {
limits: {
cpu: "250m"
memory: "128Mi"
}
requests: {
cpu: "100m"
memory: "64Mi"
}
}
securityContext: {
allowPrivilegeEscalation: false
capabilities: drop: ["all"]
privileged: false
readOnlyRootFilesystem: true
}
volumeMounts: [{
mountPath: "/tmp"
name: "tmp"
}, {
mountPath: "/tmp/.ssh"
name: "publickey"
readOnly: true
}]
}]
volumes: [
{
emptyDir: {}
name: "tmp"
},
{
name: "publickey"
secret: {
items: [{key: "jwtRS256.key.pub", path: "publickey"}]
secretName: "jwt-key"
}
},
]
}
}
}
}
// Allow HTTPRoutes in the ingress gateway namespace to reference Services
// in this namespace.
ReferenceGrant: grant: #ReferenceGrant & {
metadata: namespace: Namespace
}
// Include shared resources
#BankOfHolos.Resources
}
}
```
</TabItem>
</Tabs>
Line 6 of the `schema.gen.cue` file defines the _default_ value for
`#Organization.DisplayName` by using `string | *"..."`. In CUE, the `*`
asterisk character denotes a [default value].
Line 78 of the `bank-frontend.cue` file refers to `#Organization.DisplayName` to
configure the front end web container.
Let's change the name of the bank by defining a new value for
`#Organization.DisplayName` at the root of the configuration. Create
`projects/organization.cue` with the following content.
<Tabs groupId="B386181F-EBE7-469D-8CB5-37631067669B">
<TabItem value="projects/organization.cue" label="projects/organization.cue">
```cue showLineNumbers
package holos
#Organization: DisplayName: "The Holistic-Bank"
```
</TabItem>
</Tabs>
Let's render the platform and see if this changes the name.
<Tabs groupId="A014333C-3271-4C22-87E6-2B7BF898EA3E">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
#Organization.DisplayName: 2 errors in empty disjunction:
#Organization.DisplayName: conflicting values "Bank of Holos" and "The Holistic-Bank":
/bank-of-holos/projects/organization.cue:3:29
/bank-of-holos/schema.gen.cue:6:39
// highlight-next-line
#Organization.DisplayName: invalid value "The Holistic-Bank" (out of bound =~"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$"):
/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/author/v1alpha3/definitions_go_gen.cue:203:25
/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/author/v1alpha3/definitions_go_gen.cue:188:15
/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/author/v1alpha3/definitions_go_gen.cue:203:15
/bank-of-holos/projects/organization.cue:3:29
/bank-of-holos/schema.gen.cue:6:29
could not run: could not render component: exit status 1 at internal/render/platform.go:50
```
</TabItem>
</Tabs>
:::warning Whoops
The development team defined a value that isn't allowed by the
configuration.
:::
Someone else in the organization placed a [constraint] on the
configuration to ensure the display name contains only letters, numbers, and
spaces. This constraint is expressed as a [regular expression].
:::tip
CUE provides clear visibility where to start looking to resolve conflicts. Each
file and line number listed is a place the `#Organization.DisplayName` field is
defined.
:::
Let's try again, this time replacing the hyphen with a space.
<Tabs groupId="F93B34FA-C0C6-4793-A32F-DAD094403208">
<TabItem value="projects/organization.cue" label="projects/organization.cue">
```cue showLineNumbers
package holos
#Organization: DisplayName: "The Holistic Bank"
```
</TabItem>
</Tabs>
<Tabs groupId="5FD68778-476A-4F82-8817-71CEE205216E">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered bank-ledger-db for cluster workload in 139.863625ms
rendered bank-accounts-db for cluster workload in 151.74875ms
rendered bank-balance-reader for cluster workload in 154.356083ms
rendered bank-ledger-writer for cluster workload in 161.209541ms
rendered bank-userservice for cluster workload in 163.373417ms
rendered bank-backend-config for cluster workload in 179.271208ms
rendered bank-secrets for cluster workload in 204.35625ms
rendered gateway for cluster workload in 118.707583ms
rendered httproutes for cluster workload in 140.981541ms
rendered bank-transaction-history for cluster workload in 156.066875ms
rendered bank-frontend for cluster workload in 300.102292ms
rendered bank-contacts for cluster workload in 159.89625ms
rendered cni for cluster workload in 150.754458ms
rendered istiod for cluster workload in 222.922625ms
rendered app-projects for cluster workload in 118.422792ms
rendered ztunnel for cluster workload in 142.840625ms
rendered cert-manager for cluster workload in 190.938834ms
rendered base for cluster workload in 340.679416ms
rendered local-ca for cluster workload in 107.120334ms
rendered external-secrets for cluster workload in 145.020834ms
rendered argocd for cluster workload in 299.690917ms
rendered namespaces for cluster workload in 115.862334ms
rendered gateway-api for cluster workload in 225.783833ms
rendered external-secrets-crds for cluster workload in 339.741166ms
rendered crds for cluster workload in 421.849041ms
rendered platform in 718.015959ms
```
</TabItem>
</Tabs>
:::tip Success
Great, the platform rendered. We know the display name is valid according to
the constraints.
:::
Let's see if the new display name value updated the configuration for the bank
frontend.
<Tabs groupId="6C068651-2061-4262-BE1E-7BB3E7EB66CB">
<TabItem value="command" label="Command">
```bash
git status
```
</TabItem>
<TabItem value="output" label="Output">
```txt
On branch main
Your branch and 'jeffmccune/main' have diverged,
and have 2 and 4 different commits each, respectively.
(use "git pull" to merge the remote branch into yours)
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
// highlight-next-line
modified: deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
modified: deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
Untracked files:
(use "git add <file>..." to include in what will be committed)
projects/organization.cue
no changes added to commit (use "git add" and/or "git commit -a")
```
</TabItem>
</Tabs>
<Tabs groupId="4A20831E-461B-4EDE-8F6E-E73C3AEC12DB">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml b/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
index 7914756..250c660 100644
--- a/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
+++ b/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
@@ -9,7 +9,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
@@ -26,7 +26,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
@@ -43,7 +43,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
@@ -60,7 +60,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
diff --git a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
index dae6f93..d41516b 100644
--- a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
+++ b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
@@ -71,7 +71,7 @@ spec:
containers:
- env:
- name: BANK_NAME
- value: Bank of Holos
+ value: The Holistic Bank
- name: ENV_PLATFORM
value: local
- name: VERSION
```
</TabItem>
</Tabs>
:::danger
The new display name changed the frontend container, but it _also_ affected the
app-projects component owned by the platform team.
:::
Submitting a pull request would trigger a code review from the platform
engineering team who manages the app-projects component. Let's see how to
narrow the change down to limit the scope to the bank's user facing services.
All of these services are managed under `projects/bank-of-holos/` Move the
`organization.cue` file into this folder to limit the scope of configuration to
the the components contained within.
```bash
mv projects/organization.cue projects/bank-of-holos/
```
Render the platform and let's see what changed.
<Tabs groupId="0FFEC244-B59B-4136-9C82-837985DC2AB8">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered bank-ledger-db for cluster workload in 163.814917ms
rendered bank-accounts-db for cluster workload in 163.960208ms
rendered bank-userservice for cluster workload in 164.1625ms
rendered bank-ledger-writer for cluster workload in 169.185291ms
rendered bank-balance-reader for cluster workload in 174.5455ms
rendered bank-backend-config for cluster workload in 178.092125ms
rendered bank-secrets for cluster workload in 202.305334ms
rendered gateway for cluster workload in 122.81725ms
rendered httproutes for cluster workload in 134.121084ms
rendered bank-contacts for cluster workload in 146.4185ms
rendered bank-frontend for cluster workload in 311.35425ms
rendered bank-transaction-history for cluster workload in 160.103ms
rendered cni for cluster workload in 145.762083ms
rendered istiod for cluster workload in 216.0065ms
rendered app-projects for cluster workload in 117.684333ms
rendered ztunnel for cluster workload in 144.555292ms
rendered cert-manager for cluster workload in 178.247917ms
rendered base for cluster workload in 336.679ms
rendered external-secrets for cluster workload in 142.21825ms
rendered local-ca for cluster workload in 101.249ms
rendered argocd for cluster workload in 280.54525ms
rendered namespaces for cluster workload in 106.822042ms
rendered gateway-api for cluster workload in 200.459791ms
rendered external-secrets-crds for cluster workload in 470.125833ms
rendered crds for cluster workload in 844.388666ms
rendered platform in 1.154937084s
```
</TabItem>
</Tabs>
<Tabs groupId="DE4FEEE5-FC53-48A6-BC6F-D0EA1DBFD00C">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
index dae6f93..d41516b 100644
--- a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
+++ b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
@@ -71,7 +71,7 @@ spec:
containers:
- env:
- name: BANK_NAME
- value: Bank of Holos
+ value: The Holistic Bank
- name: ENV_PLATFORM
value: local
- name: VERSION
```
</TabItem>
</Tabs>
:::tip Success
Great! This time, the only manifest affected is our `bank-frontend.gen.yaml`.
:::
The `BANK_NAME` environment variable will change as we expect, and only the dev
teams managing the bank services components are affected by the change.
Let's commit and push this change and see if it works.
<Tabs groupId="435D9C60-F841-4CF1-A947-506422E6BAC9">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m 'frontend: rename bank to The Holistic Bank'
git push
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main fda74ec] frontend: rename bank to The Holistic Bank
2 files changed, 4 insertions(+), 1 deletion(-)
create mode 100644 projects/bank-of-holos/organization.cue
```
</TabItem>
</Tabs>
Now that we've pushed the change, let's apply the change to the platform.
## Apply the Change
Once we've pushed the change, navigate to the [bank-frontend GitOps
Application](https://argocd.holos.localhost/applications/argocd/bank-frontend?view=tree&resource=).
We can see the Deployment needs to sync to the desired state we just pushed.
![bank-frontend out of sync](./img/change-a-service-out-of-sync.png)
Clicking on the frontend Deployment, we see the diff with the change we expect.
![bank-frontend diff](./img/change-a-service-diff.png)
Sync the change, ArgoCD applies the desired configuration state to the cluster
and Kubernetes handles rolling out the updated Deployment resource.
![bank-frontend progressing](./img/change-a-service-progressing.png)
Soon, the deployment finishes and the component is in sync again.
![bank-frontend in sync](./img/change-a-service-in-sync.png)
Finally, let's see if the name actually changed on the website. Navigate to
https://bank.holos.localhost/.
![bank-frontend login page](./img/change-a-service-login-page.png)
:::tip Success
We successfully made our change and successfully applied the changed
configuration to the platform.
:::
Thanks for taking the time to work through this guide which covered:
- How multiple teams could be impacted by defining configuration at the
`projects/` path.
- How to scope our change to only affect components within the
`projects/bank-of-holos/` path, eliminating the impact on other teams.
- How CUE can [constrain] values in Holos, increasing safety.
- How to handle a [default value] in CUE.
- How CUE surfaces the file and line number of _every_ place to look for where a
value is defined, making it faster and easier to troubleshoot problems.
[Quickstart]: /docs/quickstart/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Change a Service]: /docs/guides/change-a-service/
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize
[ComponentFields]: /docs/api/author/v1alpha3/#ComponentFields
[platform-files]: /docs/quickstart/#how-platform-rendering-works
[AppProject]: https://argo-cd.readthedocs.io/en/stable/user-guide/projects/
[unification operator]: https://cuelang.org/docs/reference/spec/#unification
[code-owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[Kustomization API]: https://github.com/kubernetes-sigs/kustomize/blob/release-kustomize-v5.2/api/types/kustomization.go#L34
[cue import]: https://cuelang.org/docs/reference/command/cue-help-import/
[cue get go]: https://cuelang.org/docs/concept/how-cue-works-with-go/
[timoni-crds]: https://timoni.sh/cue/module/custom-resources/
[HTTPRoute]: https://gateway-api.sigs.k8s.io/api-types/httproute/?h=filter
[Ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
[hidden field]: https://cuelang.org/docs/tour/references/hidden/
[comprehension]: https://cuelang.org/docs/reference/spec/#comprehensions
[code owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[ReferenceGrant]: https://gateway-api.sigs.k8s.io/api-types/referencegrant/
[Local Cluster Guide]: /docs/guides/local-cluster
[Bank of Holos]: https://github.com/holos-run/bank-of-holos
[default value]: https://cuelang.org/docs/tour/types/defaults/
[constrain]: https://cuelang.org/docs/tour/basics/constraints/
[constraint]: https://cuelang.org/docs/tour/basics/constraints/
[regular expression]: https://cuelang.org/docs/tour/expressions/regexp/

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,15 @@
---
description: Helm Component
slug: /guides/helm-component
sidebar_position: 400
---
# Helm Component
The [Deploy a Service](/docs/guides/deploy-a-service/) guide is the best guide
we have on wrapping a Helm chart in a Holos Component. The [Helm] section of
the Author API may also be useful.
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize

Binary file not shown.

After

Width:  |  Height:  |  Size: 997 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1009 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 617 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 706 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 794 KiB

View File

@@ -0,0 +1,20 @@
---
description: Kubernetes Component
slug: /guides/kubernetes-component
sidebar_position: 500
---
# Kubernetes Component
:::warning
TODO
:::
This is a placeholder for a guide for managing Kubernetes resources directly
from a Holos Component with strong type checking.
In the meantime, please refer to the [Kubernetes] section of the Author API.
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize

View File

@@ -0,0 +1,20 @@
---
description: Wrap a Kustomize Kustomization in a Holos Component.
slug: /guides/kustomize-component
sidebar_position: 600
---
# Kustomize Component
:::warning
TODO
:::
This is a placeholder for a guide on wrapping a Kustomize Kustomization base
with a Holos component.
In the meantime, please refer to the [Kustomize] section of the Author API.
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize

View File

@@ -32,6 +32,10 @@ easier for multiple teams to deliver services on a platform. These teams are:
- **Security**
- **Quality Assurance**
In this guide we'll show how Holos enables teams to work autonomously
while still allowing the platform team to enforce the standards and policies
they care about to provide a secure and consistent software development platform.
Here's a screenshot of the retail banking application we'll build and deploy on
our platform. We'll keep each of these teams in mind as we work through the
guides. Each of our guides focuses on different aspects of delivering the Bank
@@ -109,15 +113,25 @@ Resolving deltas: 100% (394/394), done.
Run the rest of the commands in this guide from the root of the repository.
## Configure ArgoCD {#configure-argocd}
## Configuring GitOps {#configuring-gitops}
The Bank of Holos platform is organized as a collection of software components.
Each component represents a piece of software provided by an upstream vendor,
for example ArgoCD, or software developed in-house. Components are also used to
glue together, or integrate, other components into the platform.
We'll start by changing the platform to point to our fork. We need to do this
so we'll be able to see changes in ArgoCD as we make changes with GitOps.
The platform team provides ArgoCD as a means for teams to implement GitOps
within their software development workflow. Each team using the Bank of Holos
platform uses a Holos resource provided by the platform team to create their
ArgoCD Application definition. In doing so, the platform team has provided
a "golden path" for each team to independently make the changes they need
while still centrally enforcing the policies that provide a consistent and
safe experience.
Currently each team is using the upstream `bank-of-repo` repository as their
source of truth. We'll start by changing ArgoCD to point to our fork. This will
allow us to be able to see the results of our changes in ArgoCD using a GitOps
workflow.
<Tabs groupId="argocd-config">
<TabItem value="command" label="projects/argocd-config.cue">
@@ -172,32 +186,32 @@ holos render platform ./platform
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered bank-accounts-db for cluster workload in 142.661334ms
rendered bank-ledger-db for cluster workload in 144.041417ms
rendered bank-userservice for cluster workload in 157.828709ms
rendered bank-ledger-writer for cluster workload in 161.138292ms
rendered bank-backend-config for cluster workload in 168.923459ms
rendered bank-balance-reader for cluster workload in 171.877875ms
rendered bank-secrets for cluster workload in 207.958792ms
rendered gateway for cluster workload in 123.572583ms
rendered bank-contacts for cluster workload in 144.466291ms
rendered bank-transaction-history for cluster workload in 151.520041ms
rendered httproutes for cluster workload in 139.590834ms
rendered bank-frontend for cluster workload in 309.679834ms
rendered app-projects for cluster workload in 107.136083ms
rendered ztunnel for cluster workload in 160.679791ms
rendered cni for cluster workload in 238.937625ms
rendered cert-manager for cluster workload in 178.610834ms
rendered istiod for cluster workload in 340.953208ms
rendered argocd for cluster workload in 286.277ms
rendered local-ca for cluster workload in 98.720208ms
rendered external-secrets for cluster workload in 141.459708ms
rendered base for cluster workload in 454.356667ms
rendered namespaces for cluster workload in 115.401709ms
rendered gateway-api for cluster workload in 203.5625ms
rendered external-secrets-crds for cluster workload in 525.180209ms
rendered crds for cluster workload in 888.406167ms
rendered platform in 1.182857542s
rendered app-projects for cluster workload in 206.109166ms
rendered httproutes for cluster workload in 222.3885ms
rendered istio-cni for cluster workload in 236.061167ms
rendered istio-gateway for cluster workload in 238.973167ms
rendered istio-ztunnel for cluster workload in 240.000042ms
rendered istiod for cluster workload in 275.385709ms
rendered bank-accounts-db for cluster workload in 215.252708ms
rendered bank-backend-config for cluster workload in 245.383333ms
rendered bank-secrets for cluster workload in 273.477458ms
rendered bank-userservice for cluster workload in 234.866375ms
rendered istio-base for cluster workload in 541.965042ms
rendered argocd for cluster workload in 352.4625ms
rendered bank-frontend for cluster workload in 374.208084ms
rendered bank-ledger-db for cluster workload in 224.120125ms
rendered bank-ledger-writer for cluster workload in 237.136042ms
rendered bank-balance-reader for cluster workload in 233.804083ms
rendered bank-transaction-history for cluster workload in 236.114542ms
rendered bank-contacts for cluster workload in 222.469875ms
rendered argocd-crds for cluster workload in 812.078042ms
rendered gateway-api for cluster workload in 173.66425ms
rendered external-secrets for cluster workload in 231.326958ms
rendered local-ca for cluster workload in 183.12425ms
rendered namespaces for cluster workload in 173.256125ms
rendered cert-manager for cluster workload in 432.088959ms
rendered external-secrets-crds for cluster workload in 496.282459ms
rendered platform in 1.106735958s
```
</TabItem>
</Tabs>
@@ -220,31 +234,31 @@ Your branch is up to date with 'origin/main'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: deploy/clusters/workload/gitops/app-projects.application.gen.yaml
modified: deploy/clusters/workload/gitops/argocd-crds.application.gen.yaml
modified: deploy/clusters/workload/gitops/argocd.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-accounts-db.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-backend-config.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-balance-reader.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-contacts.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-ledger-db.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-ledger-writer.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-secrets.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-transaction-history.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-userservice.application.gen.yaml
modified: deploy/clusters/workload/gitops/cert-manager.application.gen.yaml
modified: deploy/clusters/workload/gitops/external-secrets-crds.application.gen.yaml
modified: deploy/clusters/workload/gitops/external-secrets.application.gen.yaml
modified: deploy/clusters/workload/gitops/gateway-api.application.gen.yaml
modified: deploy/clusters/workload/gitops/httproutes.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-base.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-cni.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-gateway.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-ztunnel.application.gen.yaml
modified: deploy/clusters/workload/gitops/istiod.application.gen.yaml
modified: deploy/clusters/workload/gitops/local-ca.application.gen.yaml
modified: deploy/clusters/workload/gitops/namespaces.application.gen.yaml
modified: deploy/clusters/workload/gitops/app-projects.gen.yaml
modified: deploy/clusters/workload/gitops/argocd-crds.gen.yaml
modified: deploy/clusters/workload/gitops/argocd.gen.yaml
modified: deploy/clusters/workload/gitops/bank-accounts-db.gen.yaml
modified: deploy/clusters/workload/gitops/bank-backend-config.gen.yaml
modified: deploy/clusters/workload/gitops/bank-balance-reader.gen.yaml
modified: deploy/clusters/workload/gitops/bank-contacts.gen.yaml
modified: deploy/clusters/workload/gitops/bank-frontend.gen.yaml
modified: deploy/clusters/workload/gitops/bank-ledger-db.gen.yaml
modified: deploy/clusters/workload/gitops/bank-ledger-writer.gen.yaml
modified: deploy/clusters/workload/gitops/bank-secrets.gen.yaml
modified: deploy/clusters/workload/gitops/bank-transaction-history.gen.yaml
modified: deploy/clusters/workload/gitops/bank-userservice.gen.yaml
modified: deploy/clusters/workload/gitops/cert-manager.gen.yaml
modified: deploy/clusters/workload/gitops/external-secrets-crds.gen.yaml
modified: deploy/clusters/workload/gitops/external-secrets.gen.yaml
modified: deploy/clusters/workload/gitops/gateway-api.gen.yaml
modified: deploy/clusters/workload/gitops/httproutes.gen.yaml
modified: deploy/clusters/workload/gitops/istio-base.gen.yaml
modified: deploy/clusters/workload/gitops/istio-cni.gen.yaml
modified: deploy/clusters/workload/gitops/istio-gateway.gen.yaml
modified: deploy/clusters/workload/gitops/istio-ztunnel.gen.yaml
modified: deploy/clusters/workload/gitops/istiod.gen.yaml
modified: deploy/clusters/workload/gitops/local-ca.gen.yaml
modified: deploy/clusters/workload/gitops/namespaces.gen.yaml
modified: projects/argocd-config.cue
no changes added to commit (use "git add" and/or "git commit -a")
@@ -258,19 +272,19 @@ the changed `spec.source.repoURL` field.
<Tabs groupId="665E5402-FB42-4975-B654-3922EE73EE07">
<TabItem value="command" label="Command">
```bash
git diff deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
git diff deploy/clusters/workload/gitops/bank-frontend.gen.yaml
```
</TabItem>
<TabItem value="output" label="Output">
```diff showLineNumbers
diff --git a/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml b/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
index d8ede55..aed4338 100644
--- a/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
+++ b/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
@@ -9,5 +9,5 @@ spec:
project: bank-frontend
diff --git a/deploy/clusters/workload/gitops/bank-frontend.gen.yaml b/deploy/clusters/workload/gitops/bank-frontend.gen.yaml
index 3a3dec0..22e21bb 100644
--- a/deploy/clusters/workload/gitops/bank-frontend.gen.yaml
+++ b/deploy/clusters/workload/gitops/bank-frontend.gen.yaml
@@ -11,5 +11,5 @@ spec:
project: default
source:
path: ./deploy/clusters/workload/components/bank-frontend
path: deploy/clusters/workload/components/bank-frontend
- repoURL: https://github.com/holos-run/bank-of-holos
+ repoURL: https://github.com/jeffmccune/bank-of-holos
targetRevision: main
@@ -312,6 +326,39 @@ So what happens when we run `holos render platform`? We saw `holos` write plain
manifest files, let's dive into how and why we implemented platform rendering
like this.
```mermaid
---
title: Figure 1 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/author/v1alpha3/#Platform">Platform</a>]
HC[<a href="/docs/api/author/v1alpha3/#ComponentFields">Components</a>]
BP[<a href="/docs/api/core/v1alpha3#BuildPlan">BuildPlan</a>]
H[<a href="/docs/api/author/v1alpha3/#Helm">Helm</a>]
K[<a href="/docs/api/author/v1alpha3/#Kustomize">Kustomize</a>]
O[<a href="/docs/api/author/v1alpha3/#Kubernetes">Kubernetes</a>]
P[<a href="/docs/api/core/v1alpha3#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
C[Kube API Server]
PS --> HC --> BP
BP --> H --> P
BP --> K --> P
BP --> O --> P
P --> Y --> FS
P --> G --> FS
FS --> ArgoCD --> C
FS --> Flux --> C
FS --> kubectl --> C
```
### Why do we render the platform? {#why-render-the-platform}
We built Holos to make the process of managing a platform safer, easier, and
@@ -335,6 +382,11 @@ made, affects the whole platform. Before we made Holos we were frustrated with
how difficult it was to get this zoomed-out, broad perspective of each change we
made.
:::tip
Holos implements the [rendered manifests pattern] so you don't have to build it
yourself.
:::
### How does platform rendering work? {#how-platform-rendering-works}
Holos is declarative. CUE provides resources that declare what `holos` needs to
@@ -354,20 +406,20 @@ files for now, they behave the same as these two.
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
// Manage the Component on every Cluster in the Platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
// highlight-next-line
#Platform: Components: "\(Cluster.name)/argocd-crds": {
path: "projects/platform/components/argocd/crds"
cluster: Cluster.name
}
// highlight-next-line
#Platform: Components: "\(Cluster.name)/argocd": {
path: "projects/platform/components/argocd/argocd"
cluster: Cluster.name
}
}
for Cluster in Fleet.clusters {
#Platform: Components: "\(Cluster.name):argocd-crds": {
name: "argocd-crds"
component: "projects/platform/components/argocd/crds"
cluster: Cluster.name
}
#Platform: Components: "\(Cluster.name):argocd": {
name: "argocd"
component: "projects/platform/components/argocd/argocd"
cluster: Cluster.name
}
}
}
```
</TabItem>
@@ -377,18 +429,18 @@ package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
// highlight-next-line
#Platform: Components: "\(Cluster.name)/external-secrets-crds": {
path: "projects/platform/components/external-secrets-crds"
cluster: Cluster.name
}
// highlight-next-line
#Platform: Components: "\(Cluster.name)/external-secrets": {
path: "projects/platform/components/external-secrets"
cluster: Cluster.name
}
}
for Cluster in Fleet.clusters {
#Platform: Components: "\(Cluster.name):external-secrets-crds": {
name: "external-secrets-crds"
component: "projects/platform/components/external-secrets-crds"
cluster: Cluster.name
}
#Platform: Components: "\(Cluster.name):external-secrets": {
name: "external-secrets"
component: "projects/platform/components/external-secrets"
cluster: Cluster.name
}
}
}
```
</TabItem>
@@ -404,13 +456,13 @@ services we deploy onto the platform.
manifests for a cluster.
:::important
On lines 6 and 10 we see a Component being assigned to the Platform. We also
On lines 6 and 11 we see a Component being assigned to the Platform. We also
start to dive into the syntax of CUE, which we need to understand a little
before going further.
:::
> In its simplest form, CUE looks a lot like JSON. This is because CUE is a
superset of JSON. Or, put differently: all valid JSON is CUE.
superset of JSON. Or, put differently: all valid JSON is CUE.[^1]
>
> 1. C-style comments are allowed
> 2. field names without special characters don't need to be quoted
@@ -423,75 +475,7 @@ members are called fields, which link their name, or label, to a value.
There are two important things to know about CUE to understand these two files.
First, the curly braces have been omitted which is item 5 on the list above.
Second, CUE is all about _unification_. These files could have been written
like this:
<Tabs groupId="59FFCCB6-A584-42ED-AC37-1C2BDCF5A523">
<TabItem value="argocd" label="platform/argocd.cue">
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
#Platform: {
// highlight-next-line
// Define #Platform.Components
// highlight-next-line
Components: {
"\(Cluster.name)/argocd-crds": {
path: "projects/platform/components/argocd/crds"
cluster: Cluster.name
}
}
// highlight-next-line
// Define #Platform.Components again!? Error?
// highlight-next-line
Components: {
"\(Cluster.name)/argocd": {
path: "projects/platform/components/argocd/argocd"
cluster: Cluster.name
}
}
}
}
}
```
</TabItem>
<TabItem value="external-secrets" label="platform/external-secrets.cue">
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
#Platform: {
// highlight-next-line
// Define #Platform.Components
// highlight-next-line
Components: {
"\(Cluster.name)/external-secrets-crds": {
path: "projects/platform/components/external-secrets-crds"
cluster: Cluster.name
}
}
// highlight-next-line
// Define #Platform.Components again!? Error?
// highlight-next-line
Components: {
"\(Cluster.name)/external-secrets": {
path: "projects/platform/components/external-secrets"
cluster: Cluster.name
}
}
}
}
}
```
</TabItem>
</Tabs>
Second, CUE is about _unification_.
:::important
Unlike most other languages, it is common to declare the same field in multiple
@@ -515,13 +499,20 @@ It looks like this.
```cue showLineNumbers
package holos
#Platform.Output
import api "github.com/holos-run/holos/api/author/v1alpha4"
#Platform: api.#Platform & {
Name: "guide"
}
// Render a Platform resource for holos to process
#Platform.Resource
```
</TabItem>
</Tabs>
This file provides the value of the `#Platform.Output` field, the platform spec,
to `holos`.
This file provides the value of the `#Platform.Resource` field, the platform
spec, to `holos`.
Let's take a look at that Output value:
@@ -534,24 +525,31 @@ cue export --out yaml ./platform
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: Platform
apiVersion: v1alpha3
apiVersion: v1alpha4
metadata:
name: guide
spec:
model: {}
components: # This is a trimmed list for readability.
- path: projects/bank-of-holos/security/components/bank-secrets
- name: bank-secrets
component: projects/bank-of-holos/security/components/bank-secrets
cluster: workload
- path: projects/bank-of-holos/frontend/components/bank-frontend
model: {}
- name: bank-frontend
component: projects/bank-of-holos/frontend/components/bank-frontend
cluster: workload
- path: projects/platform/components/argocd/crds
model: {}
- name: bank-backend-config
component: projects/bank-of-holos/backend/components/bank-backend-config
cluster: workload
- path: projects/platform/components/argocd/argocd
model: {}
- name: bank-accounts-db
component: projects/bank-of-holos/backend/components/bank-accounts-db
cluster: workload
- path: projects/platform/components/external-secrets-crds
cluster: workload
- path: projects/platform/components/external-secrets
model: {}
- name: bank-userservice
component: projects/bank-of-holos/backend/components/bank-userservice
cluster: workload
model: {}
```
</TabItem>
</Tabs>
@@ -572,7 +570,7 @@ defining characteristic of CUE that makes it a unique, powerful, and _safe_
configuration language.
:::
Holos takes this list of components and builds each one by executing:
Holos takes this list of components and builds each one by executing the following command.
```bash
holos render component --cluster-name="example" "path/to/the/component"
@@ -591,7 +589,7 @@ component is called a BuildPlan. A BuildPlan is a list of zero or more
kubernetes resources, Helm charts, Kustomize bases, and additional files to
write into the `deploy/` directory.
Let's take a look at the cert-manager component. Notice the
Now let's look at the cert-manager component. Notice the
`platform/cert-manager.cue` file has the field `path:
"projects/platform/components/cert-manager"`. This path indicates where to
start working with the cert-manager component.
@@ -602,26 +600,24 @@ start working with the cert-manager component.
package holos
// Produce a helm chart build plan.
// highlight-next-line
(#Helm & Chart).BuildPlan
// highlight-next-line
let Chart = {
Name: "cert-manager"
// #CertManager is defined in projects/cert-manager.cue
// highlight-next-line
Version: #CertManager.Version
// highlight-next-line
Namespace: #CertManager.Namespace
Name: "cert-manager"
Namespace: #CertManager.Namespace
Repo: name: "jetstack"
Repo: url: "https://charts.jetstack.io"
Chart: {
version: #CertManager.Version
repository: {
name: "jetstack"
url: "https://charts.jetstack.io"
}
}
// CUE offers type checking and validation of Helm values.
// highlight-next-line
Values: installCRDs: true
// highlight-next-line
Values: startupapicheck: enabled: false
Values: {
installCRDs: true
startupapicheck: enabled: false
}
}
```
</TabItem>
@@ -631,14 +627,11 @@ package holos
// Platform wide configuration
#CertManager: {
// highlight-next-line
Version: "1.15.3"
// highlight-next-line
Namespace: "cert-manager"
Version: "1.15.3"
Namespace: "cert-manager"
}
// Register the namespace
// The underscore indicates the value is defined elsewhere in CUE.
#Namespaces: (#CertManager.Namespace): _
```
</TabItem>
@@ -657,7 +650,8 @@ root, `projects/cert-manager.cue`
5. We define Helm values in CUE to take advantage of strong type checking and
manage multiple Helm charts consistently with platform wide values.
Let's take a look at the BuildPlan this CUE configuration defines.
Let's take a look at the BuildPlan that results from the CUE configuration
described above.
<Tabs groupId="B54D5791-4E5B-4148-A368-62D9BE80760C">
<TabItem value="command" label="Command">
@@ -668,52 +662,88 @@ cue export --out yaml ./projects/platform/components/cert-manager
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha3
apiVersion: v1alpha4
metadata:
name: cert-manager
spec:
components:
resources:
gitops/cert-manager:
kind: KubernetesObjects
apiVersion: v1alpha3
metadata:
name: gitops/cert-manager
namespace: cert-manager
deployFiles:
clusters/no-name/gitops/cert-manager.application.gen.yaml: |
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
component: no-component
artifacts:
- artifact: clusters/no-cluster/components/cert-manager/cert-manager.gen.yaml
generators:
- kind: Helm
output: helm.gen.yaml
helm:
chart:
name: cert-manager
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: platform
source:
path: ./deploy/clusters/no-name/components/cert-manager
repoURL: https://github.com/jeffmccune/bank-of-holos
targetRevision: main
skip: false
helmChartList:
- kind: HelmChart
apiVersion: v1alpha3
chart:
name: cert-manager
version: 1.15.3
release: cert-manager
repository:
name: jetstack
url: https://charts.jetstack.io
valuesContent: |
installCRDs: true
startupapicheck:
enabled: false
enableHooks: false
metadata:
name: cert-manager
namespace: cert-manager
apiObjectMap: {}
skip: false
version: 1.15.3
release: cert-manager
repository:
name: jetstack
url: https://charts.jetstack.io
values:
installCRDs: true
startupapicheck:
enabled: false
enableHooks: false
namespace: cert-manager
- kind: Resources
output: resources.gen.yaml
resources: {}
transformers:
- kind: Kustomize
inputs:
- helm.gen.yaml
- resources.gen.yaml
output: combined.gen.yaml
kustomize:
kustomization:
resources:
- helm.gen.yaml
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
- kind: Kustomize
inputs:
- combined.gen.yaml
output: clusters/no-cluster/components/cert-manager/cert-manager.gen.yaml
kustomize:
kustomization:
namespace: cert-manager
commonLabels:
holos.run/component.name: cert-manager
resources:
- combined.gen.yaml
- artifact: clusters/no-cluster/gitops/cert-manager.gen.yaml
generators:
- kind: Resources
output: application.gen.yaml
resources:
Application:
cert-manager:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cert-manager
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: deploy/clusters/no-cluster/components/cert-manager
repoURL: https://github.com/jeffmccune/bank-of-holos
targetRevision: main
transformers:
- kind: Kustomize
inputs:
- application.gen.yaml
output: clusters/no-cluster/gitops/cert-manager.gen.yaml
kustomize:
kustomization:
commonLabels:
holos.run/component.name: cert-manager
resources:
- application.gen.yaml
```
</TabItem>
</Tabs>
@@ -732,8 +762,8 @@ The BuildPlan API is flexible enough to write any file into the `deploy/`
directory. Holos uses this flexibility to support both Flux and ArgoCD.
:::
When we run `cue export`, we get back a Core API BuildPlan. The BuildPlan is
produced by the `#Helm` definition on line 4 which is part of the Author API.
When we run `cue export`, we get back a [Core API] BuildPlan. The BuildPlan is
produced by the `#Helm` definition on line 4 which is part of the [Author API].
The Core API is the contract between CUE and `holos`. As such, it's not as
friendly as the Author API. The Author API is the contract component authors
and platform engineers use to configure and manage the platform. The Author API
@@ -752,6 +782,16 @@ directly from CUE. We can think of the Core API as an escape hatch for the
Author API. We'll see some examples of this in action in the more advanced
guides.
## Review
Let's review the concepts we've covered in this guide:
- A Holos platform is comprised of the CUE files that define the platform specification within the `platform` directory.
- The files in the `platform` directory each model an individual Holos component, and provide the path to the directory where the component's CUE configuration resides.
- A Holos platform must be rendered to generate Kubernetes manifest files.
- Holos resources enable teams to work autonomously while still allowing for centralized enforcement of company policies.
- Changes to one component can impact other components, and we can use `holos render platform` with `git diff` to assess the impact.
## Next Steps
Thank you for finishing the Quickstart guide. Dive deeper with the next guide
@@ -759,7 +799,10 @@ on how to [Deploy a Service] which explains how to take one of your existing
Helm charts or Deployments and manage it with Holos.
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[schema]: /docs/api/schema/v1alpha3/
[core]: /docs/api/core/v1alpha3/
[core]: /docs/api/core/v1alpha4/
[Core API]: /docs/api/core/
[Author API]: /docs/api/author/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Manage a Project]: /docs/guides/manage-a-project/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern/
[^1]: [The Basics of CUE](https://cuelang.org/docs/tour/basics/json-superset/)

View File

@@ -1,64 +0,0 @@
# Introduction
⚡️ Holos will help you build your **software development platform in no time.**
💸 Building a software development platform is **time consuming and expensive**. Spend more time building features for your customers and less time managing your development platform.
💥 Already have a platform? Add new features and services to your platform easily with Holos.
🧐 Holos is a platform builder. It builds a hollistic software development platform composed of best-of-breed cloud native open source projects. Holos is also a tool to make it easier to manage cloud infrastructure by providing a typed alternative to yaml templates.
## Features
Holos was built to solve two main problems:
1. Building a platform usually takes 3 engineers 6-9 months of effort. Holos provides a reference platform that enables you to deploy and customize your platform in a fraction of the time.
2. Configuration changes often cause outages. Existing tools like Helm make it difficult to understand the impact a configuration change will have. Holos provides a unique, unified configuration model powered by CUE that makes it safer and easier to roll out configuration changes.
A core principle of Holos is that organizations gain value from owning the the platform they build on. Avoid vendor lock-in, future price hikes, and expensive licensing changes by building on a solid foundation of open source, cloud native computing foundation backed projects.
The following features are built into the Holos reference platform.
:::tip
Don't see your preferred technology in the stack? Holos is designed to enable you to swap out components of the platform tech stack.
:::
- **Continuous Delivery**
- Holos builds a GitOps workflow for each application running in the platform.
- Developers push changes which are automatically deployed.
- Powered by [ArgoCD](https://argo-cd.readthedocs.io/en/stable/)
- **Identity and Access Management** (IAM)
- Holos builds a standard OIDC identity provider for you.
- Integrates with your exisitng IAM and SSO system, or works independently.
- Powerful customer identity and access management features.
- Role based access control.
- Powered by [ZITADEL](https://zitadel.com/)
- **Zero Trust**
- Authenticate and Authorize users at the platform layer instead of or in addition to the application layer.
- Integrated with observability to measure and alert about problems before customers complain.
- Powered by [Istio](https://istio.io/)
- **Observability**
- Holos collects performance and availability metrics automatically, without requiring application changes.
- Optional, deeper integration into the application layer.
- Distributed Tracing
- Logging
- Powered by Prometheus, Grafana, Loki, and OpenTelemetry.
- **Data Platform**
- Integrated management of PostgreSQL
- Automatic backups
- Automatic restore from backup
- Quickly fail over across multiple regions
- **Multi-Region**
- Holos is designed to operate in multiple regions and multiple clouds.
- Keep customer data in the region that makes the most sense for your business.
- Easily cut over from one region to another for redundancy and business continuity.
## Development Status
Holos is being actively developed by [Open Infrastructure Services](https://openinfrastructure.co). Release can be found [here](https://github.com/holos-run/holos/releases).
## Adoption
Organizations who have officially adopted Holos can be found [here](https://github.com/holos-run/holos/blob/main/ADOPTERS.md).

View File

@@ -6,15 +6,48 @@ slug: /
# Introduction
Welcome to Holos. Holos is an open source tool to manage software development
platforms safely, easily, and consistently.
platforms safely, easily, and consistently. We built Holos to help engineering
teams work more efficiently together by empowering them to build golden paths
and paved roads for other teams to leverage for quicker delivery.
Our documentation is organized into [Guides] and the [API Reference].
## Documentation
:::tip
Please start with the [Quickstart] guide to learn who Holos is for, what
problems it solves, and if it can solve problems you have.
:::
- [Guides] are organized into example use-cases of how Holos helps engineering
teams at the fictional Bank of Holos deliver business value on the bank's
platform.
- The [API Reference] is a technical reference for when you're writing CUE code to define your platform.
## Backstory
At [Open Infrastructure Services], we've each helped dozens of companies build and operate their software development platforms. During the U.S. presidential election just before the pandemic, our second-largest client, Twitter, experienced a global outage that lasted nearly a full day. We were managing their production configuration system, allowing the core infrastructure team to focus on business-critical objectives. This gave us a front-row seat to the incident.
A close friend and engineer on the team made a trivial one-line change to the firewall configuration. Less than 30 minutes later, everything was down. That change, which passed code review, caused the host firewall to revert to its default state on hundreds of thousands of servers, blocking all connections globally—except for SSH, thankfully. Even a Presidential candidate complained loudly.
This incident forced us to reconsider key issues with Twitter's platform:
1. **Lack of Visibility** - Engineers couldn't foresee the impact of even a small change, making it difficult to assess risks.
2. **Large Blast Radius** - Small changes affected the entire global fleet in under 30 minutes. There was no way to limit the impact of a single change.
3. **Incomplete Tooling** - The right processes were in place, but the tooling didn't fully support them. The change was tested and reviewed, but critical information wasn't surfaced in time.
Over the next few years, we built features to address these issues. Meanwhile, I began exploring how these solutions could work in the Kubernetes and cloud-native space.
As Google Cloud partners, we worked with large customers to understand how they built their platforms on Kubernetes. During the pandemic, we built a platform using CNCF projects like ArgoCD, Prometheus Stack, Istio, Cert Manager, and External Secrets Operator, integrating them into a cohesive platform. We started with upstream recommendations—primarily Helm charts—and wrote scripts to integrate each piece into the platform. For example, we passed Helm outputs to Kustomize to add labels or fix bugs, and wrote umbrella charts to add Ingress, HTTPRoute, and ExternalSecret resources.
These scripts served as necessary glue to hold everything together but became difficult to manage across multiple environments, regions, and cloud providers. YAML templates and nested loops created friction, making them hard to troubleshoot. The scripts themselves made it difficult to see what was happening and to fix issues affecting the entire platform.
Still, the scripts had a key advantage: they produced fully rendered manifests in plain text, committed to version control, and applied via ArgoCD. This clarity made troubleshooting easier and reduced errors in production.
Despite the makeshift nature of the scripts, I kept thinking about the "[Why are we templating YAML]?" post on Hacker News. I wanted to replace our scripts and charts with something more robust and easier to maintain—something that addressed Twitter's issues head-on.
I rewrote our scripts and charts using CUE and Go, replacing the glue layer. The result is **Holos**—a tool designed to complement Helm, Kustomize, and Jsonnet, making it easier and safer to define golden paths and paved roads without bespoke scripts or templates.
Thanks for reading. Take Holos for a spin on your local machine with our [Quickstart] guide.
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[CUE]: https://cuelang.org/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story

View File

@@ -324,28 +324,28 @@ ArgoCD, or Flux.
```mermaid
---
title: Figure 2 - Render Pipeline
title: Figure 1 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/core/v1alpha2#PlatformSpec">PlatformSpec</a>]
BP[<a href="/docs/api/core/v1alpha2#BuildPlan">BuildPlan</a>]
HC[<a href="/docs/api/core/v1alpha2#HolosComponent">Components</a>]
PS[<a href="/docs/api/author/v1alpha3/#Platform">Platform</a>]
HC[<a href="/docs/api/author/v1alpha3/#ComponentFields">Components</a>]
BP[<a href="/docs/api/core/v1alpha3#BuildPlan">BuildPlan</a>]
H[<a href="/docs/api/core/v1alpha2#HelmChart">HelmChart</a>]
K[<a href="/docs/api/core/v1alpha2#KustomizeBuild">KustomizeBuild</a>]
O[<a href="/docs/api/core/v1alpha2#KubernetesObjects">KubernetesObjects</a>]
H[<a href="/docs/api/author/v1alpha3/#Helm">Helm</a>]
K[<a href="/docs/api/author/v1alpha3/#Kustomize">Kustomize</a>]
O[<a href="/docs/api/author/v1alpha3/#Kubernetes">Kubernetes</a>]
P[<a href="/docs/api/core/v1alpha2#Kustomize">Kustomize</a>]
P[<a href="/docs/api/core/v1alpha3#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
C[Kube API Server]
PS --> BP --> HC
HC --> H --> P
HC --> K --> P
HC --> O --> P
PS --> HC --> BP
BP --> H --> P
BP --> K --> P
BP --> O --> P
P --> Y --> FS
P --> G --> FS

23
doc/md/start/support.md Normal file
View File

@@ -0,0 +1,23 @@
---
description: Get Support for Holos
slug: /support
sidebar_position: 900
---
# Support
## Community Support
You can ask questions in our community forums in [GitHub Discussions](https://github.com/holos-run/holos/discussions) or [Google Groups](https://groups.google.com/g/holos-discuss).
## Commercial Support and Services
### Open Infrastructure Services
[Open Infrastructure Services] are the primary stewards of Holos. Contact Open
Infrastructure Services for training, support, and services related to Holos,
platform engineering, and cloud infrastructure automation.
Please email holos-support@openinfrastructure.co for more information.
[Open Infrastructure Services]: https://openinfrastructure.co/

View File

@@ -0,0 +1,683 @@
---
slug: technical-overview
title: Technical Overview
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
## Overview
Holos makes it easier for platform teams to integrate software into their
platform. Existing tools in the Kubernetes ecosystem are narrowly focused on
application management. Holos takes a holistic approach, focusing on the broad
integration layer where applications are joined into the platform. Holos
improves cross team collaboration through well defined, typed structures at the
integration layer. These definitions provide golden paths for other teams to
easily integrate their own services into the platform.
<!-- truncate -->
## The Problem
Platform teams need to develop and maintain significant glue code to integrate
Helm charts and YAML manifests into a platform built on Kubernetes. This glue
code is often implemented with home grown umbrella charts and scripts.
Maintaining these charts and scripts takes time and effort that could otherwise
be spent improving the platform. The need for each organization to develop and
maintain this glue code indicates a gap in the Kubernetes ecosystem. Holos is a
Go command line tool leveraging [CUE] to fill this gap.
## Key Features
1. Holos enables teams to provide simple definitions for other teams to use as golden paths.
2. Define integrations in [CUE] with strong type checking. No more text templates or bash scripts.
3. Simplify complex integration. Order does not matter. Validation is early and quick.
4. Reuse your existing Helm charts and Kustomize bases.
5. Implement the [rendered manifests pattern]. Changes are clearly visible platform-wide.
6. Fully render manifests to plain files. Use your existing GitOps tools and processes.
7. Post-process with Kustomize from CUE instead of plain text files. Customize your Kustomizations.
8. Mix in resources to Helm charts and Kustomize bases, for example ExternalSecrets.
9. Render all of Helm, Kustomize, CUE, JSON, and YAML consistently with the same process.
## Rendering Pipeline
```mermaid
---
title: Figure 1 - v1alpha4 Render Pipeline
---
graph LR
Platform[<a href="/docs/api/author/v1alpha4/#Platform">Platform</a>]
Component[<a href="/docs/api/author/v1alpha4/#ComponentConfig">Components</a>]
Helm[<a href="/docs/api/author/v1alpha4/#Helm">Helm</a>]
Kustomize[<a href="/docs/api/author/v1alpha4/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/api/author/v1alpha4/#Kubernetes">Kubernetes</a>]
BuildPlan[<a href="/docs/api/core/v1alpha4/#buildplan">BuildPlan</a>]
ResourcesArtifact[<a href="/docs/api/core/v1alpha4/#artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/api/core/v1alpha4/#artifact">GitOps<br/>Artifact</a>]
Generator[<a href="/docs/api/core/v1alpha4/#generators">Generator</a>]
Transformer[<a href="/docs/api/core/v1alpha4/#transformer">Transformer</a>]
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
BuildPlan --> ResourcesArtifact --> Generator
BuildPlan --> GitOpsArtifact --> Generator
Generator --> Transformer --> Files --> KubeAPI
```
## Use Case
One of the development teams at the fictional Bank of Holos wants to deploy a
simple web app for an experimental project they're working on.
The platform team at the bank wants to build a simple golden path for teams to
provision projects consistently and easily in compliance with the bank's
policies.
### Platform Team
The platform team builds a golden path for development teams to register their
project with the platform. In compliance with bank policy, the platform team
needs to manage important security resources for each new project. All of these
resources can be derived from only 3 pieces of information.
1. The name of the project the dev team is working on.
2. The name of the team who currently owns the project.
3. The services, if any, the project is exposing.
The platform team defines a structure for the dev team to register this
information. This structure provides the golden path for the dev team.
The development team registers their experimental project, creatively named
"experiment" by submitting a pull request that contains this information.
<Tabs groupId="EB9C9AF1-F1AA-4189-B746-A5B8E3043F87">
<TabItem value="projects/experiment.cue" label="projects/experiment.cue">
```cue showLineNumbers
package holos
// The development team registers a project name.
#Projects: experiment: {
// The project owner must be named.
Owner: Name: "dev-team"
// Expose Service podinfo at https://podinfo.example.com
Hostnames: podinfo: Port: 9898
}
```
</TabItem>
</Tabs>
The platform team uses these three pieces of information to derive all of the
platform resources necessary to support the development team.
1. **Namespace** for the project resources.
2. **RoleBinding** to grant the dev team access to the project namespace.
3. **SecretStore** which implements the secret management policy for the bank.
4. **ReferenceGrant** to expose the project services through the Gateway API.
5. **HTTPRoutes** to expose the project services, if any.
6. **AppProject** to deploy and manage the project Applications with ArgoCD.
7. **Common Labels** to ensure every resource is labeled for resource accounting.
Rendering the platform generates fully rendered manifests for all of these
resources. These manifests are derived from the three pieces of information the
dev team provided.
Note the platform team must manage these resources across multiple namespaces.
The first four reside in the project namespace owned by the dev team. The
HTTPRoute and AppProject go into two namespaces managed by the platform team.
Holos makes it easier for the platform team to organize these resources into
different components with different owners.
:::important
Holos supports [CODEOWNERS] by clearly defining the teams responsible for each
platform component.
:::
<Tabs groupId="2E46EA1C-B118-44BF-AE20-752E8D1CE131">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered httproutes for cluster overview in 177.823625ms
rendered app-projects for cluster overview in 180.946834ms
rendered projects for cluster overview in 181.98725ms
rendered namespaces for cluster overview in 182.30725ms
rendered platform in 182.31075ms
```
:::tip
If you'd like to try this for yourself, `cd` into [examples/tech-overview] and
render the platform.
:::
</TabItem>
</Tabs>
The fully rendered manifests are written into the `deploy/` directory organized
by cluster and component for GitOps.
<Tabs groupId="07FBE14E-E9EA-437B-9FA1-C6D8806524AD">
<TabItem value="deploy/clusters/overview/components/namespaces/namespaces.gen.yaml" label="namespaces">
```
cat deploy/clusters/overview/components/namespaces/namespaces.gen.yaml
```
```yaml showLineNumbers
apiVersion: v1
kind: Namespace
metadata:
labels:
argocd.argoproj.io/instance: namespaces
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: namespaces
kubernetes.io/metadata.name: experiment
name: experiment
```
</TabItem>
<TabItem value="deploy/clusters/overview/components/projects/projects.gen.yaml" label="projects">
```
cat deploy/clusters/overview/components/projects/projects.gen.yaml
```
```yaml showLineNumbers
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
argocd.argoproj.io/instance: projects
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: projects
name: admin
namespace: experiment
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: oidc:sg-dev-team@example.com
---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
labels:
argocd.argoproj.io/instance: projects
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: projects
name: default
namespace: experiment
spec:
provider:
kubernetes:
auth:
token:
bearerToken:
key: token
name: eso-reader
remoteNamespace: experiment
server:
caBundle: LS0tLS1CRUd...QVRFLS0tLS0K
url: https://management.example.com:6443
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
labels:
argocd.argoproj.io/instance: projects
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: projects
name: istio-ingress
namespace: experiment
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: istio-ingress
to:
- group: ""
kind: Service
```
</TabItem>
<TabItem value="deploy/clusters/overview/components/httproutes/httproutes.gen.yaml" label="httproutes">
```
cat deploy/clusters/overview/components/httproutes/httproutes.gen.yaml
```
```yaml showLineNumbers
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
labels:
argocd.argoproj.io/instance: httproutes
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: httproutes
name: podinfo.example.com
namespace: istio-ingress
spec:
hostnames:
- podinfo.example.com
parentRefs:
- name: default
namespace: istio-ingress
rules:
- backendRefs:
- name: podinfo
namespace: experiment
port: 9898
matches:
- path:
type: PathPrefix
value: /
```
</TabItem>
</Tabs>
The rendered manifests are derived from the project registration information by
definitions implemented by the platform team. The [Author API] provides a
[Project] schema, but does not define an implementation. The platform team
implements the [Project] schema by writing a `#Projects` definition to manage
resources according to bank policies.
:::important
The Author API is intended as a convenient, ergonomic reference for component
authors. Definitions **are not** confined to the Author API.
:::
The following example shows how the platform team wrote the `#Projects`
definition to derive the Namespace from the project registration provided by the
dev team.
<Tabs groupId="5732727B-295E-46E1-B851-F8A1C5D7DF88">
<TabItem value="projects/platform/components/namespaces/namespaces.cue" label="Namespaces Component">
```txt
projects/platform/components/namespaces/namespaces.cue
```
```cue showLineNumbers
package holos
_Kubernetes: #Kubernetes & {
Name: "namespaces"
Resources: Namespace: #Namespaces
}
// Produce a kubernetes objects build plan.
_Kubernetes.BuildPlan
```
1. This is the namespaces component which manages a collection of Namespace resources derived from the project registration data shown in the second tab.
2. Line 5 manages a Namespace for each value of the `#Namespaces` struct. See the second tab for how the platform team defines this structure.
</TabItem>
<TabItem value="projects/projects.cue" label="#Projects Definition">
```txt
projects/projects.cue
```
```cue showLineNumbers
package holos
import api "github.com/holos-run/holos/api/author/v1alpha4"
// Projects defines the structure other teams register with to manage project
// resources. The platform team defines the schema, development teams provide
// the values.
#Projects: api.#Projects & {
[NAME=string]: {
Name: NAME
// The platform team requires the development teams to indicate an owner of
// the project.
Owner: Name: string
// The default value for the owner email address is derived from the owner
// name, but development teams can provide a different email address if
// needed.
Owner: Email: string | *"sg-\(Owner.Name)@\(#Organization.Domain)"
// The platform team constrains the project to a single namespace.
Namespaces: close({(NAME): Name: NAME})
// The platform team constrains the exposed services to the project
// namespace.
Hostnames: [HOST=string]: {
Name: HOST
Namespace: Namespaces[NAME].Name
Service: HOST
Port: number | *80
}
CommonLabels: {
"\(#Organization.Domain)/project.name": Name
"\(#Organization.Domain)/owner.name": Owner.Name
"\(#Organization.Domain)/owner.email": Owner.Email
}
}
}
for Project in #Projects {
// Register project namespaces with the namespaces component.
#Namespaces: {
for Namespace in Project.Namespaces {
(Namespace.Name): metadata: labels: Project.CommonLabels
}
}
}
```
1. On lines 8-35 the platform team derives most fields from the project name (line 9), and the owner name (line 13). The purpose is to fill in the remaining fields defined by the Author API.
2. Line 13 The dev team is expected to provide a concrete owner name, indicated by the `string` value.
3. Line 17 The platform team provides a default value for the email address. The project team may define a different value.
4. Line 19 The Author API allows a project to have many namespaces. The platform team constrains this down to one namespace per project by closing the struct. The namespace name must be the same as the project name.
5. Lines 22-27 The platform team derives values for a Gateway API [BackendObjectReference] from the hostname provided by the project team. These values are used later to build HTTPRoutes to expose their service.
6. Lines 30-32 Common labels are derived to mix into resources associated with this project.
7. Lines 37-44 The platform team adds a namespace with common labels for each project to the struct we saw in the first tab.
</TabItem>
</Tabs>
The RoleBinding, SecretScore, and ReferenceGrant are managed in the
[projects](https://github.com/holos-run/bank-of-holos/blob/v0.2.0/examples/tech-overview/projects/platform/components/projects/projects.cue)
component, similar to the previous namespaces example.
The HTTPRoute is managed separately in the
[httproutes](https://github.com/holos-run/bank-of-holos/blob/v0.2.0/examples/tech-overview/projects/platform/components/httproutes/httproutes.cue)
component.
All components are registered with the platform in the
[platform](https://github.com/holos-run/bank-of-holos/tree/v0.2.0/examples/tech-overview/platform)
directory.
:::important
Multiple components, potentially owned by different teams, derive fully rendered
resources from the same three project values. The dev team added these three
values to the `#Projects` definition. The platform team wrote the definition to
integrate software according to bank policies. CUE powers this _unified_
platform configuration model.
:::
:::tip
Components map 1:1 to ArgoCD Applications or Flux Kustomizations.
:::
### Development Team
The development team has the platform resources they need, but they still need
to deploy their container. The development team submits a pull request adding
the following two files to deploy their existing Helm chart.
<Tabs groupId="7AD1DDA9-8001-462B-8BE0-D9410EB51233">
<TabItem value="projects/experiment/components/podinfo/podinfo.cue" label="Helm Component">
```txt
projects/experiment/components/podinfo/podinfo.cue
```
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
_HelmChart.BuildPlan
_HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
This file represents a Helm chart component to add to the platform. The second
tab registers this component with the platform.
</TabItem>
<TabItem value="platform/podinfo.cue" label="Component Registration">
```
platform/podinfo.cue
```
```cue showLineNumbers
package holos
// Manage the component on every workload Cluster, but not management clusters.
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name):podinfo": {
name: "podinfo"
component: "projects/experiment/components/podinfo"
cluster: Cluster.name
tags: project: "experiment"
}
}
```
This file registers the component with the platform. When the platform is
rendered the dev team's Helm chart will be rendered on all workload clusters
across the platform.
</TabItem>
</Tabs>
The project tag associates the `#Platform` component with the correct entry in the `#Projects` struct.
:::important
You can add your own key=value tags in your platform specification to inject
values into components. This feature is useful to reuse one component path for
several environments or customers.
:::
Once the dev team's component is registered, rendering the platform will render
their component.
<Tabs groupId="1BAF7AD2-BBCD-4797-A3A6-55A626732845">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered namespaces for cluster overview in 185.64075ms
rendered app-projects for cluster overview in 186.729292ms
rendered httproutes for cluster overview in 195.222833ms
rendered projects for cluster overview in 195.217125ms
// highlight-next-line
rendered podinfo for cluster overview in 195.830042ms
rendered platform in 195.90275ms
```
</TabItem>
</Tabs>
<Tabs groupId="77BF500B-105A-4AB4-A615-DEC19F501AE1">
<TabItem value="command" label="Command">
```bash
cat deploy/clusters/overview/components/podinfo/podinfo.gen.yaml
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
helm.sh/chart: podinfo-6.6.2
holos.run/component.name: podinfo
name: podinfo
spec:
ports:
- name: http
port: 9898
protocol: TCP
targetPort: http
- name: grpc
port: 9999
protocol: TCP
targetPort: grpc
selector:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: podinfo
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
helm.sh/chart: podinfo-6.6.2
holos.run/component.name: podinfo
name: podinfo
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: podinfo
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
prometheus.io/port: "9898"
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: podinfo
spec:
containers:
- command:
- ./podinfo
- --port=9898
- --cert-path=/data/cert
- --port-metrics=9797
- --grpc-port=9999
- --grpc-service-name=podinfo
- --level=info
- --random-delay=false
- --random-error=false
env:
- name: PODINFO_UI_COLOR
value: '#34577c'
image: ghcr.io/stefanprodan/podinfo:6.6.2
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/healthz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: podinfo
ports:
- containerPort: 9898
name: http
protocol: TCP
- containerPort: 9797
name: http-metrics
protocol: TCP
- containerPort: 9999
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/readyz
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits: null
requests:
cpu: 1m
memory: 16Mi
volumeMounts:
- mountPath: /data
name: data
terminationGracePeriodSeconds: 30
volumes:
- emptyDir: {}
name: data
```
</TabItem>
</Tabs>
Note the rendered Helm chart resources have consistent project labels. The
platform team added a constraint to the project so all Helm charts are post
processed with Kustomize to add these common labels. The platform team
accomplishes this by adding a constraint in the project directory. This can be
seen in
[schema.cue](https://github.com/holos-run/bank-of-holos/blob/v0.2.0/schema.cue#L35-L40)
where the platform team configures all component kinds for the platform.
We've covered how the platform team provides a golden path for development teams
to register their projects by defining a Projects structure. We've also covered
how the development team deploys their existing Helm chart onto the platform.
## Support & Resources
1. See our [Quickstart] guide to get started with Holos.
2. Check out our other [Guides] which cover specific topics.
3. Refer to the [Author API] when writing components.
4. Consider the [Core API] if you need to do something more advanced than the Author API supports.
5. Community and commercial [Support] is available.
6. [Discussions Forum](https://github.com/holos-run/holos/discussions)
[Support]: /docs/support/
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[CUE]: https://cuelang.org/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story
[Holos]: https://holos.run/
[Quickstart]: /docs/quickstart/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern/
[examples/tech-overview]: https://github.com/holos-run/bank-of-holos/tree/v0.2.0/examples/tech-overview
[BackendObjectReference]: https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.BackendObjectReference
[CODEOWNERS]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[Project]: /docs/api/author/v1alpha3/#Project

View File

@@ -1,8 +0,0 @@
---
slug: welcome
title: Welcome
authors: [jeff]
tags: [holos]
---
TODO - Coming Soon

View File

@@ -0,0 +1,94 @@
---
slug: holos-platform-manager
title: Holos Platform Manager
authors: [jeff]
tags: [holos]
---
## Introducing Holos
Im excited to announce Holos, a tool designed to help engineering teams
manage their software development platforms built on the Kubernetes resource
model.
:::tip
For a hands-on introduction, check out our [Quickstart] Guide.
:::
<!-- truncate -->
### The Backstory
In our roles at [Open Infrastructure Services], and earlier at Puppet, we helped
many companies automate infrastructure management. In 2017, we had the
opportunity to work with Twitter to improve their configuration management
system. This opportunity gave us insight into the challenges of managing a
large-scale platform with multiple engineering teams. Our work involved
everything from observability systems to application deployment workflows and of
course, managing the core infrastructure.
This experience demonstrated the value of platform engineering. As the pandemic
hit, I began thinking about what a fully cloud-native platform might look like
using the Kubernetes resource model. Around the same time, I came across the
Hacker News post, “[Why Are We Templating YAML]?”, which sparked a good
discussion. It was clear I wasnt alone in my frustration with managing YAML
files and ensuring clear, predictable changes before merging them into
production.
A common pain point and theme is the complexity of working with nested YAML
configurations, especially with tools like ArgoCD and Helm. The lack of a
standard for rendering YAML templates makes it difficult to see what changes are
actually being applied to the Kubernetes API. This often results in trial and
error, costly blue-green deployments, and hours of debugging.
During the pandemic, I began experimenting with a tool to address this issue,
drawing on lessons from our work at Twitter. The key problems we aimed to solve
are:
- **Lack of visibility**: Engineers struggled to foresee the impact of small changes.
- **Large blast radius**: Small changes affected global systems, with no way to limit the impact.
- **Incomplete tooling**: While processes were in place, the right information wasnt surfaced at the right time.
We built several iterations of a reference platform based on Kubernetes,
initially focusing on fully rendering manifests into plain files—a pattern now
called the [rendered manifests pattern]. Over time, we realized we were spending
most of our time maintaining bash scripts and YAML templates. This led back to
the question: Why are we templating YAML? What _should_ replace templates?
We'd previously seen a colleague use CUE effectively to generate large scale
configurations for Envoy, and ran into CUE again when we worked on a project
involving Dagger, but I still hadn't taken a deep look at CUE.
At the end of 2023, I decided to dive deep with [CUE]. I quickly came to
appreciate CUEs unified approach where **order is irrelevant**. Before CUE, we
handled configuration data in a hierarchy with a precedence ordering, similar to
how we handled data in Puppet with Hiera. CUE's promise of no longer needing to
think about ordering and precedence rules held, alleviating a large cognitive
burden when dealing with complex configurations. CUE quickly allowed me to
replace the unmaintainable bash scripts and complex Helm templates, simplifying
our workflow.
### Enter Holos
Holos adds CUE as a well-specified integration layer over tools like Helm,
Kustomize, ArgoCD, and Crossplane. With Holos, we can now efficiently integrate
upstream Helm charts and Kustomize bases into our platform without the
complexity of templates and scripts. This has also made it easy for one team to
define "golden paths" that other teams can follow—like automatically configuring
namespaces and security policies when dev teams start new projects.
We've found Holos incredibly useful and hope you do too. Let us know your
thoughts!
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[CUE]: https://cuelang.org/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story
[Holos]: https://holos.run/
[Quickstart]: /docs/quickstart/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern/

View File

@@ -1,6 +1,6 @@
jeff:
name: Jeff McCune
title: Holos maintainer & creator
title: Holos maintainer & author
url: https://github.com/jeffmccune
image_url: https://github.com/jeffmccune.png

View File

@@ -2,3 +2,8 @@ holos:
label: Holos
permalink: /holos
description: Holos Platform
tech:
label: Technical
permalink: /tech
description: Technical Articles

View File

@@ -4,7 +4,7 @@ import type * as Preset from '@docusaurus/preset-classic';
const config: Config = {
title: 'Holos',
tagline: 'The Holistic Platform Manager',
tagline: 'An easier way for platform teams to integrate software into their platform',
favicon: 'img/favicon.ico',
// Set the production url of your site here
@@ -12,6 +12,7 @@ const config: Config = {
// Set the /<baseUrl>/ pathname under which your site is served
// For GitHub pages deployment, it is often '/<projectName>/'
baseUrl: '/',
// trailing slash is necessary for Cloudflare pages.
trailingSlash: true,
// GitHub pages deployment config.
@@ -65,13 +66,17 @@ const config: Config = {
blogSidebarTitle: "All posts",
feedOptions: {
type: 'all',
copyright: `Copyright © ${new Date().getFullYear()}, The Holos Authors.`,
copyright: `Copyright © ${new Date().getFullYear()}, The Holos Authors`,
},
showReadingTime: false,
},
theme: {
customCss: './src/css/custom.css',
},
gtag: {
trackingID: 'G-M00QMB1N05',
anonymizeIP: true,
}
} satisfies Preset.Options,
],
],
@@ -95,9 +100,10 @@ const config: Config = {
type: 'doc',
docId: 'guides/quickstart',
position: 'left',
label: 'Try Holos',
label: 'Quickstart',
},
{ to: '/docs', label: 'Docs', position: 'left' },
{ to: '/docs/technical-overview', label: 'Docs', position: 'left' },
{ to: '/docs/guides', label: 'Guides', position: 'left' },
{
type: 'doc',
docId: 'api',
@@ -106,13 +112,7 @@ const config: Config = {
},
{ to: '/blog', label: 'Blog', position: 'left' },
{
"href": "https://pkg.go.dev/github.com/holos-run/holos?tab=doc",
"label": "GoDoc",
"position": "left",
"className": "header-godoc-link",
},
{
href: 'https://github.com/holos-run/holos',
href: 'https://github.com/holos-run',
label: 'GitHub',
position: 'right',
},
@@ -146,7 +146,19 @@ const config: Config = {
title: 'Community',
items: [
{
label: 'Discuss',
label: 'Support',
href: '/docs/support',
},
{
label: 'Announcements List',
href: 'https://groups.google.com/g/holos-announce',
},
{
label: 'Discussion List',
href: 'https://groups.google.com/g/holos-discuss',
},
{
label: 'Discussion Forum',
href: 'https://github.com/holos-run/holos/discussions',
},
],
@@ -162,6 +174,10 @@ const config: Config = {
label: 'GitHub',
href: 'https://github.com/holos-run/holos',
},
{
label: 'GoDoc',
href: 'https://pkg.go.dev/github.com/holos-run/holos?tab=doc',
}
],
},
],

View File

@@ -13,6 +13,7 @@ import type { SidebarsConfig } from '@docusaurus/plugin-content-docs';
const sidebars: SidebarsConfig = {
doc: [
'introduction',
'technical-overview',
{
label: 'Getting Started',
type: 'category',

View File

@@ -8,53 +8,56 @@ type FeatureItem = {
description: JSX.Element;
};
// TODO: Consider focusing on the three pillars of Safe, Easy, Consistent.
// We don't focus on features, but rather problems and solutions.
const FeatureList: FeatureItem[] = [
{
title: 'Kustomize Helm',
Svg: require('@site/static/img/base00/undraw_together_re_a8x4.svg').default,
title: 'For Platform Engineers',
Svg: require('@site/static/img/base00/undraw_software_engineer_re_tnjc.svg').default,
description: (
<>
Super charge your existing Helm charts by providing well defined,
validated input values, post-processing the output with Kustomize,
and mixing in your own custom resources. All without modifying upstream
charts to alleviate the pain of upgrades.
<p align="left">
<ul>
<li>Provide simple definitions for other teams to use as golden paths.</li>
<li>Define integrations in <a href="https://cuelang.org/">CUE</a> with strong type checking. No more text templates or bash scripts.</li>
<li>Reuse your existing Helm charts and Kustomize bases.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
</>
),
},
{
title: 'Unified Data Model',
Svg: require('@site/static/img/base00/undraw_fitting_pieces_re_nss7.svg').default,
title: 'For Software Developers',
Svg: require('@site/static/img/base00/undraw_through_the_park_lxnl.svg').default,
description: (
<>
Unify all of your platform components into one well-defined, strongly
typed data model with CUE. Holos makes it easier and safer to integrate
seamlessly with software distributed with current and future tools that
produce Kubernetes resource manifests.
<p align="left">
<ul>
<li>Move faster using paved paths from your platform and security teams.</li>
<li>Develop locally or in the cloud.</li>
<li>Spend more time developing software and fewer cycles fighting infrastructure challenges.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
</>
),
},
{
title: 'Deep Insights',
Svg: require('@site/static/img/base00/undraw_code_review_re_woeb.svg').default,
title: 'For Security Teams',
Svg: require('@site/static/img/base00/undraw_security_on_re_e491.svg').default,
description: (
<>
Reduce risk and increase confidence in your configuration changes.
Holos offers clear visibility into platform wide changes before the
change is made.
<p align="left">
<ul>
<li>Define security policy as reusable, typed configurations.</li>
<li>Automatically enforce security policy on new projects.</li>
<li>Ensure a consistent security posture cross-platform with fewer code changes.</li>
</ul>
</p>
<a href="/docs/technical-overview">Learn More</a>
</>
),
},
{
title: 'Interoperable',
Svg: require('@site/static/img/base00/undraw_version_control_re_mg66.svg').default,
description: (
<>
Holos is designed for compatibility with your preferred tools and
processes, for example git diff and code reviews.
</>
),
},
}
];
function Feature({ title, Svg, description }: FeatureItem) {

View File

@@ -10,6 +10,37 @@
overflow-wrap: anywhere;
} */
.hero__title {
text-align: left;
}
.hero__subtitle {
text-align: left;
}
.projectDesc {
text-align: left;
}
.hero__buttons {
float: left
}
/* Ensure img in hero banner scales well even on mobile */
@media screen and (max-width: 996px) {
div.diagramImg {
width: 100%;
max-width: 100px;
height: auto;
}
}
div.diagramImg {
width: 30%;
min-width: 300px;
float: right;
}
/* You can override the default Infima variables here. */
:root {
--ifm-link-color: #268bd2;

View File

@@ -12,25 +12,30 @@ function HomepageHeader() {
return (
<header className={clsx('hero hero--primary', styles.heroBanner)}>
<div className="container">
<div className="diagramImg">
<img src="./img/holos-diagram-color-transparent.svg" alt="Holos Diagram" />
</div>
<Heading as="h1" className="hero__title">
{siteConfig.title}
</Heading>
<p className="hero__subtitle">{siteConfig.tagline}</p>
<p className="projectDesc">
Holos adds CUE's type safety, unified structure, and strong validation
features to your Kubernetes configuration manifests, including Helm
and Kustomize. These features make the experience of integrating
software into a holistic platform a pleasant journey.
</p>
<div className={styles.buttons}>
<Link
className="button button--secondary button--lg"
to="docs/quickstart">
Learn More
</Link>
<span className={styles.divider}></span>
<div className="hero__buttons">
<div className={styles.buttons}>
<Link
className="button button--secondary button--lg"
to="docs/quickstart">
Get Started
</Link>
<span className={styles.divider}></span>
<Link
className="button button--primary button--lg"
to="docs/technical-overview/">
Learn More
</Link>
<span className={styles.divider}></span>
</div>
</div>
</div>
</div >
</header >
);
}

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 33 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 33 KiB

View File

@@ -4,4 +4,4 @@ set -euo pipefail
TOPLEVEL="$(cd $(dirname "$0") && git rev-parse --show-toplevel)"
cd "${TOPLEVEL}" && npx cspell ./doc/md/**/*.{md,mdx,markdown}
cd "${TOPLEVEL}" && npx cspell ./doc/md/**/*.{md,mdx,markdown} ./doc/md/*.{md,mdx,markdown}

View File

@@ -1,6 +1,8 @@
// Package holos defines types for the rest of the system.
package holos
import "context"
// A PathCueMod is a string representing the absolute filesystem path of a cue
// module. It is given a unique type so the API is clear.
type PathCueMod string
@@ -8,3 +10,33 @@ type PathCueMod string
// A InstancePath is a string representing the absolute filesystem path of a
// holos instance. It is given a unique type so the API is clear.
type InstancePath string
// FilePath represents the path of a file relative to the current working
// directory of holos at runtime.
type FilePath string
// FileContent represents the contents of a file as a string.
type FileContent string
// TypeMeta represents the kind and version of a resource holos needs to
// process. Useful to discriminate generated resources.
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
}
// Builder builds file artifacts.
type Builder interface {
Build(context.Context, ArtifactMap) error
}
// ArtifactMap sets and gets data for file artifacts.
//
// Concrete values must ensure Set is write once, returning an error if a given
// FilePath was previously Set. Concrete values must be safe for concurrent
// reads and writes.
type ArtifactMap interface {
Get(path string) (data []byte, ok bool)
Set(path string, data []byte) error
Save(dir, path string) error
}

View File

@@ -0,0 +1,69 @@
package artifact
import (
"fmt"
"os"
"path/filepath"
"sync"
"github.com/holos-run/holos/internal/errors"
)
func New() *Artifact {
return &Artifact{m: make(map[string][]byte)}
}
// Artifact represents the fully rendered manifests build from the holos
// rendering pipeline. Files are organized by keys representing paths relative
// to the current working directory. Values represent the file content.
type Artifact struct {
mu sync.RWMutex
m map[string][]byte
}
// Set sets an artifact file with write locking. Set returns an error if the
// artifact was previously set.
func (a *Artifact) Set(path string, data []byte) error {
a.mu.Lock()
defer a.mu.Unlock()
if _, ok := a.m[path]; ok {
return errors.Format("%s already set", path)
}
a.m[path] = data
return nil
}
// Get gets the content of an artifact with read locking.
func (a *Artifact) Get(path string) (data []byte, ok bool) {
a.mu.RLock()
defer a.mu.RUnlock()
data, ok = a.m[path]
return
}
// Save writes a file to the filesystem.
func (a *Artifact) Save(dir, path string) error {
fullPath := filepath.Join(dir, path)
msg := fmt.Sprintf("could not save %s", fullPath)
data, ok := a.Get(path)
if !ok {
return errors.Format("%s: missing %s", msg, path)
}
if err := os.MkdirAll(filepath.Dir(fullPath), 0777); err != nil {
return errors.Format("%s: %w", msg, err)
}
if err := os.WriteFile(fullPath, data, 0666); err != nil {
return errors.Format("%s: %w", msg, err)
}
return nil
}
func (a *Artifact) Keys() []string {
a.mu.RLock()
defer a.mu.RUnlock()
keys := make([]string, 0, len(a.m))
for key := range a.m {
keys = append(keys, key)
}
return keys
}

View File

@@ -15,10 +15,12 @@ import (
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
"cuelang.org/go/cue/load"
v1 "github.com/holos-run/holos/api/core/v1alpha3"
"github.com/holos-run/holos/api/v1alpha1"
"github.com/holos-run/holos"
core_v1alpha2 "github.com/holos-run/holos/api/core/v1alpha2"
core_v1alpha3 "github.com/holos-run/holos/api/core/v1alpha3"
meta_v1alpha2 "github.com/holos-run/holos/api/meta/v1alpha2"
"github.com/holos-run/holos/api/v1alpha1"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
@@ -26,10 +28,10 @@ import (
)
const (
KubernetesObjects = v1.KubernetesObjectsKind
KubernetesObjects = core_v1alpha3.KubernetesObjectsKind
// Helm is the value of the kind field of holos build output indicating helm
// values and helm command information.
Helm = v1.HelmChartKind
Helm = core_v1alpha3.HelmChartKind
// Skip is the value when the instance should be skipped
Skip = "Skip"
// KustomizeBuild is the value of the kind field of cue output indicating
@@ -43,11 +45,7 @@ type Option func(*config)
type config struct {
args []string
cluster string
}
type Builder struct {
cfg config
ctx *cue.Context
tags []string
}
// BuildData represents the data necessary to produce a build plan. It is a
@@ -59,8 +57,26 @@ type BuildData struct {
Dir string
}
func (bd *BuildData) TypeMeta() (tm holos.TypeMeta, err error) {
jsonBytes, err := bd.Value.MarshalJSON()
if err != nil {
err = errors.Format("could not marshal json %s: %w", bd.Dir, err)
return
}
err = json.NewDecoder(bytes.NewReader(jsonBytes)).Decode(&tm)
if err != nil {
err = errors.Format("could not decode type meta %s: %w", bd.Dir, err)
}
return
}
type Builder struct {
cfg config
ctx *cue.Context
}
type buildPlanWrapper struct {
buildPlan *v1.BuildPlan
buildPlan *core_v1alpha3.BuildPlan
}
func (b *buildPlanWrapper) validate() error {
@@ -72,7 +88,7 @@ func (b *buildPlanWrapper) validate() error {
return fmt.Errorf("invalid BuildPlan: is nil")
}
errs := make([]string, 0, 2)
if bp.Kind != v1.BuildPlanKind {
if bp.Kind != core_v1alpha3.BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", v1alpha1.BuildPlanKind, bp.Kind))
}
if len(errs) > 0 {
@@ -116,6 +132,11 @@ func Cluster(name string) Option {
return func(cfg *config) { cfg.cluster = name }
}
// Tags configures tags to pass to cue when building the instance.
func Tags(tags []string) Option {
return func(cfg *config) { cfg.tags = tags }
}
// Cluster returns the cluster name of the component instance being built.
func (b *Builder) Cluster() string {
return b.cfg.cluster
@@ -141,15 +162,28 @@ func (b *Builder) Unify(ctx context.Context, cfg *client.Config) (bd BuildData,
return bd, errors.Wrap(fmt.Errorf("could not load platform model: %w", err))
}
// TODO(jeff): Changing these tag names breaks backwards compatibility. We
// need to refactor this unification into a versioned builder, at least at the
// component level. Right now it's executed when rendering the initial
// Platform spec, which should be backwards compatible but isn't because this
// package is shared by all versions.
tags := make([]string, 0, len(b.cfg.tags)+2)
// TODO: Use instance.FillPath to fill the platform config.
// Refer to https://pkg.go.dev/cuelang.org/go/cue#Value.FillPath
tags = append(tags, "holos_platform_config="+string(platformConfigData))
// TODO(jeff): This is hacky after I switched to reserved holos_ tag names in
// v1alpha4. Could use some serious clean up now that --cluster-name is
// deprecated for --inject holos_cluster=foo, but it was kind of nice to have
// a required argument.
if cluster := cfg.Holos().ClusterName(); cluster != "" {
tags = append(tags, "holos_cluster="+cluster)
}
tags = append(tags, b.cfg.tags...)
cueConfig := load.Config{
Dir: bd.ModuleRoot,
ModuleRoot: bd.ModuleRoot,
// TODO: Use instance.FillPath to fill the platform config.
// Refer to https://pkg.go.dev/cuelang.org/go/cue#Value.FillPath
Tags: []string{
"cluster=" + cfg.Holos().ClusterName(),
"platform_config=" + string(platformConfigData),
},
Tags: tags,
}
// Make args relative to the module directory
@@ -229,6 +263,8 @@ func loadUserData(ctx *cue.Context, moduleRoot string) (val cue.Value, err error
// provided to the entrypoint through a json encoded string tag named
// platform_config. The resulting cue.Value is unified with all user data files
// at the path "#UserData".
//
// Deprecated: Use holos.Builder instead
func (b *Builder) Run(ctx context.Context, cfg *client.Config) (results []*render.Result, err error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building instances")
@@ -240,7 +276,7 @@ func (b *Builder) Run(ctx context.Context, cfg *client.Config) (results []*rende
return b.build(ctx, bd)
}
func (b Builder) build(ctx context.Context, bd BuildData) (results []*render.Result, err error) {
func (b *Builder) build(ctx context.Context, bd BuildData) (results []*render.Result, err error) {
log := logger.FromContext(ctx).With("dir", bd.InstancePath)
value := bd.Value
@@ -279,7 +315,7 @@ func (b Builder) build(ctx context.Context, bd BuildData) (results []*render.Res
switch tm.Kind {
case "BuildPlan":
var bp v1.BuildPlan
var bp core_v1alpha3.BuildPlan
if err = decoder.Decode(&bp); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode BuildPlan %s: %w", bd.Dir, err))
return
@@ -295,7 +331,7 @@ func (b Builder) build(ctx context.Context, bd BuildData) (results []*render.Res
return results, err
}
func (b *Builder) buildPlan(ctx context.Context, buildPlan *v1.BuildPlan, path holos.InstancePath) (results []*render.Result, err error) {
func (b *Builder) buildPlan(ctx context.Context, buildPlan *core_v1alpha3.BuildPlan, path holos.InstancePath) (results []*render.Result, err error) {
log := logger.FromContext(ctx)
bpw := buildPlanWrapper{buildPlan: buildPlan}
@@ -381,3 +417,62 @@ func (b *Builder) findCueMod() (dir holos.PathCueMod, err error) {
}
return dir, nil
}
// Platform builds a platform
// TODO: Refactor, lift up into NewPlatform RunE.
func (b *Builder) Platform(ctx context.Context, cfg *client.Config) (*core_v1alpha2.Platform, error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building platform instance")
bd, err := b.Unify(ctx, cfg)
if err != nil {
return nil, errors.Wrap(err)
}
return b.runPlatform(ctx, bd)
}
func (b *Builder) runPlatform(ctx context.Context, bd BuildData) (*core_v1alpha2.Platform, error) {
path := holos.InstancePath(bd.Dir)
log := logger.FromContext(ctx).With("dir", path)
value := bd.Value
if err := bd.Value.Err(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not load: %w", err))
}
log.DebugContext(ctx, "cue: validating instance")
if err := value.Validate(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not validate: %w", err))
}
log.DebugContext(ctx, "cue: decoding holos platform")
jsonBytes, err := value.MarshalJSON()
if err != nil {
return nil, errors.Wrap(fmt.Errorf("could not marshal cue instance %s: %w", bd.Dir, err))
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
// Discriminate the type of build plan.
tm := &meta_v1alpha2.TypeMeta{}
err = decoder.Decode(tm)
if err != nil {
return nil, errors.Wrap(fmt.Errorf("invalid platform: %s: %w", bd.Dir, err))
}
log.DebugContext(ctx, "cue: discriminated build kind: "+tm.GetKind(), "kind", tm.GetKind(), "apiVersion", tm.GetAPIVersion())
decoder = json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
var pf core_v1alpha2.Platform
switch tm.GetKind() {
case "Platform":
if err = decoder.Decode(&pf); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode platform %s: %w", bd.Dir, err))
return nil, err
}
return &pf, nil
default:
err = errors.Wrap(fmt.Errorf("unknown kind: %v", tm.GetKind()))
}
return nil, err
}

View File

@@ -1,75 +0,0 @@
package builder
import (
"bytes"
"context"
"encoding/json"
"fmt"
"github.com/holos-run/holos"
core "github.com/holos-run/holos/api/core/v1alpha2"
meta "github.com/holos-run/holos/api/meta/v1alpha2"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
)
// Platform builds a platform
func (b *Builder) Platform(ctx context.Context, cfg *client.Config) (*core.Platform, error) {
log := logger.FromContext(ctx)
log.DebugContext(ctx, "cue: building platform instance")
bd, err := b.Unify(ctx, cfg)
if err != nil {
return nil, errors.Wrap(err)
}
return b.runPlatform(ctx, bd)
}
func (b Builder) runPlatform(ctx context.Context, bd BuildData) (*core.Platform, error) {
path := holos.InstancePath(bd.Dir)
log := logger.FromContext(ctx).With("dir", path)
value := bd.Value
if err := bd.Value.Err(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not load: %w", err))
}
log.DebugContext(ctx, "cue: validating instance")
if err := value.Validate(); err != nil {
return nil, errors.Wrap(fmt.Errorf("could not validate: %w", err))
}
log.DebugContext(ctx, "cue: decoding holos platform")
jsonBytes, err := value.MarshalJSON()
if err != nil {
return nil, errors.Wrap(fmt.Errorf("could not marshal cue instance %s: %w", bd.Dir, err))
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
// Discriminate the type of build plan.
tm := &meta.TypeMeta{}
err = decoder.Decode(tm)
if err != nil {
return nil, errors.Wrap(fmt.Errorf("invalid platform: %s: %w", bd.Dir, err))
}
log.DebugContext(ctx, "cue: discriminated build kind: "+tm.GetKind(), "kind", tm.GetKind(), "apiVersion", tm.GetAPIVersion())
// New decoder for the full object
decoder = json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
var pf core.Platform
switch tm.GetKind() {
case "Platform":
if err = decoder.Decode(&pf); err != nil {
err = errors.Wrap(fmt.Errorf("could not decode platform %s: %w", bd.Dir, err))
return nil, err
}
return &pf, nil
default:
err = errors.Wrap(fmt.Errorf("unknown kind: %v", tm.GetKind()))
}
return nil, err
}

View File

@@ -0,0 +1,550 @@
package v1alpha4
import (
"bytes"
"context"
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
"strings"
"syscall"
"time"
h "github.com/holos-run/holos"
"github.com/holos-run/holos/api/core/v1alpha4"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/logger"
"github.com/holos-run/holos/internal/util"
"golang.org/x/sync/errgroup"
"gopkg.in/yaml.v3"
)
// Platform represents a platform builder.
type Platform struct {
Platform v1alpha4.Platform
Concurrency int
Stderr io.Writer
}
// Build builds a Platform by concurrently building a BuildPlan for each
// platform component. No artifact files are written directly, only indirectly
// by rendering each component.
func (p *Platform) Build(ctx context.Context, _ h.ArtifactMap) error {
parentStart := time.Now()
components := p.Platform.Spec.Components
total := len(components)
g, ctx := errgroup.WithContext(ctx)
// Limit the number of concurrent goroutines due to CUE memory usage concerns
// while rendering components. One more for the producer.
g.SetLimit(p.Concurrency + 1)
// Spawn a producer because g.Go() blocks when the group limit is reached.
g.Go(func() error {
for idx := range components {
select {
case <-ctx.Done():
return ctx.Err()
default:
// Capture idx to avoid issues with closure. Fixed in Go 1.22.
idx := idx
component := &components[idx]
// Worker go routine. Blocks if limit has been reached.
g.Go(func() error {
select {
case <-ctx.Done():
return ctx.Err()
default:
start := time.Now()
log := logger.FromContext(ctx).With(
"name", component.Name,
"path", component.Component,
"cluster", component.Cluster,
"num", idx+1,
"total", total,
)
log.DebugContext(ctx, "render component")
tags := make([]string, 0, 3+len(component.Tags))
tags = append(tags, "holos_name="+component.Name)
tags = append(tags, "holos_component="+component.Component)
tags = append(tags, "holos_cluster="+component.Cluster)
for key, value := range component.Tags {
tags = append(tags, fmt.Sprintf("%s=%s", key, value))
}
// Execute a sub-process to limit CUE memory usage.
args := make([]string, 0, 10)
args = append(args,
"render",
"component",
)
for _, tag := range tags {
args = append(args, "--inject", tag)
}
if component.WriteTo != "" {
args = append(args, "--write-to", component.WriteTo)
}
args = append(args, component.Component)
result, err := util.RunCmd(ctx, "holos", args...)
// I've lost an hour+ digging into why I couldn't see log output
// from sub-processes. Make sure to surface at least stderr from
// sub-processes.
_, _ = io.Copy(p.Stderr, result.Stderr)
if err != nil {
return errors.Wrap(fmt.Errorf("could not render component: %w", err))
}
duration := time.Since(start)
msg := fmt.Sprintf(
"rendered %s for cluster %s in %s",
component.Name,
component.Cluster,
duration,
)
log.InfoContext(ctx, msg, "duration", duration)
return nil
}
})
}
}
return nil
})
// Wait for completion and return the first error (if any)
if err := g.Wait(); err != nil {
return err
}
duration := time.Since(parentStart)
msg := fmt.Sprintf("rendered platform in %s", duration)
logger.FromContext(ctx).InfoContext(ctx, msg, "duration", duration, "version", p.Platform.APIVersion)
return nil
}
// BuildPlan represents a component builder.
type BuildPlan struct {
BuildPlan v1alpha4.BuildPlan
Concurrency int
Stderr io.Writer
// WriteTo --write-to=deploy flag
WriteTo string
// Path represents the path to the component
Path h.InstancePath
}
// Build builds a BuildPlan into Artifact files.
func (b *BuildPlan) Build(ctx context.Context, am h.ArtifactMap) error {
name := b.BuildPlan.Metadata.Name
component := b.BuildPlan.Spec.Component
log := logger.FromContext(ctx).With("name", name, "component", component)
msg := fmt.Sprintf("could not build %s", name)
if b.BuildPlan.Spec.Disabled {
log.WarnContext(ctx, fmt.Sprintf("%s: disabled", msg))
return nil
}
g, ctx := errgroup.WithContext(ctx)
// One more for the producer
g.SetLimit(b.Concurrency + 1)
// Producer.
g.Go(func() error {
for _, a := range b.BuildPlan.Spec.Artifacts {
msg := fmt.Sprintf("%s artifact %s", msg, a.Artifact)
log := log.With("artifact", a.Artifact)
if a.Skip {
log.WarnContext(ctx, fmt.Sprintf("%s: skipped field is true", msg))
continue
}
select {
case <-ctx.Done():
return ctx.Err()
default:
// https://golang.org/doc/faq#closures_and_goroutines
a := a
// Worker. Blocks if limit has been reached.
g.Go(func() error {
for _, gen := range a.Generators {
switch gen.Kind {
case "Resources":
if err := b.resources(log, gen, am); err != nil {
return errors.Format("could not generate resources: %w", err)
}
case "Helm":
if err := b.helm(ctx, log, gen, am); err != nil {
return errors.Format("could not generate helm: %w", err)
}
case "File":
if err := b.file(log, gen, am); err != nil {
return errors.Format("could not generate file: %w", err)
}
default:
return errors.Format("%s: unsupported kind %s", msg, gen.Kind)
}
}
for _, t := range a.Transformers {
switch t.Kind {
case "Kustomize":
if err := b.kustomize(ctx, log, t, am); err != nil {
return errors.Wrap(err)
}
case "Join":
s := make([][]byte, 0, len(t.Inputs))
for _, input := range t.Inputs {
if data, ok := am.Get(string(input)); ok {
s = append(s, data)
} else {
return errors.Format("%s: missing %s", msg, input)
}
}
data := bytes.Join(s, []byte(t.Join.Separator))
if err := am.Set(string(t.Output), data); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact: " + string(t.Output))
default:
return errors.Format("%s: unsupported kind %s", msg, t.Kind)
}
}
// Write the final artifact
if err := am.Save(b.WriteTo, string(a.Artifact)); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+filepath.Join(b.WriteTo, string(a.Artifact)))
return nil
})
}
}
return nil
})
// Wait for completion and return the first error (if any)
return g.Wait()
}
func (b *BuildPlan) file(
log *slog.Logger,
g v1alpha4.Generator,
am h.ArtifactMap,
) error {
data, err := os.ReadFile(filepath.Join(string(b.Path), string(g.File.Source)))
if err != nil {
return errors.Wrap(err)
}
if err := am.Set(string(g.Output), data); err != nil {
return errors.Wrap(err)
}
log.Debug("set artifact: " + string(g.Output))
return nil
}
func (b *BuildPlan) helm(
ctx context.Context,
log *slog.Logger,
g v1alpha4.Generator,
am h.ArtifactMap,
) error {
chartName := g.Helm.Chart.Name
log = log.With("chart", chartName)
// Unnecessary? cargo cult copied from internal/cli/render/render.go
if chartName == "" {
return errors.New("missing chart name")
}
// Cache the chart by version to pull new versions. (#273)
cacheDir := filepath.Join(string(b.Path), "vendor", g.Helm.Chart.Version)
cachePath := filepath.Join(cacheDir, filepath.Base(chartName))
if _, err := os.Stat(cachePath); os.IsNotExist(err) {
timeout, cancel := context.WithTimeout(ctx, 5*time.Minute)
defer cancel()
err := onceWithLock(log, timeout, cachePath, func() error {
return b.cacheChart(ctx, log, cacheDir, g.Helm.Chart)
})
if err != nil {
return errors.Format("could not cache chart: %w", err)
}
}
// Write values file
tempDir, err := os.MkdirTemp("", "holos.helm")
if err != nil {
return errors.Format("could not make temp dir: %w", err)
}
defer util.Remove(ctx, tempDir)
data, err := yaml.Marshal(g.Helm.Values)
if err != nil {
return errors.Format("could not marshal values: %w", err)
}
valuesPath := filepath.Join(tempDir, "values.yaml")
if err := os.WriteFile(valuesPath, data, 0666); err != nil {
return errors.Wrap(fmt.Errorf("could not write values: %w", err))
}
log.DebugContext(ctx, "wrote"+valuesPath)
// Run charts
args := []string{"template"}
if !g.Helm.EnableHooks {
args = append(args, "--no-hooks")
}
args = append(args,
"--include-crds",
"--values", valuesPath,
"--namespace", g.Helm.Namespace,
"--kubeconfig", "/dev/null",
"--version", g.Helm.Chart.Version,
g.Helm.Chart.Release,
cachePath,
)
helmOut, err := util.RunCmd(ctx, "helm", args...)
if err != nil {
stderr := helmOut.Stderr.String()
lines := strings.Split(stderr, "\n")
for _, line := range lines {
if strings.HasPrefix(line, "Error:") {
err = fmt.Errorf("%s: %w", line, err)
}
}
return errors.Format("could not run helm template: %w", err)
}
// Set the artifact
if err := am.Set(string(g.Output), helmOut.Stdout.Bytes()); err != nil {
return errors.Format("could not store helm output: %w", err)
}
log.Debug("set artifact: " + string(g.Output))
return nil
}
func (b *BuildPlan) resources(
log *slog.Logger,
g v1alpha4.Generator,
am h.ArtifactMap,
) error {
var size int
for _, m := range g.Resources {
size += len(m)
}
list := make([]v1alpha4.Resource, 0, size)
for _, m := range g.Resources {
for _, r := range m {
list = append(list, r)
}
}
msg := fmt.Sprintf("could not generate %s for %s path %s", g.Output, b.BuildPlan.Metadata.Name, b.BuildPlan.Spec.Component)
buf, err := marshal(list)
if err != nil {
return errors.Format("%s: %w", msg, err)
}
if err := am.Set(string(g.Output), buf.Bytes()); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact " + string(g.Output))
return nil
}
func (b *BuildPlan) kustomize(
ctx context.Context,
log *slog.Logger,
t v1alpha4.Transformer,
am h.ArtifactMap,
) error {
tempDir, err := os.MkdirTemp("", "holos.kustomize")
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, tempDir)
msg := fmt.Sprintf("could not transform %s for %s path %s", t.Output, b.BuildPlan.Metadata.Name, b.BuildPlan.Spec.Component)
// Write the kustomization
data, err := yaml.Marshal(t.Kustomize.Kustomization)
if err != nil {
return errors.Format("%s: %w", msg, err)
}
path := filepath.Join(tempDir, "kustomization.yaml")
if err := os.WriteFile(path, data, 0666); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+path)
// Write the inputs
for _, input := range t.Inputs {
path := string(input)
if err := am.Save(tempDir, path); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.DebugContext(ctx, "wrote "+filepath.Join(tempDir, path))
}
// Execute kustomize
r, err := util.RunCmd(ctx, "kubectl", "kustomize", tempDir)
if err != nil {
kErr := r.Stderr.String()
err = errors.Format("%s: could not run kustomize: %w", msg, err)
log.ErrorContext(ctx, fmt.Sprintf("%s: stderr:\n%s", err.Error(), kErr), "err", err, "stderr", kErr)
return err
}
// Store the artifact
if err := am.Set(string(t.Output), r.Stdout.Bytes()); err != nil {
return errors.Format("%s: %w", msg, err)
}
log.Debug("set artifact " + string(t.Output))
return nil
}
func marshal(list []v1alpha4.Resource) (buf bytes.Buffer, err error) {
encoder := yaml.NewEncoder(&buf)
defer encoder.Close()
for _, item := range list {
if err = encoder.Encode(item); err != nil {
err = errors.Wrap(err)
return
}
}
return
}
// cacheChart stores a cached copy of Chart in the chart subdirectory of path.
//
// We assume the only method responsible for writing to chartDir is cacheChart
// itself. cacheChart runs concurrently when rendering a platform.
//
// We rely on the atomicity of moving temporary directories into place on the
// same filesystem via os.Rename. If a syscall.EEXIST error occurs during
// renaming, it indicates that the cached chart already exists, which is
// expected when this function is called concurrently.
//
// TODO(jeff): Break the dependency on v1alpha4, make it work across versions as
// a utility function.
func (b *BuildPlan) cacheChart(
ctx context.Context,
log *slog.Logger,
cacheDir string,
chart v1alpha4.Chart,
) error {
// Add repositories
repo := chart.Repository
if repo.URL == "" {
// repo update not needed for oci charts so this is debug instead of warn.
log.DebugContext(ctx, "skipped helm repo add and update: repo url is empty")
} else {
if r, err := util.RunCmd(ctx, "helm", "repo", "add", repo.Name, repo.URL); err != nil {
_, _ = io.Copy(b.Stderr, r.Stderr)
return errors.Format("could not run helm repo add: %w", err)
}
if r, err := util.RunCmd(ctx, "helm", "repo", "update", repo.Name); err != nil {
_, _ = io.Copy(b.Stderr, r.Stderr)
return errors.Format("could not run helm repo update: %w", err)
}
}
cacheTemp, err := os.MkdirTemp(cacheDir, chart.Name)
if err != nil {
return errors.Wrap(err)
}
defer util.Remove(ctx, cacheTemp)
cn := chart.Name
if chart.Repository.Name != "" {
cn = fmt.Sprintf("%s/%s", chart.Repository.Name, chart.Name)
}
helmOut, err := util.RunCmd(ctx, "helm", "pull", "--destination", cacheTemp, "--untar=true", "--version", chart.Version, cn)
if err != nil {
return errors.Wrap(fmt.Errorf("could not run helm pull: %w", err))
}
log.Debug("helm pull", "stdout", helmOut.Stdout, "stderr", helmOut.Stderr)
items, err := os.ReadDir(cacheTemp)
if err != nil {
return errors.Wrap(fmt.Errorf("could not read directory: %w", err))
}
if len(items) != 1 {
return errors.Format("want: exactly one item, have: %+v", items)
}
item := items[0]
src := filepath.Join(cacheTemp, item.Name())
dst := filepath.Join(cacheDir, chart.Name)
if err := os.Rename(src, dst); err != nil {
var linkErr *os.LinkError
if errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EEXIST) {
log.DebugContext(ctx, "cache already exists", "chart", chart.Name, "chart_version", chart.Version, "path", dst)
} else {
return errors.Wrap(fmt.Errorf("could not rename: %w", err))
}
} else {
log.DebugContext(ctx, fmt.Sprintf("renamed %s to %s", src, dst), "src", src, "dst", dst)
}
log.InfoContext(ctx,
fmt.Sprintf("cached %s %s", chart.Name, chart.Version),
"chart", chart.Name,
"chart_version", chart.Version,
"path", dst,
)
return nil
}
// onceWithLock obtains a filesystem lock with mkdir, then executes fn. If the
// lock is already locked, onceWithLock waits for it to be released then returns
// without calling fn.
func onceWithLock(log *slog.Logger, ctx context.Context, path string, fn func() error) error {
if err := os.MkdirAll(filepath.Dir(path), 0777); err != nil {
return errors.Wrap(err)
}
// Obtain a lock with a timeout.
lockDir := path + ".lock"
log = log.With("lock", lockDir)
err := os.Mkdir(lockDir, 0777)
if err == nil {
defer os.RemoveAll(lockDir)
log.DebugContext(ctx, fmt.Sprintf("acquired %s", lockDir))
if err := fn(); err != nil {
return errors.Wrap(err)
}
log.DebugContext(ctx, fmt.Sprintf("released %s", lockDir))
return nil
}
// Wait until the lock is released then return.
if os.IsExist(err) {
log.DebugContext(ctx, fmt.Sprintf("blocked %s", lockDir))
stillBlocked := time.After(5 * time.Second)
deadLocked := time.After(10 * time.Second)
for {
select {
case <-stillBlocked:
log.WarnContext(ctx, fmt.Sprintf("waiting for %s to be released", lockDir))
case <-deadLocked:
log.WarnContext(ctx, fmt.Sprintf("still waiting for %s to be released (dead lock?)", lockDir))
case <-time.After(100 * time.Millisecond):
if _, err := os.Stat(lockDir); os.IsNotExist(err) {
log.DebugContext(ctx, fmt.Sprintf("unblocked %s", lockDir))
return nil
}
case <-ctx.Done():
return errors.Wrap(ctx.Err())
}
}
}
// Unexpected error
return errors.Wrap(err)
}

View File

@@ -20,6 +20,7 @@ func makeBuildRunFunc(cfg *client.Config) command.RunFunc {
ctx := cmd.Root().Context()
logger.FromContext(ctx).DebugContext(ctx, "RunE", "args", args)
build := builder.New(builder.Entrypoints(args), builder.Cluster(cfg.Holos().ClusterName()))
//nolint:staticcheck
results, err := build.Run(ctx, cfg)
if err != nil {
return err

View File

@@ -1,11 +1,18 @@
package render
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"runtime"
"strings"
h "github.com/holos-run/holos"
"github.com/holos-run/holos/internal/artifact"
"github.com/holos-run/holos/internal/builder"
"github.com/holos-run/holos/internal/builder/v1alpha4"
"github.com/holos-run/holos/internal/cli/command"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
@@ -26,10 +33,10 @@ func New(cfg *holos.Config) *cobra.Command {
// New returns the component subcommand for the render command
func NewComponent(cfg *holos.Config) *cobra.Command {
cmd := command.New("component DIRECTORY [DIRECTORY...]")
cmd.Args = cobra.MinimumNArgs(1)
cmd := command.New("component DIRECTORY")
cmd.Args = cobra.ExactArgs(1)
cmd.Short = "render specific components"
cmd.Example = " holos render component --cluster-name=aws2 ./components/monitoring/kube-prometheus-stack"
cmd.Example = " holos render component --inject holos_cluster=aws2 ./components/monitoring/kube-prometheus-stack"
cmd.Flags().AddGoFlagSet(cfg.WriteFlagSet())
cmd.Flags().AddGoFlagSet(cfg.ClusterFlagSet())
@@ -38,46 +45,103 @@ func NewComponent(cfg *holos.Config) *cobra.Command {
cmd.PersistentFlags().AddGoFlagSet(config.TokenFlagSet())
flagSet := flag.NewFlagSet("", flag.ContinueOnError)
tagMap := make(tags)
cmd.PersistentFlags().VarP(&tagMap, "inject", "t", "set the value of a cue @tag field from a key=value pair")
var concurrency int
flagSet.IntVar(&concurrency, "concurrency", min(runtime.NumCPU(), 8), "number of concurrent build steps")
cmd.Flags().AddGoFlagSet(flagSet)
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
build := builder.New(builder.Entrypoints(args), builder.Cluster(cfg.ClusterName()))
build := builder.New(
builder.Entrypoints(args),
builder.Cluster(cfg.ClusterName()),
builder.Tags(tagMap.Tags()),
)
log := logger.FromContext(ctx)
results, err := build.Run(ctx, config)
log.DebugContext(ctx, "cue: building component instance")
bd, err := build.Unify(ctx, config)
if err != nil {
return errors.Wrap(err)
}
// TODO: Avoid accidental over-writes if two or more holos component
// instances result in the same file path. Write files into a blank
// temporary directory, error if a file exists, then move the directory into
// place.
var result Result
for _, result = range results {
log := logger.FromContext(ctx).With(
"cluster", cfg.ClusterName(),
"name", result.Name(),
)
if result.Continue() {
continue
typeMeta, err := bd.TypeMeta()
if err != nil {
return errors.Wrap(err)
}
if typeMeta.Kind != "BuildPlan" {
return errors.Format("invalid kind: want: BuildPlan have: %s", typeMeta.Kind)
}
log.DebugContext(ctx, "discriminated "+typeMeta.APIVersion+" "+typeMeta.Kind)
jsonBytes, err := bd.Value.MarshalJSON()
if err != nil {
return errors.Format("could not marshal json %s: %w", bd.Dir, err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
art := artifact.New()
switch version := typeMeta.APIVersion; version {
case "v1alpha4":
builder := v1alpha4.BuildPlan{
WriteTo: cfg.WriteTo(),
Concurrency: concurrency,
Stderr: cmd.ErrOrStderr(),
Path: h.InstancePath(args[0]),
}
// DeployFiles from the BuildPlan
if err := result.WriteDeployFiles(ctx, cfg.WriteTo()); err != nil {
if err := decoder.Decode(&builder.BuildPlan); err != nil {
return errors.Format("could not decode build plan %s: %w", bd.Dir, err)
}
return render.Component(ctx, &builder, art)
// Legacy method.
case "v1alpha3", "v1alpha2", "v1alpha1":
//nolint:staticcheck
results, err := build.Run(ctx, config)
if err != nil {
return errors.Wrap(err)
}
// API Objects
if result.SkipWriteAccumulatedOutput() {
log.DebugContext(ctx, "skipped writing k8s objects for "+result.Name())
} else {
path := result.Filename(cfg.WriteTo(), cfg.ClusterName())
if err := result.Save(ctx, path, result.AccumulatedOutput()); err != nil {
// TODO: Avoid accidental over-writes if two or more holos component
// instances result in the same file path. Write files into a blank
// temporary directory, error if a file exists, then move the directory into
// place.
var result Result
for _, result = range results {
log := logger.FromContext(ctx).With(
"cluster", cfg.ClusterName(),
"name", result.Name(),
)
if result.Continue() {
continue
}
// DeployFiles from the BuildPlan
if err := result.WriteDeployFiles(ctx, cfg.WriteTo()); err != nil {
return errors.Wrap(err)
}
// API Objects
if result.SkipWriteAccumulatedOutput() {
log.DebugContext(ctx, "skipped writing k8s objects for "+result.Name())
} else {
path := result.Filename(cfg.WriteTo(), cfg.ClusterName())
if err := result.Save(ctx, path, result.AccumulatedOutput()); err != nil {
return errors.Wrap(err)
}
}
log.InfoContext(ctx, "rendered "+result.Name(), "status", "ok", "action", "rendered")
}
log.InfoContext(ctx, "rendered "+result.Name(), "status", "ok", "action", "rendered")
default:
return errors.Format("component version not supported: %s", version)
}
return nil
}
return cmd
@@ -99,18 +163,91 @@ func NewPlatform(cfg *holos.Config) *cobra.Command {
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
build := builder.New(builder.Entrypoints(args))
log := logger.FromContext(ctx)
platform, err := build.Platform(ctx, config)
log.DebugContext(ctx, "cue: building platform instance")
bd, err := build.Unify(ctx, config)
if err != nil {
return errors.Wrap(err)
}
return render.Platform(ctx, concurrency, platform, cmd.ErrOrStderr())
tm, err := bd.TypeMeta()
if err != nil {
return errors.Wrap(err)
}
if tm.Kind != "Platform" {
return errors.Format("invalid kind: want: Platform have: %s", tm.Kind)
}
log.DebugContext(ctx, "discriminated "+tm.APIVersion+" "+tm.Kind)
jsonBytes, err := bd.Value.MarshalJSON()
if err != nil {
return errors.Format("could not marshal json %s: %w", bd.Dir, err)
}
decoder := json.NewDecoder(bytes.NewReader(jsonBytes))
decoder.DisallowUnknownFields()
switch version := tm.APIVersion; version {
case "v1alpha4":
builder := v1alpha4.Platform{
Concurrency: concurrency,
Stderr: cmd.ErrOrStderr(),
}
if err := decoder.Decode(&builder.Platform); err != nil {
return errors.Format("could not decode platform %s: %w", bd.Dir, err)
}
return render.Platform(ctx, &builder)
// Legacy versions prior to the render.Builder interface.
case "v1alpha3", "v1alpha2", "v1alpha1":
platform, err := build.Platform(ctx, config)
if err != nil {
return errors.Wrap(err)
}
//nolint:staticcheck
return render.LegacyPlatform(ctx, concurrency, platform, cmd.ErrOrStderr())
default:
return errors.Format("platform version not supported: %s", version)
}
}
return cmd
}
// tags represents a map of key values for CUE tags for flag parsing.
type tags map[string]string
func (t tags) Tags() []string {
parts := make([]string, 0, len(t))
for k, v := range t {
parts = append(parts, fmt.Sprintf("%s=%s", k, v))
}
return parts
}
func (t tags) String() string {
return strings.Join(t.Tags(), ",")
}
func (t tags) Set(value string) error {
for _, item := range strings.Split(value, ",") {
parts := strings.SplitN(item, "=", 2)
if len(parts) != 2 {
return errors.Format("invalid format, must be tag=value")
}
t[parts[0]] = parts[1]
}
return nil
}
func (t tags) Type() string {
return "strings"
}
// Deprecated: use render.Artifact instead.
type Result interface {
Continue() bool
Name() string

View File

@@ -13,6 +13,16 @@ import (
// ErrUnsupported is errors.ErrUnsupported
var ErrUnsupported = errors.ErrUnsupported
func NotImplemented() error {
return wrap(New("not implemented"), 2)
}
// Format calls fmt.Errorf(format, a...) then wraps the error with the source
// location of the caller.
func Format(format string, a ...any) error {
return wrap(fmt.Errorf(format, a...), 2)
}
// As calls errors.As
func As(err error, target any) bool {
return errors.As(err, target)
@@ -67,11 +77,7 @@ func (e *ErrorAt) Error() string {
return e.Source.Loc() + ": " + e.Err.Error()
}
// Wrap wraps err in a ErrorAt or returns err if err is nil, already a
// ErrorAt, or caller info is not available.
//
// XXX: Refactor to Err(error, ...slog.Attr). Often want to add attributes for the top level logger.
func Wrap(err error) error {
func wrap(err error, skip int) error {
// Nothing to do
if err == nil {
return nil
@@ -84,7 +90,7 @@ func Wrap(err error) error {
}
// Try to wrap err with caller info
if _, file, line, ok := runtime.Caller(1); ok {
if _, file, line, ok := runtime.Caller(skip); ok {
return &ErrorAt{
Err: err,
Source: Source{
@@ -97,6 +103,15 @@ func Wrap(err error) error {
return err
}
// Wrap wraps err in a ErrorAt or returns err if err is nil, already a
// ErrorAt, or caller info is not available.
//
// XXX: Refactor to Err(error, ...slog.Attr). Often want to add attributes for
// the top level logger.
func Wrap(err error) error {
return wrap(err, 2)
}
// Log logs err with Source location if Err is a ErrorAt
func Log(log *slog.Logger, ctx context.Context, level slog.Level, err error, msg string, args ...any) {
var errAt *ErrorAt

File diff suppressed because one or more lines are too long

View File

@@ -181,3 +181,74 @@ import (
// platform operators to define.
Domain: string & (string | *"holos.localhost")
}
// Organization represents organizational metadata useful across the platform.
#Organization: {
Name: string
DisplayName: string
Domain: string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
#OrganizationStrict: {
#Organization
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name: string & (=~"^[a-z][0-9a-z-]{1,61}[0-9a-z]$" & !~"--")
// DisplayName represents the human readable organization name.
DisplayName: string & (=~"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$" & !~" ")
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
#Projects: {[string]: #Project}
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
#Project: {
// Name represents project name.
Name: string
// Owner represents the team who own this project.
Owner: #Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces: {[string]: #Namespace} @go(,map[core.NameLabel]Namespace)
// Hostnames represents the host names to expose for this project.
Hostnames: {[string]: #Hostname} @go(,map[core.NameLabel]Hostname)
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
#Owner: {
Name: string
Email: string
}
// Namespace represents a Kubernetes namespace.
#Namespace: {
Name: string
}
// Hostname represents the left most dns label of a domain name.
#Hostname: {
// Name represents the subdomain to expose, e.g. "www"
Name: string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace: string
// Service represents the Service metadata.name field of backend object
// reference.
Service: string
// Port represents the Service port of the backend object reference.
Port: int
}

View File

@@ -0,0 +1,318 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/author/v1alpha4
// # Author API
//
// Package v1alpha4 contains ergonomic CUE definitions for Holos component
// authors. These definitions serve as adapters to produce [Core API] resources
// for the holos command line tool.
//
// [Core API]: https://holos.run/docs/api/core/v1alpha4/
package v1alpha4
import core "github.com/holos-run/holos/api/core/v1alpha4"
// Platform assembles a Core API [Platform] in the Resource field for the holos
// render platform command. Use the Components field to register components
// with the platform using a struct. This struct is converted into a list for
// final output to holos.
//
// See related:
//
// - [Component] collection of components composing the platform.
// - [Platform] resource assembled for holos to process.
//
// [Platform]: https://holos.run/docs/api/core/v1alpha4/#Platform
// [Component]: https://holos.run/docs/api/core/v1alpha4/#Component
#Platform: {
Name: string
Components: {[string]: core.#Component} @go(,map[NameLabel]core.Component)
Resource: core.#Platform
}
// Cluster represents a cluster managed by the Platform.
#Cluster: {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
name: string @go(Name)
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
primary: bool & (true | *false) @go(Primary)
}
// Fleet represents a named collection of similarly configured Clusters. Useful
// to segregate workload clusters from their management cluster.
#Fleet: {
name: string @go(Name)
// Clusters represents a mapping of Clusters by their name.
clusters: {[string]: #Cluster} & {[Name=_]: name: Name} @go(Clusters,map[string]Cluster)
}
// StandardFleets represents the standard set of Clusters in a Platform
// segmented into Fleets by their purpose. The management Fleet contains a
// single Cluster, for example a GKE autopilot cluster with no workloads
// deployed for reliability and cost efficiency. The workload Fleet contains
// all other Clusters which contain workloads and sync Secrets from the
// management cluster.
#StandardFleets: {
// Workload represents a Fleet of zero or more workload Clusters.
workload: #Fleet & {name: "workload"} @go(Workload)
// Management represents a Fleet with one Cluster named management.
management: #Fleet & {name: "management"} @go(Management)
}
// ArgoConfig represents the ArgoCD GitOps configuration associated with a
// [BuildPlan]. Useful to define once at the root of the Platform configuration
// and reuse across all components.
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
#ArgoConfig: {
// Enabled causes holos to render an Application resource when true.
Enabled: bool & (true | *false)
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL: string
// Root represents the path from the git repository root to the WriteTo output
// directory, the behavior of the holos render component --write-to flag and
// the Core API Component WriteTo field. Used as a prefix for the
// Application.spec.source.path field.
Root: string & (string | *"deploy")
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision: string & (string | *"main")
// AppProject represents the ArgoCD Project to associate the Application with.
AppProject: string & (string | *"default")
}
// Organization represents organizational metadata useful across the platform.
#Organization: {
Name: string
DisplayName: string
Domain: string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
#OrganizationStrict: {
#Organization
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name: string & (=~"^[a-z][0-9a-z-]{1,61}[0-9a-z]$" & !~"--")
// DisplayName represents the human readable organization name.
DisplayName: string & (=~"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$" & !~" ")
}
// Kubernetes provides a [BuildPlan] via the Output field which contains inline
// API Objects provided directly from CUE in the Resources field of
// [ComponentConfig].
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
#Kubernetes: {
#ComponentConfig
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan: core.#BuildPlan
}
// Helm provides a [BuildPlan] via the Output field which generates manifests
// from a helm chart with optional mix-in resources provided directly from CUE
// in the Resources field.
//
// This definition is a convenient way to produce a [BuildPlan] composed of
// three [Resources] generators with one [Kustomize] transformer.
//
// See related:
//
// - [ComponentConfig]
// - [Chart]
// - [Values]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Chart]: https://holos.run/docs/api/core/v1alpha4/#Chart
// [Values]: https://holos.run/docs/api/core/v1alpha4/#Values
#Helm: {
#ComponentConfig
// Chart represents a Helm chart.
Chart: core.#Chart
// Values represents data to marshal into a values.yaml for helm.
Values: core.#Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks: bool
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan: core.#BuildPlan
}
// Kustomize provides a [BuildPlan] via the Output field which generates
// manifests from a kustomize kustomization with optional mix-in resources
// provided directly from CUE in the Resources field.
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
#Kustomize: {
#ComponentConfig
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan: core.#BuildPlan
}
// ComponentConfig represents the configuration common to all kinds of
// component.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// See the following resources for additional details:
//
// - [Resources]
// - [ArgoConfig]
// - [KustomizeConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Resources]: https://holos.run/docs/api/core/v1alpha4/#Resources
#ComponentConfig: {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name: string
// Component represents the path to the component producing the BuildPlan.
Component: string
// Cluster represents the name of the cluster this BuildPlan is for.
Cluster: string
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources: core.#Resources
// ArgoConfig represents the ArgoCD GitOps configuration for this BuildPlan.
ArgoConfig: #ArgoConfig
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels: {[string]: string} @go(,map[string]string)
// Namespace manages the metadata.namespace field on all resources except the
// ArgoCD Application.
Namespace?: string
// KustomizeConfig represents the configuration for kustomize.
KustomizeConfig: #KustomizeConfig
}
// KustomizeConfig represents the configuration for kustomize post processing.
// The Files field is used to mixing in static manifest files from the component
// directory. The Resources field is used for mixing in manifests from network
// locations urls.
//
// See related:
//
// - [ComponentConfig]
// - [Kustomization]
//
// [Kustomization]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
#KustomizeConfig: {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization?: {...} @go(,map[string]any)
// Files represents files to copy from the component directory for kustomization.
Files: {[string]: Source: string} & {[NAME=_]: Source: NAME} @go(,map[string]struct{Source string})
// Resources represents additional entries to included in the resources list.
Resources: {[string]: Source: string} & {[NAME=_]: Source: NAME} @go(,map[string]struct{Source string})
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
#Projects: {[string]: #Project}
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
#Project: {
// Name represents project name.
Name: string
// Owner represents the team who own this project.
Owner: #Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces: {[string]: #Namespace} @go(,map[NameLabel]Namespace)
// Hostnames represents the host names to expose for this project.
Hostnames: {[string]: #Hostname} @go(,map[NameLabel]Hostname)
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels: {[string]: string} @go(,map[string]string)
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
#Owner: {
Name: string
Email: string
}
// Namespace represents a Kubernetes namespace.
#Namespace: {
Name: string
}
// Hostname represents the left most dns label of a domain name.
#Hostname: {
// Name represents the subdomain to expose, e.g. "www"
Name: string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace: string
// Service represents the Service metadata.name field of backend object
// reference.
Service: string
// Port represents the Service port of the backend object reference.
Port: int
}
// NameLabel signals the common use case of converting a struct to a list where
// the name field of each value unifies with the field name of the outer struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
#NameLabel: string

View File

@@ -18,6 +18,12 @@ import "google.golang.org/protobuf/types/known/structpb"
// output.
#InternalLabel: string
// NameLabel is a unique identifier useful to convert a CUE struct to a list
// when the values have a Name field with a default value. This type is
// intended to indicate the common use case of converting a struct to a list
// where the Name field of the value aligns with the struct field name.
#NameLabel: string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type
// checking.
#Kind: string

View File

@@ -0,0 +1,434 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/core/v1alpha4
// # Core API
//
// Package v1alpha4 contains the core API contract between the holos cli and CUE
// configuration code. Platform designers, operators, and software developers
// use this API to write configuration in CUE which holos loads. The Core API
// is declarative. Each resource represents a desired state necessary for holos
// to fully render Kubernetes manifests into plain files.
//
// The following resources provide important context for the Core API. The
// [Author API] is intended for component authors as a convenient adapter for
// the Core API resources Holos expects.
//
// 1. [Technical Overview]
// 2. [Quickstart]
// 3. [Author API]
//
// # Platform
//
// [Platform] defines the complete configuration of a platform. A platform
// represents a [Component] collection.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
//
// # Component
//
// A [Component] is the combination of CUE code along one path relative to the
// platform root directory plus data injected from the [PlatformSpec] via CUE tags.
// The platform configuration root is the directory containing cue.mod.
//
// A [Component] always produces exactly one [BuildPlan].
//
// # BuildPlan
//
// A [BuildPlan] contains an [Artifact] collection. A BuildPlan often produces
// two artifacts, one containing the fully rendered Kubernetes API resources,
// the other containing an additional resource to manage the former with GitOps.
// For example, a BuildPlan for a podinfo component produces a manifest
// containing a Deployment and a Service, along with a second manifest
// containing an ArgoCD Application.
//
// Inspect a BuildPlan resource holos render component would process by executing:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// # Artifact
//
// An [Artifact] is one fully rendered manifest file produced from the final
// [Transformer] in a sequence of transformers. An Artifact may also be
// produced directly from a [Generator], but this use case is uncommon.
//
// # Transformer
//
// A [Transformer] takes multiple inputs from prior [Generator] or [Transformer]
// outputs, then transforms the data into one output. [Kustomize] is the most
// commonly used transformer, though a simple [Join] is also supported.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// # Generators
//
// A [Generator] generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
// [Author API]: https://holos.run/docs/api/author/
// [Quickstart]: https://holos.run/docs/quickstart/
// [Technical Overview]: https://holos.run/docs/technical-overview/
package v1alpha4
// BuildPlan represents a build plan for holos to execute. Each [Platform]
// component produces exactly one BuildPlan.
//
// One or more [Artifact] files are produced by a BuildPlan, representing the
// fully rendered manifests for the Kubernetes API Server.
//
// # Example BuildPlan
//
// Command:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// Output:
//
// kind: BuildPlan
// apiVersion: v1alpha4
// metadata:
// name: dev-namespaces
// spec:
// component: projects/platform/components/namespaces
// artifacts:
// - artifact: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// generators:
// - kind: Resources
// output: resources.gen.yaml
// resources:
// Namespace:
// dev-jeff:
// metadata:
// name: dev-jeff
// labels:
// kubernetes.io/metadata.name: dev-jeff
// kind: Namespace
// apiVersion: v1
// transformers:
// - kind: Kustomize
// inputs:
// - resources.gen.yaml
// output: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// kustomize:
// kustomization:
// commonLabels:
// holos.run/component.name: dev-namespaces
// resources:
// - resources.gen.yaml
#BuildPlan: {
// Kind represents the type of the resource.
kind: string & "BuildPlan" @go(Kind)
// APIVersion represents the versioned schema of the resource.
apiVersion: string & (string | *"v1alpha4") @go(APIVersion)
// Metadata represents data about the resource such as the Name.
metadata: #Metadata @go(Metadata)
// Spec specifies the desired state of the resource.
spec: #BuildPlanSpec @go(Spec)
}
// BuildPlanSpec represents the specification of the [BuildPlan].
#BuildPlanSpec: {
// Component represents the component that produced the build plan.
// Represented as a path relative to the platform root.
component: string @go(Component)
// Disabled causes the holos cli to disregard the build plan.
disabled?: bool @go(Disabled)
// Artifacts represents the artifacts for holos to build.
artifacts: [...#Artifact] @go(Artifacts,[]Artifact)
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
#Artifact: {
artifact?: #FilePath @go(Artifact)
generators?: [...#Generator] @go(Generators,[]Generator)
transformers?: [...#Transformer] @go(Transformers,[]Transformer)
skip?: bool @go(Skip)
}
// Generator generates an intermediate manifest for a [Artifact].
//
// Each Generator in a [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// Refer to [Resources], [Helm], and [File].
#Generator: {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
kind: string & ("Resources" | "Helm" | "File") @go(Kind)
// Output represents a file for a Transformer or Artifact to consume.
output: #FilePath @go(Output)
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
resources?: #Resources @go(Resources)
// Helm generator. Ignored unless kind is Helm.
helm?: #Helm @go(Helm)
// File generator. Ignored unless kind is File.
file?: #File @go(File)
}
// Resource represents one kubernetes api object.
#Resource: {...}
// Resources represents a kubernetes resources [Generator] from CUE.
#Resources: {[string]: [string]: #Resource}
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
#File: {
// Source represents a file sub-path relative to the component path.
source: #FilePath @go(Source)
}
// Helm represents a [Chart] manifest [Generator].
#Helm: {
// Chart represents a helm chart to manage.
chart: #Chart @go(Chart)
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
values: #Values @go(Values)
// EnableHooks enables helm hooks when executing the `helm template` command.
enableHooks?: bool @go(EnableHooks)
// Namespace represents the helm namespace flag
namespace?: string @go(Namespace)
}
// Values represents [Helm] Chart values generated from CUE.
#Values: {...}
// Chart represents a [Helm] Chart.
#Chart: {
// Name represents the chart name.
name: string @go(Name)
// Version represents the chart version.
version: string @go(Version)
// Release represents the chart release when executing helm template.
release: string @go(Release)
// Repository represents the repository to fetch the chart from.
repository?: #Repository @go(Repository)
}
// Repository represents a [Helm] [Chart] repository.
#Repository: {
name: string @go(Name)
url: string @go(URL)
}
// Transformer transforms [Generator] manifests within a [Artifact].
#Transformer: {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
kind: string & ("Kustomize" | "Join") @go(Kind)
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
inputs: [...#FilePath] @go(Inputs,[]FilePath)
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
output: #FilePath @go(Output)
// Kustomize transformer. Ignored unless kind is Kustomize.
kustomize?: #Kustomize @go(Kustomize)
// Join transformer. Ignored unless kind is Join.
join?: #Join @go(Join)
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
#Join: {
separator: string & (string | *"---\n") @go(Separator)
}
// Kustomize represents a kustomization [Transformer].
#Kustomize: {
// Kustomization represents the decoded kustomization.yaml file
kustomization: #Kustomization @go(Kustomization)
// Files holds file contents for kustomize, e.g. patch files.
files?: #FileContentMap @go(Files)
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
#Kustomization: {...}
// FileContent represents file contents.
#FileContent: string
// FileContentMap represents a mapping of file paths to file contents.
#FileContentMap: {[string]: #FileContent}
// FilePath represents a file path.
#FilePath: string
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
#InternalLabel: string
// Kind is a discriminator. Defined as a type for clarity and type checking.
#Kind: string
// NameLabel is a unique identifier useful to convert a CUE struct to a list
// when the values have a Name field with a default value. NameLabel indicates
// the common use case of converting a struct to a list where the Name field of
// the value aligns with the outer struct field name.
//
// For example:
//
// Outer: [NAME=_]: Name: NAME
#NameLabel: string
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
#Platform: {
// Kind is a string value representing the resource.
kind: string & "Platform" @go(Kind)
// APIVersion represents the versioned schema of this resource.
apiVersion: string & (string | *"v1alpha4") @go(APIVersion)
// Metadata represents data about the resource such as the Name.
metadata: #Metadata @go(Metadata)
// Spec represents the specification.
spec: #PlatformSpec @go(Spec)
}
// Metadata represents data about the resource such as the Name.
#Metadata: {
// Name represents the resource name.
name: string @go(Name)
}
// PlatformSpec represents the specification of a [Platform]. Think of a
// platform spec as a [Component] collection for multiple kubernetes clusters
// combined with the user-specified Platform Model.
#PlatformSpec: {
// Components represents a list of holos components to manage.
components: [...#Component] @go(Components,[]Component)
}
// Component represents the complete context necessary to produce a [BuildPlan].
// Component carries information injected from holos render platform to holos
// render component to produce each [BuildPlan].
//
// All of these fields are passed to the holos render component command using
// flags, which in turn are injected to CUE using tags. For clarity, CUE field
// and tag names should match the struct json tag names below.
#Component: {
// Name represents the name of the component. Injected as the tag variable
// "holos_name" to set the BuildPlan metadata.name field. Necessary for clear
// user feedback during platform rendering.
name: string @go(Name)
// Component represents the path of the component relative to the platform
// root. Injected as the tag variable "holos_component".
component: string @go(Component)
// Cluster is the cluster name to provide when rendering the component.
// Injected as the tag variable "holos_cluster".
cluster: string @go(Cluster)
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
// Injected as the tag "holos_model".
model: {...} @go(Model,map[string]any)
// Tags represents cue @tag variables injected into the holos render component
// command from the holos render platform command. Tags with a "holos_"
// prefix are reserved for use by the Holos Authors.
tags?: {[string]: string} @go(Tags,map[string]string)
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
writeTo?: string @go(WriteTo)
}
// Tags represents standardized fields injected into the component [BuildPlan]
// from the [Platform].
//
// Note, tags should have a reasonable default value to easily use cue eval and
// cue export without needing to make a bunch of decisions about tag values.
//
// Example:
//
// import core "github.com/holos-run/holos/api/core/v1alpha4"
// _Tags: core.#Tags & {
// cluster: _ @tag(cluster, type=string)
// environment: _ @tag(environment, type=string)
// component: _ @tag(component, type=string)
// name: _ @tag(name, type=string)
// }
#Tags: {
// Name represents the BuildPlan metadata.name field injected from the Platform.
name: string & (string | *"no-name") @go(Name)
// Cluster represents the cluster name injected from
cluster: string & (string | *"no-cluster") @go(Cluster)
// Environment represents the build plan environment.
environment: string & (string | *"no-environment") @go(Environment)
// Component represents the path of the component relative to the platform root.
component: string & (string | *"no-component") @go(Component)
}

View File

@@ -0,0 +1,349 @@
package v1alpha4
import (
ks "sigs.k8s.io/kustomize/api/types"
app "argoproj.io/application/v1alpha1"
core "github.com/holos-run/holos/api/core/v1alpha4"
)
#Platform: {
Name: string | *"no-platform-name"
Components: _
Resource: {
metadata: name: Name
spec: components: [for x in Components {x}]
}
}
// https://holos.run/docs/api/author/v1alpha4/#Kubernetes
#Kubernetes: {
Name: _
Component: _
Cluster: _
Resources: _
ArgoConfig: _
CommonLabels: _
Namespace?: _
KustomizeConfig: {
Files: _
Resources: _
Kustomization: ks.#Kustomization & {
apiVersion: "kustomize.config.k8s.io/v1beta1"
kind: "Kustomization"
}
}
// Kustomize to add custom labels and manage the namespace. More advanced
// functionality than this should use the Core API directly and propose
// extending the Author API if the need is common.
_TransformerArgo: core.#Transformer & {
kind: "Kustomize"
kustomize: kustomization: ks.#Kustomization & {
commonLabels: "holos.run/component.name": BuildPlan.metadata.name
commonLabels: CommonLabels
}
}
// Add the argocd.argoproj.io/instance label to resources, but not to the
// argocd Application config.
_Transformer: _TransformerArgo & {
kustomize: kustomization: commonLabels: {
"argocd.argoproj.io/instance": Name
}
}
_Artifacts: {
component: {
_path: "clusters/\(Cluster)/components/\(Name)"
artifact: "\(_path)/\(Name).gen.yaml"
let ResourcesOutput = "resources.gen.yaml"
let IntermediateOutput = "combined.gen.yaml"
generators: [{
kind: "Resources"
output: ResourcesOutput
resources: Resources
}]
transformers: [
core.#Transformer & {
kind: "Kustomize"
inputs: [ResourcesOutput]
output: IntermediateOutput
kustomize: kustomization: KustomizeConfig.Kustomization & {
resources: inputs
}
},
_Transformer & {
inputs: [IntermediateOutput]
output: artifact
kustomize: kustomization: resources: inputs
if Namespace != _|_ {
kustomize: kustomization: namespace: Namespace
}
},
]
}
// Mix in the ArgoCD Application gitops artifact.
(#ArgoArtifact & {
name: Name
cluster: Cluster
config: ArgoConfig
transformer: _TransformerArgo
component: _Artifacts.component._path
}).Artifact
}
BuildPlan: {
metadata: name: Name
spec: component: Component
spec: artifacts: [for x in _Artifacts {x}]
}
}
// https://holos.run/docs/api/author/v1alpha4/#Kustomize
#Kustomize: {
Name: _
Component: _
Cluster: _
Resources: _
ArgoConfig: _
CommonLabels: _
Namespace?: _
KustomizeConfig: {
Files: _
Resources: _
Kustomization: ks.#Kustomization & {
apiVersion: "kustomize.config.k8s.io/v1beta1"
kind: "Kustomization"
}
}
// Kustomize to add custom labels and manage the namespace. More advanced
// functionality than this should use the Core API directly and propose
// extending the Author API if the need is common.
_TransformerArgo: core.#Transformer & {
kind: "Kustomize"
kustomize: kustomization: ks.#Kustomization & {
commonLabels: "holos.run/component.name": BuildPlan.metadata.name
commonLabels: CommonLabels
}
}
// Add the argocd.argoproj.io/instance label to resources, but not to the
// argocd Application config.
_Transformer: _TransformerArgo & {
kustomize: kustomization: commonLabels: {
"argocd.argoproj.io/instance": Name
}
}
_Artifacts: {
component: {
_path: "clusters/\(Cluster)/components/\(Name)"
artifact: "\(_path)/\(Name).gen.yaml"
generators: [
{
kind: "Resources"
output: "resources.gen.yaml"
resources: Resources
},
for x in KustomizeConfig.Files {
{
kind: "File"
file: source: x.Source
output: file.source
}
},
]
let Intermediate = "intermediate.gen.yaml"
transformers: [
core.#Transformer & {
kind: "Kustomize"
inputs: [for x in generators {x.output}]
output: Intermediate
kustomize: kustomization: KustomizeConfig.Kustomization & {
resources: [
for x in inputs {x},
for x in KustomizeConfig.Resources {x.Source},
]
}
},
_Transformer & {
inputs: [Intermediate]
output: artifact
kustomize: kustomization: resources: inputs
if Namespace != _|_ {
kustomize: kustomization: namespace: Namespace
}
},
]
}
// Mix in the ArgoCD Application gitops artifact.
(#ArgoArtifact & {
name: Name
cluster: Cluster
config: ArgoConfig
transformer: _TransformerArgo
component: _Artifacts.component._path
}).Artifact
}
BuildPlan: {
metadata: name: Name
spec: component: Component
spec: artifacts: [for x in _Artifacts {x}]
}
}
// https://holos.run/docs/api/author/v1alpha4/#Helm
#Helm: {
Name: _
Component: _
Cluster: _
Resources: _
ArgoConfig: _
CommonLabels: _
Namespace?: _
Chart: {
name: string | *Name
release: string | *name
}
Values: _
EnableHooks: true | *false
KustomizeConfig: {
Files: _
Resources: _
Kustomization: ks.#Kustomization & {
apiVersion: "kustomize.config.k8s.io/v1beta1"
kind: "Kustomization"
}
}
// Kustomize to add custom labels and manage the namespace. More advanced
// functionality than this should use the Core API directly and propose
// extending the Author API if the need is common.
_TransformerArgo: core.#Transformer & {
kind: "Kustomize"
kustomize: kustomization: ks.#Kustomization & {
commonLabels: "holos.run/component.name": BuildPlan.metadata.name
commonLabels: CommonLabels
}
}
// Add the argocd.argoproj.io/instance label to resources, but not to the
// argocd Application config.
_Transformer: _TransformerArgo & {
kustomize: kustomization: commonLabels: {
"argocd.argoproj.io/instance": Name
}
}
_Artifacts: {
component: {
_path: "clusters/\(Cluster)/components/\(Name)"
artifact: "\(_path)/\(Name).gen.yaml"
let HelmOutput = "helm.gen.yaml"
let ResourcesOutput = "resources.gen.yaml"
let IntermediateOutput = "combined.gen.yaml"
generators: [
{
kind: "Helm"
output: HelmOutput
helm: core.#Helm & {
chart: Chart
values: Values
enableHooks: EnableHooks
if Namespace != _|_ {
namespace: Namespace
}
}
},
{
kind: "Resources"
output: ResourcesOutput
resources: Resources
},
]
transformers: [
core.#Transformer & {
kind: "Kustomize"
inputs: [HelmOutput, ResourcesOutput]
output: IntermediateOutput
kustomize: kustomization: KustomizeConfig.Kustomization & {
resources: inputs
}
},
_Transformer & {
inputs: [IntermediateOutput]
output: artifact
kustomize: kustomization: resources: inputs
if Namespace != _|_ {
kustomize: kustomization: namespace: Namespace
}
},
]
}
// Mix in the ArgoCD Application gitops artifact.
(#ArgoArtifact & {
name: Name
cluster: Cluster
config: ArgoConfig
transformer: _TransformerArgo
component: _Artifacts.component._path
}).Artifact
}
BuildPlan: {
metadata: name: Name
spec: component: Component
spec: artifacts: [for x in _Artifacts {x}]
}
}
#ArgoArtifact: {
name: string
let Name = name
cluster: string
let Cluster = cluster
config: #ArgoConfig
let ArgoConfig = config
transformer: core.#Transformer
component: string
Artifact: {}
if ArgoConfig.Enabled {
Artifact: {
argocd: core.#Artifact & {
artifact: "clusters/\(Cluster)/gitops/\(Name).gen.yaml"
generators: [{
kind: "Resources"
output: "application.gen.yaml"
resources: Application: (Name): app.#Application & {
metadata: name: Name
metadata: namespace: string | *"argocd"
spec: {
destination: server: string | *"https://kubernetes.default.svc"
project: ArgoConfig.AppProject
source: {
repoURL: ArgoConfig.RepoURL
path: "\(ArgoConfig.Root)/\(component)"
targetRevision: ArgoConfig.TargetRevision
}
}
}
}]
transformers: [transformer & {
inputs: [for x in generators {x.output}]
output: artifact
kustomize: kustomization: resources: inputs
}]
}
}
}
}

View File

@@ -0,0 +1,15 @@
package v1alpha4
#Component: tags?: [_]: !~"^holos_"
#Transformer: {
kind: _
if kind == "Kustomize" {
kustomize: _
}
if kind == "Join" {
join: _
}
}

View File

@@ -2,8 +2,13 @@ package holos
import api "github.com/holos-run/holos/api/author/v1alpha3"
#Platform: api.#Platform
#Fleets: api.#StandardFleets
// Define the default organization name
#Organization: DisplayName: string | *"Bank of Holos"
#Organization: Name: string | *"bank-of-holos"
#Organization: api.#OrganizationStrict
#Platform: api.#Platform
#Fleets: api.#StandardFleets
_ComponentConfig: {
Resources: #Resources

View File

@@ -0,0 +1,9 @@
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
vendor/
node_modules/

View File

@@ -0,0 +1 @@
{}

View File

@@ -0,0 +1,10 @@
package holos
import api "github.com/holos-run/holos/api/author/v1alpha4"
#Platform: api.#Platform & {
Name: "guide"
}
// Render a Platform resource for holos to process
#Platform.Resource

View File

@@ -0,0 +1,49 @@
package holos
import (
corev1 "k8s.io/api/core/v1"
appsv1 "k8s.io/api/apps/v1"
rbacv1 "k8s.io/api/rbac/v1"
batchv1 "k8s.io/api/batch/v1"
ci "cert-manager.io/clusterissuer/v1"
rgv1 "gateway.networking.k8s.io/referencegrant/v1beta1"
certv1 "cert-manager.io/certificate/v1"
hrv1 "gateway.networking.k8s.io/httproute/v1"
gwv1 "gateway.networking.k8s.io/gateway/v1"
ap "argoproj.io/appproject/v1alpha1"
es "external-secrets.io/externalsecret/v1beta1"
ss "external-secrets.io/secretstore/v1beta1"
)
#Resources: {
[Kind=string]: [InternalLabel=string]: {
kind: Kind
metadata: name: string | *InternalLabel
}
AppProject?: [_]: ap.#AppProject
Certificate?: [_]: certv1.#Certificate
ClusterIssuer?: [_]: ci.#ClusterIssuer
ClusterRole?: [_]: rbacv1.#ClusterRole
ClusterRoleBinding?: [_]: rbacv1.#ClusterRoleBinding
ConfigMap?: [_]: corev1.#ConfigMap
CronJob?: [_]: batchv1.#CronJob
Deployment?: [_]: appsv1.#Deployment
ExternalSecret?: [_]: es.#ExternalSecret
HTTPRoute?: [_]: hrv1.#HTTPRoute
Job?: [_]: batchv1.#Job
Namespace?: [_]: corev1.#Namespace
ReferenceGrant?: [_]: rgv1.#ReferenceGrant
Role?: [_]: rbacv1.#Role
RoleBinding?: [_]: rbacv1.#RoleBinding
Secret?: [_]: corev1.#Secret
SecretStore?: [_]: ss.#SecretStore
Service?: [_]: corev1.#Service
ServiceAccount?: [_]: corev1.#ServiceAccount
StatefulSet?: [_]: appsv1.#StatefulSet
Gateway?: [_]: gwv1.#Gateway & {
spec: gatewayClassName: string | *"istio"
}
}

View File

@@ -0,0 +1,57 @@
package holos
import api "github.com/holos-run/holos/api/author/v1alpha4"
// Manage a workload cluster named workload for use with the guides.
#Fleets: api.#StandardFleets
// Define the default organization name.
#Organization: api.#OrganizationStrict & {
DisplayName: string | *"Bank of Holos"
Name: string | *"bank-of-holos"
Domain: string | *"holos.localhost"
}
// Projects represents a way to organize components into projects with owners.
// https://holos.run/docs/api/author/v1alpha4/#Projects
#Projects: api.#Projects
// ArgoConfig represents the configuration of ArgoCD Application resources for
// each component.
// https://holos.run/docs/api/author/v1alpha4/#ArgoConfig
#ArgoConfig: api.#ArgoConfig
#ComponentConfig: api.#ComponentConfig & {
Name: _Tags.name
Component: _Tags.component
Cluster: _Tags.cluster
ArgoConfig: #ArgoConfig & {
if _Tags.project != "no-project" {
AppProject: _Tags.project
}
}
Resources: #Resources
// Mix in project labels if the project is defined by the platform.
if _Tags.project != "no-project" {
CommonLabels: #Projects[_Tags.project].CommonLabels
}
}
// https://holos.run/docs/api/author/v1alpha4/#Kubernetes
#Kubernetes: close({
#ComponentConfig
api.#Kubernetes
})
// https://holos.run/docs/api/author/v1alpha4/#Kustomize
#Kustomize: close({
#ComponentConfig
api.#Kustomize
})
// https://holos.run/docs/api/author/v1alpha4/#Helm
#Helm: close({
#ComponentConfig
api.#Helm
})

View File

@@ -0,0 +1,24 @@
package holos
import (
"encoding/json"
dto "github.com/holos-run/holos/service/gen/holos/object/v1alpha1:object"
)
// Note, tags should have a reasonable default value to easily use cue eval and
// cue export without needing to make a bunch of decisions about tag values.
// _PlatformConfig represents all of the data passed from holos to cue, used to
// carry the platform and project models.
_PlatformConfig: dto.#PlatformConfig & json.Unmarshal(_PlatformConfigJSON)
_PlatformConfigJSON: string | *"{}" @tag(holos_platform_config, type=string)
_Tags: {
// Reserved tags
name: string | *"no-name" @tag(holos_name, type=string)
cluster: string | *"no-cluster" @tag(holos_cluster, type=string)
component: string | *"no-component" @tag(holos_component, type=string)
// User defined tags
environment: string | *"no-environment" @tag(environment, type=string)
project: string | *"no-project" @tag(project, type=string)
}

View File

@@ -89,7 +89,7 @@ func New(opts ...Option) *Config {
ServerConfig: &ServerConfig{},
}
writeFlagSet.StringVar(&cfg.writeTo, "write-to", cfg.writeTo, "write to directory")
clusterFlagSet.StringVar(&cfg.clusterName, "cluster-name", cfg.clusterName, "cluster name")
clusterFlagSet.StringVar(&cfg.clusterName, "cluster-name", cfg.clusterName, "cluster name (deprecated: use --inject holos_cluster=example instead)")
kvDefault := ""
if home := homedir.HomeDir(); home != "" {
kvDefault = filepath.Join(home, ".holos", "kubeconfig.provisioner")

View File

@@ -29,12 +29,12 @@ var validLogFormats = []string{"text", "json", "console"}
// stringSlice is a comma separated list of string values
type stringSlice []string
func (s *stringSlice) String() string {
return strings.Join((*s)[:], ",")
func (s stringSlice) String() string {
return strings.Join((s)[:], ",")
}
func (s *stringSlice) Set(value string) error {
*s = append(*s, strings.Split(value, ",")...)
func (s stringSlice) Set(value string) error {
_ = append(s, strings.Split(value, ",")...)
return nil
}
@@ -157,7 +157,7 @@ func (c *Config) handler(w io.Writer) (h slog.Handler) {
// NewLogger returns a *slog.Logs configured by c *Config which writes to w
func (c *Config) NewLogger(w io.Writer) *slog.Logger {
return slog.New(c.handler(w)).With("version", version.Version)
return slog.New(c.handler(w)).With("version", version.Version, "pid", os.Getpid())
}
// NewConfig returns a new logging Config struct

View File

@@ -0,0 +1,27 @@
package render
import (
"context"
"github.com/holos-run/holos"
"github.com/holos-run/holos/internal/artifact"
"github.com/holos-run/holos/internal/errors"
)
// Platform renders a platform, writing fully rendered manifests to files.
func Platform(ctx context.Context, builder holos.Builder) error {
// Artifacts are currently written by each `holos render component`
// subprocess, not the parent `holos render platform` process.
if err := builder.Build(ctx, artifact.New()); err != nil {
return errors.Wrap(err)
}
return nil
}
// Component renders a component writing fully rendered manifests to files.
func Component(ctx context.Context, builder holos.Builder, am holos.ArtifactMap) error {
if err := builder.Build(ctx, am); err != nil {
return errors.Wrap(err)
}
return nil
}

View File

@@ -14,7 +14,8 @@ import (
"golang.org/x/sync/errgroup"
)
func Platform(ctx context.Context, concurrency int, pf *core.Platform, stderr io.Writer) error {
// Deprecated: Use Platform instead.
func LegacyPlatform(ctx context.Context, concurrency int, pf *core.Platform, stderr io.Writer) error {
parentStart := time.Now()
total := len(pf.Spec.Components)

View File

@@ -4,5 +4,5 @@ deps:
- remote: buf.build
owner: bufbuild
repository: protovalidate
commit: a6c49f84cc0f4e038680d390392e2ab0
digest: shake256:3deb629c655e469d87c58babcfbed403275a741fb4a269366c4fd6ea9db012cf562a1e64819508d73670c506f96d01f724c43bc97b44e2e02aa6e8bbdd160ab2
commit: 5a7b106cbb87462d9a8c9ffecdbd2e38
digest: shake256:2f7efa5a904668219f039d4f6eeb51e871f8f7f5966055a10663cba335bd65f76cac84da3fa758ab7b5dcb489ec599521390ce3951d119fb56df1fc2def16bb0

View File

@@ -1 +1 @@
95
97

View File

@@ -1 +1 @@
2
0