Compare commits

...

188 Commits

Author SHA1 Message Date
Jeff McCune
8d2984a635 try release 2024-11-14 15:15:27 -07:00
Jeff McCune
032f72b435 render: log helm pull errors (#332)
Previously errors were not logged, giving no indication what went wrong.
This patch changes the error handler to log errors from helm.
2024-11-14 09:44:27 -07:00
Jeff McCune
2380223794 docs: add argocd application example (#340)
When we moved from v1alpha4 to v1alpha5 we removed ArgoConfig from the
author schema.  There was no longer a clear example of how to configure
an ArgoCD Application for every component in v1alpha5.

This patch adds a topic document with an example of how to add an
Application along side the resources by mixing an additional Artifact
into the BuildPlan.
2024-11-13 16:30:59 -07:00
Jeff McCune
e6892c3b16 v0.99.0 2024-11-13 12:49:28 -07:00
Jeff McCune
847fd2958e helm: add support for helm template --kube-version capabilities (#330)
Previously the Helm generator had no support for the --kube-version
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:43:01 -07:00
Jeff McCune
cf622835db helm: add support for helm template --api-versions capabilities (#330)
Previously the Helm generator had no support for the --api-versions
flag.  This is a problem for helm charts that conditionally render
resources based on this capability.

This patch plumbs support through the author and core schemas with a new
field similar to how the enable hooks field is handled.
2024-11-13 12:42:50 -07:00
Jeff McCune
1f5dc3a082 docs: add note about tested helm version (#335)
To help users understand what should definitely work.
2024-11-13 09:45:56 -07:00
Jeff McCune
9f4da68dc9 v0.98.2 2024-11-13 09:19:30 -07:00
Jeff McCune
2ee056be9f cue: fix panic with no args (#334)
Fixes:

```
❯ holos
panic: runtime error: slice bounds out of range [2:1]

goroutine 1 [running]:
github.com/holos-run/holos/internal/cli.newCueCmd(...)
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:121
github.com/holos-run/holos/internal/cli.New(0xc0002837c0, {0x3826e00, 0x4f60860})
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/root.go:102 +0x772
main.main.MakeMain.func1()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/internal/cli/main.go:22 +0x5b
main.main()
       /home/mike/go/pkg/mod/github.com/holos-run/holos@v0.98.1/cmd/holos/main.go:10 +0x3e
```
2024-11-13 09:04:37 -07:00
Jeff McCune
394e2cb0b2 docs: add cue tutorial (#318)
Show how to use the ComponentConfig Resources field to mix in resources.
2024-11-13 08:00:37 -07:00
Jeff McCune
cf95c9664d docs: change hello holos parameters to greeting (#328)
Version doesn't make as much sense since we're doing a hello world
tutorial.

Also consolidate the values information into one step, and consolidate
the breaking it down section to make it shorter and clearer.
2024-11-12 09:46:19 -07:00
Jeff McCune
0192eeeb7e docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest

This time in the correct directory.
2024-11-11 17:25:17 -07:00
Jeff McCune
ed54bcc58f docs: rename cue-generator to cue
The main use case is to manage resources from CUE, but CUE has many uses
in Holos such as validation and driving Kustomize.
2024-11-11 17:16:53 -07:00
Jeff McCune
9ac7f185f9 docs: fix broken validators link in diagram 2024-11-11 17:11:35 -07:00
Jeff McCune
7de72d3dab docs: add component parameters example to hello holos (#328)
The important note was weird because we didn't show an example of how to
use component parameters.  This patch replaces the note with an example.
2024-11-11 16:56:16 -07:00
Jeff McCune
2e3c998454 docs: add directory tree to hello holos doc (#324)
Feedback from Zack, give a tree so people skimming know where to figure
out the lay of the land.
2024-11-11 16:19:48 -07:00
Jeff McCune
580afffa7f 0.98.1 - holos init platform 2024-11-11 14:44:00 -07:00
Jeff McCune
67535e1e1d doc: remove init platform from setup guide
We have it in the hello guide, setup should just be install only and not
how to use the tool yet.
2024-11-11 14:41:58 -07:00
Nate McCurdy
767ea69d2e docs: Add a tree view to Hello Holos
A tree view of the `holos-tutorial/` directory should give readers a
quick, high-level understanding of the folder structure of a typical
Holos platform project.
2024-11-11 14:04:40 -07:00
Jeff McCune
21e1a116e4 cli: hide help flags and command (#325)
They're unnecessary.
2024-11-11 14:02:25 -07:00
Jeff McCune
65fe7779be cli: rename generate to init (#325)
This patch changes the `holos generate` command to `holos init` to match
other tools like `go mod init`.
2024-11-11 14:02:25 -07:00
Jeff McCune
0e7abf0173 docs: consolidate diagrams to @site/src/diagrams/
So we don't have two different copies in two different places.
2024-11-11 13:40:46 -07:00
Jeff McCune
cca022ac99 docs: move architecture diagrams (#323) 2024-11-11 12:03:21 -07:00
Jeff McCune
43e939d06a doc: refactor breaking it down table in hello holos
So it displays nicely on mobile.
2024-11-09 14:52:11 -08:00
Jeff McCune
8096268826 docs: fix diagram urls again 2024-11-09 14:40:08 -08:00
Jeff McCune
631b23091d docs: fix rendering overview diagram on blog 2024-11-09 14:33:54 -08:00
Jeff McCune
09c6476282 docs: upgrade docusaurus to 3.6.1
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-09 14:30:44 -08:00
Jeff McCune
a768d16c5f docs: set current version to v1alpha5
Previously the current version would always be unreleased at /docs/next
and we'd have to copy the doc/md/ folder into the
doc/website/versioned_docs/version-v1alpha5/ every time we made a
change.

We're going to be working on v1alpha5 for awhile so it makes sense to
just have the current version published at /docs/v1alpha5/ and we can
start /docs/v1alpha6/ whenever we're ready.

This also has the nice effect of giving us permalinks if we change the
structure again.  /docs/v1alpha5/ will remain over time.
2024-11-09 14:29:27 -08:00
Jeff McCune
3834a7ef85 docs: add missing link to kustomize tutorial 2024-11-08 22:25:34 -08:00
Jeff McCune
606a1aae73 docs: add nav bar title back 2024-11-08 19:29:05 -08:00
Jeff McCune
340d07ee7a docs: fix announcing holos blog (#321) 2024-11-08 19:24:46 -08:00
Jeff McCune
12d2cec4d5 docs: fix rendering overview diagram links (#321) 2024-11-08 19:12:23 -08:00
Jeff McCune
e93feb49b7 docs: add version drop down to nav bar (#321) 2024-11-08 17:05:37 -08:00
Jeff McCune
dcf8602a0b docs: release v1alpha5 (#321) 2024-11-08 17:00:34 -08:00
Jeff McCune
e07c4d11c8 docs: revise helm values and kustomize tutorials (#316)
These are now where I'd like them to be.
2024-11-08 15:12:22 -08:00
Jeff McCune
b7e1c14192 docs: kustomize tutorial (#316)
Add httpbin using kustomize and patch the result, all from CUE.  The is
the second half of the v1alpha4 helm guide split into a dedicated
tutorial.
2024-11-08 14:08:48 -08:00
Jeff McCune
29f44cdac9 docs: helm values (#316)
Add a helm values tutorial which is a cut down version of the v1alpha4
helm guide.  The httpbin kustomize will immediately follow building on
the prometheus and blackbox charts.
2024-11-08 12:03:14 -08:00
Jeff McCune
96be7a4ae3 docs: add generate platform to hello holos (#311) 2024-11-07 17:59:07 -08:00
Jeff McCune
d6bd030a72 docs: fixup overview 2024-11-07 17:59:07 -08:00
Jeff McCune
75047b590f docs: hello holos edits (#311) 2024-11-07 17:19:53 -08:00
Gary Larizza
a05881df0f Add the Hello Holos tutorial 2024-11-07 16:47:36 -08:00
Jeff McCune
5f406fce5c docs: organize docs for new structure (#301) 2024-11-07 15:10:15 -08:00
Jeff McCune
49c945a037 docs: setup tutorial with diagrams (#301) 2024-11-07 14:13:49 -08:00
Gary Larizza
54de20f0b8 docs: setup tutorial (#301) 2024-11-07 10:24:49 -08:00
Gary Larizza
80b4ab9852 Merge pull request #310 from holos-run/jeff/308-holos-field
docs: rewrite technical overview for v1alpha5
2024-11-07 10:15:01 -08:00
Jeff McCune
acd98aa63c docs: rewrite technical overview for v1alpha5
Attribution: following the structure and length of the tokio docs, with
some more diagrams.
2024-11-07 10:07:45 -08:00
Jeff McCune
0afaab8f2b render: nest output under the holos top level field (#308)
Previously the holos command line expected a Platform and BuildPlan
resource at the top level of the exported data from CUE.  This forced us
to use hidden fields for everything else.

This patch modifies the BuildData struct to first look for a holos top
level field and use it if present.  This opens up other top level fields
for use by end users.

Our intent is to reserve any top level field prefixed with holos.

Note this follows how Timoni works as well.
2024-11-07 07:00:26 -08:00
Jeff McCune
7ded38bc3f v1alpha5: strip down the core and author schemas (#306)
This patch strips down the v1alpha4 core and author schemas to only with
is absolutely necessary for all holos users.  Aspects of platform
configuration applicable to some, even most, but not all users will be
moved into documentation topics organized as a recipe book.

The functionality removed from the v1alpha4 author schemas in v1alpha5
will move into self contained examples documented as topics on the docs
site.

The overall purpose is to have a focused, composeable, maintainable
author schema to help people get started and ideally we can support for
years with making breaking changes.

With this patch the v1alpha5 helm guide test passes.  We're not going to
have this guide anymore but it demonstrates we're back to where we were
with v1alpha4.
2024-11-06 15:22:17 -08:00
Jeff McCune
840676709a docs: partially write the overview doc for v1alpha5
Partial, switching gears to v1alpha5 to unblock others working on
discrete topics.
2024-11-05 10:19:59 -08:00
Jeff McCune
ee30c52673 docs: generate version specific api docs (#303)
Without this patch each version of the core and author schemas are
duplicated into each docs version.  This is unnecessary and difficult to
maintain now that we have docusaurus versioned docs enabled.

This patch updates the schema generation script to check if the docs
version has been released, and if so write into a markdown file in the
versioned docs folder.  If not, the version is written into the next
version folder.

This patch also updates some, but not all, document links to the md or
mdx relative file paths.  This is necessary to generate the correct
versioned links.

A nice outcome of this change is that technical docs no longer need to
link to version specific pages.  For example, `[Core Schema]:
./api/core.md` will always refer to the correct auto generated docs
associated with the docs version.
2024-11-05 07:20:53 -08:00
Jeff McCune
117a00334f docs: restructure docs into tutorial and topics (#301)
The docs for v1alpha4 have the right information, but in the wrong
places.  The most important bits are tucked away in the Core API docs.
One of our first users entirely missed the `holos generate platform`
command mentioned in the Helm guide.

We'll fix this by organizing the docs into two distinct categories.
First, a tutorial written as a series progressively building up the
minimum knowledge to use holos effectively and gain the benefits.  Think
of it as a tour of the essential bits.

The second category are focused topics which stand alone.  They're the
things most people using holos will need to know eventually, but aren't
essential for everyone to know.  For example, Clusters and Fleets will
move from the Author API to stand alone examples of how to implement
these features if necessary.

Then there's a Glossary which serves as the place to describe our
concepts and domain specific language.

Finally there's the API documentation which should be cut down to the
specific version.  The next release version will be v1alpha5.

Attribution: We're copying the Tokio docs structure, it's concise and a
similar size and complexity to our own project.

The Go docs are also an inspiration, but the project is much larger so
not directly comparable.  The organization of https://go.dev/doc/ feels
complete at first glance, despite the size and age of the project.  The
site also makes clear who each section is for without needing to come
right out and say it. Getting started, Using and understanding Go,
Writing modules, using databases, etc...
2024-11-04 20:25:04 -08:00
Jeff McCune
1e03debfac tests: add make unity target
For https://cuelabs.dev/unity/
2024-11-04 19:08:44 -08:00
Jeff McCune
72137b2fa9 docs: upgrade docusaurus to 3.6.0
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-04 06:47:48 -08:00
Jeff McCune
5abf967116 docs: npm run docusaurus docs:version v1alpha4 (#299)
Tag version v1alpha4 so we can start working on v1alpha5 as the next
version in main.
2024-11-04 06:43:15 -08:00
Jeff McCune
5d882f465d website: fix resources.yaml tab in helm guide (#293)
We switched from using a kustomize remote base to a local file so the
tests don't need to make a network round trip to github.  It's also
better practice to use local files for this sort of thing.

In doing so I botched the location of the file, putting it in the
platform registration section.  This patch claifies how `resources.yaml`
is linked to `httpbin.cue` through the `KustomizeConfig: Files:
"resources.yaml": _` field.
2024-11-03 10:57:52 -08:00
Jeff McCune
45bdaac833 main: cue v0.10.1 and add e2e test for helm guide (#293)
Previously there was no test coverage of the
https://holos.run/docs/guides/helm/ guide.  This patch uses Roger's
testscript package, which the CUE folks also use to add comprehensive
test coverage of each step in the guide.  Ideally we would execute these
commands directly from the guide itself, but for now we'll duplicate the
commands into the test script.  This could be enhanced by generating the
test script from the document itself in some way.

When updating the script, use the `holos txtar` command to embed entire
helm charts into the test script.  It's not super fast, but it's better
than network access and it's not terribly slow either.  A few seconds to
unpack.

---

txtar: quote files for testscript unquote

For the helm guide test script we want to include the entire helm chart
which may have files that need to be quoted.  This patch changes the
default behavior of the holos txtar command to quote files if necessary
and list them in an unquote script command in the comment of the
archive.

The purpose is for testscript authors to copy and paste the entire thing
into a test script and include the unquote command at the top.

---

This change also updates CUE to v0.10.1
2024-11-03 10:27:46 -08:00
Jeff McCune
7ae1f990ef website: update quickstart diagram to match helm
Avoid confusion, got a question about this in discord.
2024-11-03 08:53:43 -08:00
Jeff McCune
b526fd1669 testdata: clean up old v1alpha1 tests (#292)
No longer necessary now that we're on v1alpha4.  Test coverage for
v1alpha4 and the user facing guides will be added back soon for use both
in the holos repo and in Unity.
2024-11-01 15:22:13 -07:00
Jeff McCune
5e07655f35 website: fix port in helm guide
Should be 9115 not 6115.
2024-11-01 06:39:48 -07:00
Jeff McCune
6fb6afe8d5 v0.97.3 2024-10-31 21:04:22 -07:00
Jeff McCune
d6f89052d9 website: update helm guide to apply patches (#291)
Updated the helm guide to apply patches while still showing the diff in
the documentation markdown.  The only gotcha is it creates orig files.
2024-10-31 20:54:57 -07:00
Jeff McCune
e4aa7f5994 website: update change-a-service to use hidden fields (#291)
Use _Foo instead of #Foo to hold concrete values.
2024-10-31 20:25:40 -07:00
Jeff McCune
6e4c65cb6c website: update deploy-a-service to use hidden fields (#291)
Use _Foo instead of #Foo to hold concrete values.
2024-10-31 20:13:17 -07:00
Jeff McCune
4f091677e2 website: update quickstart for v1alpha4 hidden fields (#291) 2024-10-31 16:35:41 -07:00
Jeff McCune
0c05df1162 website: update technical overview with consistent fields (#291) 2024-10-31 11:30:20 -07:00
Jeff McCune
64a745fd34 v1alpha4: use hidden fields consistently (#291)
Previously it wasn't clear for users if platform wide structs should be
definitions or hidden fields in CUE.  They should be hidden fields when
they contain data and definitions when they define a schema.

This patch updates the generate platform v1alpha4 subcommand to use the
correct field names consistently for clarity.
2024-10-31 10:45:47 -07:00
Jeff McCune
490f91f580 cli: hide unsupported commands (#289)
Use a simple feature flag system that checks env vars if a feature is
enabled.
2024-10-31 10:04:01 -07:00
Jeff McCune
79b065cda8 website: add open graph image for helm guide try 6 2024-10-30 12:23:18 -07:00
Jeff McCune
0fa6047552 website: add open graph image for helm guide try 5 2024-10-30 12:20:19 -07:00
Jeff McCune
11ecc0cc3a website: add open graph image for helm guide try 4 2024-10-30 12:04:39 -07:00
Jeff McCune
a62e4ba117 website: add open graph image for helm guide try 3 2024-10-30 11:56:01 -07:00
Jeff McCune
07fe667f30 website: add open graph image for helm guide try 2 2024-10-30 11:40:39 -07:00
Jeff McCune
3ad994cbb9 website: add open graph image for helm guide 2024-10-30 11:26:54 -07:00
Jeff McCune
b3d9bd32af website: add why cue for configuration blog post
This is going to be one of the first questions we get.
2024-10-28 21:30:11 -07:00
Jeff McCune
d398b49d7f website: fix head title tag try 2
The open graph title was still showing up poorly, docusaurus generates
it with the Holos | Holos repetition, so we need to override it.
2024-10-28 14:37:43 -07:00
Jeff McCune
12179a6991 website: fix head title tag and social card
Generate the social card manually from https://www.opengraph.xyz/
Override the page title tag, otherwise it shows up as "Announcing Holos
| Holos" in social links, which is weird.
2024-10-28 14:07:40 -07:00
Jeff McCune
fee472bb66 website: add stock social card for annoucement 2024-10-28 13:31:10 -07:00
Jeff McCune
c6a13059f3 v0.97.1 2024-10-28 11:12:55 -07:00
Jeff McCune
ff3eb896f3 webite: put ois logo back
Until we get a better logo.
2024-10-28 10:46:59 -07:00
Jeff McCune
70f70ae6b9 website: fix launch announcement 2024-10-28 10:45:43 -07:00
Jeff McCune
2580ec1c5f website: fix order of api references
The api references are in reverse order and don't have good descriptions
in the index listings.  This patch adds front matter to each generated
document to order them correctly and add a nice description.
2024-10-27 20:43:54 -07:00
Jeff McCune
4fa99e0faa website: add helm prometheus blackbox httpbin guide
The purpose of this guide is to demonstrate how holos offers value
above and beyond helm and kustomize alone.
2024-10-27 19:48:42 -07:00
Jeff McCune
7341d25483 website: add at proto did for bsky 2024-10-25 14:26:42 -07:00
Jeff McCune
3074b3a241 website: add discord invite link 2024-10-24 10:27:37 -07:00
Jeff McCune
9a5e7869c6 v1alpha4: omit the platform model if empty
The platform model distracts from getting started:

  cue export --out yaml ./platform

  kind: Platform
  apiVersion: v1alpha4
  metadata:
    name: default
  spec:
    components:
      - name: prometheus
        component: projects/platform/components/prometheus
        cluster: local
        model: {}

With this patch it's absent by default.
2024-10-23 13:46:58 -07:00
Jeff McCune
1064ceba31 v1alpha4: manage a single workload cluster named local by default
Manage a single cluster by default after generating the platform.
Assume it's a local cluster for use with the guides.
2024-10-23 13:42:46 -07:00
Jeff McCune
4bccaa3710 v1alpha4: _Platform not #Platform for component registration
`_Platform` is a hidden field representing the platform components,
`#Platform` defines the schema of the hidden field.
2024-10-23 13:33:28 -07:00
Jeff McCune
95efae1343 docs: update rendered manifest figure in technical overview
It's too wide with KubeAPI, take it out.
2024-10-20 10:11:57 -07:00
Jeff McCune
ba88125877 v1alpha4: enable config map generator for Kustomize
Without this patch it's difficult to mix in a plain file as a config
map.  This is necessary for the use case of using a Job to generate a
secret in-cluster.  We want a plain shell script to be carried through
and transformed into the job.

We already have the KustomizeConfig fields to support this, they just
weren't wired up to the #Kustomization component kind.

I didn't check if it's wired up to Helm and Kustomize for expedience.
They may be missing there as well.
2024-10-19 10:57:37 -07:00
Jeff McCune
d12c1a0c11 Merge pull request #284 from holos-run/gl/deploy-a-service-v4alpha-update
Update deploy-a-service guide for Author API v1alpha4
2024-10-18 20:35:51 -07:00
Jeff McCune
d56d3400a7 docs: replace tabs in deploy-a-service guide 2024-10-18 20:33:54 -07:00
Gary Larizza
4f0f9dced5 Update deploy-a-service guide for Author API v1alpha4
PROBLEM:

Version v1alpha4 of the Author API has been updated with backwards
incompatible changes, and the deploy-a-service guide uses code from
version v1alpha3.

SOLUTION:

Update any code, links, and data that is out of date, and then run
through the guide to make sure it works locally.

OUTCOME:

The instructions in the deploy-a-service guide will work successfully
with version v1alpha4 of the Author API.
2024-10-18 15:21:56 -07:00
Jeff McCune
6bf0cb8d8e v1alpha4: v0.97.0 2024-10-17 07:50:49 -07:00
Jeff McCune
766c8912b7 Merge pull request #281 from holos-run/jeff/280-authorapi-v1alpha4
v1alpha4 Author API
2024-10-17 07:17:14 -07:00
Jeff McCune
be1dee5f1c v1alpha4: update technical overview guide (#280)
Update the guide and the bank of holos repository to use v1alpha4
instead of v1alpha3.
2024-10-17 07:13:51 -07:00
Jeff McCune
6ad56525ac v1alpha4: refactor --tag to --inject and remove environment (#276)
Cue uses --inject, -t as the flags to set variables for fields tagged
using @tag(var,type=string).

We used --tag, which is different and requires a mental mapping.  Let's
use the same flag and also pass it multiple times like they require so
we can copy and paste the command line output from the debug logs into a
cue export command to see what's going on.

This patch deprecates the --cluster-name flag, use --inject
holos_cluster=mycluster instead.

This patch also removes the environment field from the Component core
API, leaving this to the user namespace to define via tags.  We don't
want to be too opinionated on how users manage their platform, baking
environment into the schema is a slippery slope toward those kinds of
opinions.

Closes: #276
2024-10-16 22:07:47 -07:00
Jeff McCune
791ec5ee71 v1alpha4: refactor core.Component Tags to map[string]string (#280)
Previously it was a []string slice that must be formatted as key=value.
This is more difficult to work with than a map[string]string.
2024-10-16 20:10:14 -07:00
Jeff McCune
638ac7473c fixup 2024-10-16 20:05:05 -07:00
Jeff McCune
ee24b5ce13 fixup 2024-10-16 20:04:18 -07:00
Jeff McCune
fa2fdbe4e8 fixup 2024-10-16 20:00:53 -07:00
Jeff McCune
63e1df1d4c v1alpha4: add common labels to projects schema (#280)
Now that we have CommonLabels as part of the ComponentConfig for all
components, it makes sense to also mix in CommonLabels for a Project.

Common labes are key aspect of the Technical Overview document.
2024-10-16 17:40:40 -07:00
Jeff McCune
2ad0c2a93e render: refactor tm to typeMeta
Gary and I noted tm wasn't clear when I was showing him code.
2024-10-16 17:11:09 -07:00
Jeff McCune
3a6a04f318 v1alpha4: add projects to author api (#280)
Projects are a key element of the Technical Overview guide, so we need
the schema for them in the Author API.
2024-10-16 12:29:35 -07:00
Jeff McCune
8afeece890 v1alpha4: embed ComponentConfig in Helm, Kustomize, Kubernetes (#280)
For the Author API, it would be nice to define a schema for the fields
common to all component kinds.  Users could then configure all kinds by
unifying the schema into their own platform tree.

This makes a clear use case to extract the common fields back into an
embedded struct like we did in v1alpha3.  I removed the embedded struct
in v1alpha4 because it wasn't clear why it should be separate, but now
the use case is clear, to configure all component kinds.
2024-10-16 12:16:48 -07:00
Jeff McCune
bc9c43a0b9 fix argocd application project 2024-10-15 20:52:56 -07:00
Jeff McCune
5a98c77e4c add argocd.argoproj.io/instance label to resources
But not the ArgoCD Application resource.
2024-10-15 20:41:39 -07:00
Jeff McCune
b3f7de39ec v1alpha4: feedback in case of chart cache dead lock (#280)
Without this patch holos render platform may hang until the overall
context timeout is reached.  This is a problem because the user has no
idea why it's hung.

This patch adds a warning at the 5 second and another at the 10 second
mark indicating the lock may be deadlocked.  The user can then remove
the directory.
2024-10-15 16:43:36 -07:00
Jeff McCune
ca4ecf1b28 v1alpha4: KustomizeConfig Resources and Files (#280)
The Kustomize build plan kind needs to support both copying files from
the component directory and pulling resources from https URL's.  Without
this patch this support is missing from the Author API

With this patch the Kustomize build plan kind has a KustomizeConfig
field with two structs, Files and Resources.  The kustomization
resources list is built up from both of these.

Two transformers are used so we don't affect the GitOps transfomer which
really only needs CommonLabels.

I decided to keep this field exclusive to the Kustomize kind, but it
could replace the Kustomization field of the other kinds as well.
2024-10-15 16:11:05 -07:00
Jeff McCune
9ce28660ce v1alpha4: intermediate kustomization (#280)
Without this patch the user facing API doesn't have a way to kustomize
the output of all the build plan kinds.  This patch ensures the
Kustomization field is present on all of Helm, Kustomize, and
Kubernetes.

This field is inteded for patches and transforms.  The second
kustomization in the transformer sequence is intended for common labels
and annotations, managed by a corresponding field instead of a full on
Kustomization resource.
2024-10-15 15:02:15 -07:00
Jeff McCune
728e8ba06e v1alpha4: default helm chart release to chart name (#280) 2024-10-15 14:47:42 -07:00
Jeff McCune
e4b07dad6d v1alpha4: helm enable hooks default false (#280)
Fix:

could not run: could not marshal json projects/platform/components/cert-manager: cue: marshal error: spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON at internal/builder/builder.go:63
spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON:
    /Users/jeff/Holos/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha4/types_go_gen.cue:235:16
could not run: could not render component: exit status 1 at builder/v1alpha4/builder.go:94
2024-10-15 14:43:05 -07:00
Jeff McCune
b7c0bba2b9 v1alpha4: add resources to schema.cue (#280)
Without this patch kustomize cannot execute because resources from CUE
are incomplete.
2024-10-15 14:41:49 -07:00
Jeff McCune
847ab8441c v1alpha4: add default chart name (#280)
Defaults to the build plan name.
2024-10-15 14:40:36 -07:00
Jeff McCune
5f72af3d53 v1alpha4: link helm to the chart struct (#280) 2024-10-15 14:32:16 -07:00
Jeff McCune
33eed43fd1 v1alpha4: surface kustomize stderr output (#280)
Without this patch kustomize errors aren't surfaced when executing holos
render platform.

This patch gives a fighting chance to the user to figure out what's
going on.  The stderr is copied, logged, and surfaced up to the parent
holos render platform command.
2024-10-15 14:32:11 -07:00
Jeff McCune
d2fbbdd1cc logger: log the process pid (#280)
Useful when troubleshooting concurrent processes.
2024-10-15 14:31:42 -07:00
Jeff McCune
e42da118dc v1alpha4: add Helm and Kustomize to author api (#280)
Previously the #Helm and #Kustomize build plan helpers were not defined
in the v1alpha4 Author API.  We need this definition to update the
Quickstart guide for v1alpha4 from v1alpha3.

This patch defines the #Helm and #Kustomize helpers in the Author API
similar to how #Kubernetes is defined.
2024-10-15 10:32:16 -07:00
Jeff McCune
7d36567dcf v1alpha4: define Kubernetes in author api (#280)
Previously #Kubernetes was defined in the platform code.  This is a
problem because every platform engineer would need to copy and paste
this code.

This patch moves the #Kubernetes helper into the cue.mod directory so it
can be imported and used ergonomically.
2024-10-14 20:45:04 -07:00
Jeff McCune
bee698bebe v1alpha4: add platform to the author api (#280)
This patch gets the Author API rendering the namespaces component in the
Bank of Holos guide.  It's not the final form of the API yet, we still
need to decide how best to expose the Kubernetes, Helm, and Kustomize
definitions.

I'm thinking we abstract away the transformers and generators within the
Author API Kubernetes definition.
2024-10-14 17:19:49 -07:00
Jeff McCune
58df0626d0 v1alpha4: plumb --write-to flag from platform (#280)
Without this patch the --write-to flag can't be controlled from the
PlatformSpec in the CoreAPI.  We need to surface this for the ArgoConfig
struct in the AuthorAPI.

That is to say, in v1alpha3 the --write-to flag was previously assumed
to be deploy/ in ArgoConfig using the DeployFiles functionality.  We no
longer have DeployFiles in Core API v1alpha4, all artifacts are instead
written relative to the --write-to flag.  Still, we need to expose this
flag in the PlatformSpec so users can use something other than the
deploy directory.
2024-10-14 15:16:36 -07:00
Jeff McCune
c817a24704 v1alpha4: improve core api documentation 2024-10-10 20:37:38 -07:00
Jeff McCune
bd56f3118c v0.96.0 - v1alpha4 Core API
Release v0.96.0 with the new Core API design.
2024-10-10 13:55:06 -07:00
Jeff McCune
d3aa748e92 v1alpha4: add file generator (#277)
Previously the file generator was unimplemented.  This patch implements
it as a simple file read into the ArtifactMap for use by the Kustomize
or Join transformers.

With this patch all v1alpha4 Core API features are implemented.
Resources, Helm, and File generators.  Kustomize and Join transformers.
2024-10-10 13:52:08 -07:00
Jeff McCune
6738248756 v1alpha4: fix blank log lines
Blank lines show up in the output which is confusing. This patch fixes
the only source location identified with the following command.

    export HOLOS_LOG_LEVEL=debug
    export HOLOS_LOG_FORMAT=json
    holos render platform ./platform 2>&1 | jq -r 'select (.msg == "")'
2024-10-10 13:44:08 -07:00
Jeff McCune
011b488775 v1alpha4: cache helm charts by version (#273)
Previously helm charts were cached only by name, which is a problem
because the wrong version would be used when previously cached.

This patch caches charts by name and version to ensure changes in the
version results in pulling the new cached version.  It is the user's
responsibility to remove old versions.

This patch also ensures only a single go routine can run cacheChart() at
a time across processes.  This is necessary when rendering a platform
because multiple processes will run the Helm generator concurrently, for
example when the same chart is used for multiple environments or
customers.

The mkdir system call serves as the locking mechanism, which is robust
and atomic on all commonly used file systems.
2024-10-10 12:15:20 -07:00
Jeff McCune
c8d89f3291 v1alpha4: add helm chart generator
Previously the helm generator was not implemented and returned an error.
This patch is a first pass copying the helm method from
internal/render/helm.go

Basic testing performed with a podinfo chart.  It works as the previous
versions in v1alpha3 and before works.  This patch does not address the
cached version issue in #273
2024-10-10 08:25:13 -07:00
Jeff McCune
a44ebe5171 refactor Artifact to use strings instead of FilePath
holos.FilePath is intended for paths relative to the platform root
directory.  We use the Artifact to store lots of stuff not related to
the platform root directory, for example kustomization.yaml in a temp
dir.  Most entries are not relative to the platform root directory given
the implicit cfg.WriteTo prefix.
2024-10-09 20:16:13 -07:00
Jeff McCune
66a3b6a874 improve error of missing key
Previously:

  could not run: could not build dev-join: could not get foo.yaml: not set at builder/v1alpha4/builder.go:180

This is confusing because set has nothing to do with the missing input
from the cue code the user writes.

Result:

  could not run: could not build test-join: missing foo.yaml at builder/v1alpha4/builder.go:180

This is better because it at doesn't distract the user from the fact
they're missing a foo.yaml generator output to align with the
transformer input.
2024-10-09 19:39:33 -07:00
Jeff McCune
0886788238 Merge pull request #272 from holos-run/jeff/268-v1alpha4
v1alpha4: add join transformer
2024-10-09 17:45:28 -07:00
Jeff McCune
4fd6785a10 v1alpha4: add Join transformer
The Join transformer was not implemented.  This patch completes the
transformers we support, the Join and Kustomize transformers.
2024-10-09 17:42:56 -07:00
Jeff McCune
e4695fa204 artifact: add Save() method to save artifacts
The code was inlined in a number of places, it makes sense to move it to
the interface.  It'll also make it easier to test, we can provide a null
writer concrete value.
2024-10-09 16:53:31 -07:00
Jeff McCune
4cd9395e6c v1alpha4: add concurrent build plan artifacts
Previously the Artifact collection was processed sequentially.  This
patch provides a modest performance improvement, about 16% faster for
our simple 2 artifact use case, by processing each artifact
concurrently.
2024-10-09 16:15:46 -07:00
Jeff McCune
6f4f355ee0 Merge pull request #270 from holos-run/jeff/268-v1alpha4
v1alpha4 - Resources and Kustomize
2024-10-09 15:28:13 -07:00
Jeff McCune
0fde16f477 v1alpha4: fix lint 2024-10-09 14:43:03 -07:00
Jeff McCune
426b4323f7 v1alpha4: inject component build plan name from platform
Platform rendering provides poor user feedback:

```
❯ holos render platform ./platforms/minimal
rendered namespaces for cluster local in 143.068583ms
rendered namespaces for cluster local in 143.861834ms
rendered namespaces for cluster local in 144.072666ms
rendered namespaces for cluster local in 144.219417ms
rendered platform in 144.326625ms
```

We want to see the metadata.name field of each BuildPlan.  This patch
injects the build plan name from the platform spec to make the name
available through the end to end platform rendering process.

Result:

```
❯ holos render platform ./platforms/minimal
rendered stage-namespaces for cluster local in 146.078375ms
rendered prod-namespaces for cluster local in 146.544583ms
rendered test-namespaces for cluster local in 147.0535ms
rendered dev-namespaces for cluster local in 147.499166ms
rendered platform in 147.553875ms
```
2024-10-09 14:06:42 -07:00
Jeff McCune
ee1e4988a6 v1alpha4: write fully rendered build plan artifacts
With this patch the first use case of CUE Resources + Kustomize is fully
working, artifacts are written into the deploy directory.

❯ holos render platform ./platforms/minimal
rendered namespaces for cluster local in 143.068583ms
rendered namespaces for cluster local in 143.861834ms
rendered namespaces for cluster local in 144.072666ms
rendered namespaces for cluster local in 144.219417ms
rendered platform in 144.326625ms

The output indicates we need to plumb the BuildPlan metadata.name from
the PlatfromSpec through to the render component command.  This is
necessary so we can report the correct name instead of just the base
path.
2024-10-09 14:06:42 -07:00
Jeff McCune
5c391e8444 v1alpha4: refactor error handing for resources and kustomize
Consistently use errors.Format("%s: %w", msg, err)
2024-10-09 14:06:41 -07:00
Jeff McCune
aba1b44f4d v1alpha4: fix resources manifest generation
Without this patch holos writes a single yaml document that is a list.
It needs to write a file that contains multiple documents, each document
a map[string]any representing the kubernetes resource.

This patch fixes the problem.  With this patch kustomize fully executes.
2024-10-09 14:06:41 -07:00
Jeff McCune
44f9615a93 v1alpha4: implement kustomize transformer 2024-10-09 14:06:41 -07:00
Jeff McCune
69a6d2acad v1alpha4: implement resources generator 2024-10-09 14:06:40 -07:00
Jeff McCune
d2c94dc8df refactor artifact manifest field to inputs and output
The manifest field isn't clear.

Much more clear to have generators produce one Output.  Transformers
take multiple Inputs and produce one Output.

The final Transformer, or a single Generator, must produce the final
Artifact.

The Inputs and Output naming to produce an Artifact makes clear the
rendering pipeline we're implementing.

This also makes clear that multiple generators must have at least one
transformer to produce the final output artifact.  We model a simple
Join transformer for this case, which is what `holos` was implicitly
doing previously.
2024-10-09 14:06:40 -07:00
Jeff McCune
01720b38fc refactor BuildContext back to Component
Component makes much more sense, that's the domain terminology we use.
BuildContext was meant to be re-used elsewhere, but we never did so the
name serves no purpose.
2024-10-09 14:06:40 -07:00
Jeff McCune
632e3c2725 refactor BuildStep to Artifact
Step doesn't make sense, they're not sequential.  They're meant to be
concurrently produced, and the whole point is to produce one artifact so
we might as well call it an Artifact.

```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
  name: prod-namespaces
spec:
  component: projects/platform/components/namespaces
  artifacts:
    - artifact: clusters/no-cluster/components/prod-namespaces/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: resources.gen.yaml
          resources:
            Namespace:
              prod-jeff:
                metadata:
                  name: prod-jeff
                  labels:
                    kubernetes.io/metadata.name: prod-jeff
                kind: Namespace
                apiVersion: v1
              prod-gary:
                metadata:
                  name: prod-gary
                  labels:
                    kubernetes.io/metadata.name: prod-gary
                kind: Namespace
                apiVersion: v1
              prod-nate:
                metadata:
                  name: prod-nate
                  labels:
                    kubernetes.io/metadata.name: prod-nate
                kind: Namespace
                apiVersion: v1
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
    - artifact: clusters/no-cluster/gitops/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: application.gen.yaml
          resources:
            Application:
              argocd:
                apiVersion: argoproj.io/v1alpha1
                kind: Application
                metadata:
                  name: prod-namespaces
                  namespace: argocd
                spec:
                  destination:
                    server: https://kubernetes.default.svc
                  project: default
                  source:
                    path: examples/v1alpha4/deploy/clusters/no-cluster/components/prod-namespaces
                    repoURL: https://github.com/holos-run/bank-of-holos
                    targetRevision: main
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
```
2024-10-09 14:06:39 -07:00
Jeff McCune
18e0b48012 refactor generators and transformers
The repeated enabled booleans and file fields are awkward.  It's clear
it's three separate things smashed into one.

kustomize isn't really a generator.  It's useless because there is no
way to reference a plain file in a component directory.

This patch replaces the kustomize generator with a file generator which
simply reads one single file.  Multiple of these generators may be used
to read one or more files.

Then, kustomize may transform these generated files, which are generated
by simply reading from the filesystem.

This API is much improved over the previous.

```
kind: BuildPlan
apiVersion: v1alpha4
metadata:
  name: prod-namespaces
spec:
  component: projects/platform/components/namespaces
  steps:
    - artifact: clusters/no-cluster/components/prod-namespaces/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: resources.gen.yaml
          resources:
            Namespace:
              prod-jeff:
                metadata:
                  name: prod-jeff
                  labels:
                    kubernetes.io/metadata.name: prod-jeff
                kind: Namespace
                apiVersion: v1
              prod-gary:
                metadata:
                  name: prod-gary
                  labels:
                    kubernetes.io/metadata.name: prod-gary
                kind: Namespace
                apiVersion: v1
              prod-nate:
                metadata:
                  name: prod-nate
                  labels:
                    kubernetes.io/metadata.name: prod-nate
                kind: Namespace
                apiVersion: v1
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
    - artifact: clusters/no-cluster/gitops/prod-namespaces.gen.yaml
      generators:
        - kind: Resources
          manifest: application.gen.yaml
          resources:
            Application:
              argocd:
                apiVersion: argoproj.io/v1alpha1
                kind: Application
                metadata:
                  name: prod-namespaces
                  namespace: argocd
                spec:
                  destination:
                    server: https://kubernetes.default.svc
                  project: default
                  source:
                    path: examples/v1alpha4/deploy/clusters/no-cluster/components/prod-namespaces
                    repoURL: https://github.com/holos-run/bank-of-holos
                    targetRevision: main
      transformers:
        - kind: Kustomize
          kustomize:
            kustomization:
              commonLabels:
                holos.run/component.name: prod-namespaces
              resources:
                - resources.gen.yaml
                - application.gen.yaml
```
2024-10-09 14:06:39 -07:00
Jeff McCune
7dfa9dcb93 remove unnecessary buildstep name
The manifest field is the critical piece of information, not an
arbitrary name.
2024-10-09 14:06:17 -07:00
Jeff McCune
3dedd857ea add manifest field to each build step
A build step either produces kubernetes objects or a gitops manifest.
Both are effectively the same, they're just kubernetes resources.

For the use case of applying common labels to both, we'll have the
Author API pass the same Kustomization to two separate build steps.  One
step to produce the resources, a second to produce the argocd
application or flux kustomization.
2024-10-09 14:06:17 -07:00
Jeff McCune
cd379167cc don't provide a default value for files
Leave this for the author api to avoid always including them in the
exported output.
2024-10-09 14:06:16 -07:00
Jeff McCune
52a5348f82 add build plan build step name
Each step produces a manifest and a gitops file, so we need a unique
name for each step.  The most common case will be a single build step
matching the name of the build plan itself.
2024-10-09 14:06:16 -07:00
Jeff McCune
7c5c8fe692 add fields to store generator output and transformer input
The kustomize transformer needs a filename to store the output from
generators so it has an input for the transformer.  This patch adds
fields for each kind of generator so the kustomize.#Kustomization can be
configured with the files `holos` will write generated output to.
2024-10-09 14:06:15 -07:00
Jeff McCune
ec371ed688 inject missing component tag from render platform to render component 2024-10-09 14:06:15 -07:00
Jeff McCune
1984410577 lift up component path to build plan spec
Hard to use when it's deep inside each build step.
2024-10-09 14:06:15 -07:00
Jeff McCune
438e01fbad replace structpb.Struct with map[string]any
So we can decode using a plain json decoder without needing to marshal
the api objects into yaml from CUE.
2024-10-09 14:06:14 -07:00
Jeff McCune
d122cadae6 render: draft 1 of v1alpha4 core api (#268)
First draft, now to try and wire it up to a platform and namespaces
component.
2024-10-09 14:06:14 -07:00
Jeff McCune
53fcdf307b render: build plan builder (#268)
This patch implements the v1alpha4 component rendering builder for a
component BuildPlan.  We don't yet have the CUE definitions, so this
hasn't been end to end tested yet, the next step is defining the
generators and transforms in the core API BuildPlan.
2024-10-08 06:15:15 -07:00
Jeff McCune
5e6bb96147 render: plumb v1alpha4 component (#268)
This patch plumbs the switch statement to branch on a v1alpha4
BuildPlan.  Tags need to be passed from the render platform subcommand
to the render component subcommand via the --tags argument.
2024-10-07 17:23:21 -07:00
Jeff McCune
94d03f9c59 render: render v1alpha4 platform (#268)
This patch implements minimal rendering of a v1alpha4 platform using the
new render.Builder interface.

Tags aren't wired up yet, but this patch does cleanly separate Builder
interface from the Artifacts.  Platform rendering doesn't have an
artifact itself, all artifacts are produced by rendering each component,
so we'll see how that works when we make the same changes to component
rendering, breaking it down to a render.Builder interface that sets
values in an Artifact.
2024-10-07 16:09:06 -07:00
Jeff McCune
0c01c9177d render: switch platform on api version (#268)
The holos cli does not use an interface to handle different Platform api
versions.  This makes it difficult to evolve the API in a backwards
compatible way.

This patch adds a top level switch statement to the `holos render
platform` command.  The switch discriminates on the Platform API
version.  v1alpha3 and earlier are classified as legacy versions and
will use the existing strict types.  v1alpha4 and later versions will
use an interface to render the platform, allowing for multiple types to
implement the platform rendering interface.
2024-10-07 16:08:59 -07:00
Gary Larizza
36f542da7a Merge pull request #269 from holos-run/gl/landing-page-condense
Make the landing page more concise
2024-10-07 14:17:08 -07:00
Gary Larizza
e41f0aa70c Make the landing page more concise
PROBLEM:

The landing page contains a lot of text, and much of that text was
written before we refined our messaging within the guides and technical
overview pages.

SOLUTION:

* Whittle down landing page text to only the key messages we want to convey.
* Provide messaging bullets for the features.
* Steer folks (via links) to the quickstart guide or technical overview document.

OUTCOME:

Visitors don't need to wade through a lot of text to receive key
messaging talking points or links to the pages they should read.
2024-10-07 11:17:21 -07:00
Jeff McCune
351c8ba74a docs: landing page learn more button to technical overview
Instead of the introduction.
2024-10-04 13:15:28 -07:00
Gary Larizza
a0e0b5bb75 Merge pull request #267 from holos-run/gl/diagram-landing-page
Add Holos diagram to landing page
2024-10-04 12:44:23 -07:00
Gary Larizza
7bc94e314c Add Holos diagram to landing page
PROBLEM:

There's a lot of text to grok on the landing page. A diagram would help
to visually convey what Holos does.

SOLUTION:

* Create a diagram
* Add to landing page

OUTCOME:

A visual aide is present on the landing page that helps explain where
Holos sits.
2024-10-04 12:25:02 -07:00
Jeff McCune
9681ce02e7 docs: clean up tech overview pass 3 (#263) 2024-10-04 08:56:44 -07:00
Jeff McCune
07e8e9f5f3 docs: clean up tech overview pass 2 (#263) 2024-10-03 16:09:13 -07:00
Jeff McCune
437d8a7824 docs: clean up tech overview pass 1 (#263) 2024-10-03 09:56:43 -07:00
Jeff McCune
6cc8214636 docs: fix spelling in technical overview (#263) 2024-10-02 11:29:14 -07:00
Jeff McCune
7d8f324014 blog: add holos technical overview article (#263) 2024-10-02 11:22:08 -07:00
Jeff McCune
8555d56f8c website: enable gtag 2024-09-26 11:55:23 -07:00
Jeff McCune
5884d720f2 website: add mailing lists to footer 2024-09-26 11:38:58 -07:00
Jeff McCune
92d274ced1 website: add support page 2024-09-26 11:30:50 -07:00
Jeff McCune
4dd78bd826 guides: add rendering pipeline diagram to quickstart 2024-09-24 15:24:34 -07:00
Jeff McCune
aeb8fd8e72 blog: add launch article 2024-09-24 15:05:16 -07:00
Gary Larizza
1e9744f748 website: update quickstart narrative (#260)
PROBLEM:

The Quickstart is lacking narrative tying the changes we're asking
people to make to the underlying organizational problems.

SOLUTION:

Improve the narrative to surface the problems we are solving and how
this affects the different teams at the Bank of Holos

OUTCOME:

Clarity on the problems the quickstart is solving.

Closes: #259
2024-09-24 11:37:24 -07:00
Jeff McCune
a95abe65f6 website: refine introduction to be more concise 2024-09-24 07:10:41 -07:00
Jeff McCune
c58510b92c website: refine landing to be more concise 2024-09-24 06:59:14 -07:00
Jeff McCune
302a7bfcf0 guides: fix grammar and tweak the change a service guide 2024-09-23 20:48:35 -07:00
Jeff McCune
decbbaab0f guides: fix typos in change a service guide 2024-09-23 20:04:58 -07:00
Jeff McCune
822f599202 guides: move reset cluster step in change a service
Move to the correct place, after forking the repo.
2024-09-23 20:02:03 -07:00
Jeff McCune
6a47edbc3d guides: Add Change Service Guide (#253)
This patch also adds Organization to the Author API as an example of a
regular expression constraint.
2024-09-23 19:54:46 -07:00
Jeff McCune
67d00f1dd4 website: get rid of What is Holos? on landing page 2024-09-23 14:08:09 -07:00
Jeff McCune
bfa02cd6ed website: introduction (#258) 2024-09-23 13:45:28 -07:00
Jeff McCune
2d8ca474f3 website: Landing Page 3 (#256) 2024-09-23 10:08:25 -07:00
Jeff McCune
c7cd6f5190 website: Landing Page 2 (#256) 2024-09-20 17:14:34 -07:00
Jeff McCune
21e3e6f5e4 website: Landing Page 1 (#256) 2024-09-20 14:57:40 -07:00
Jeff McCune
16a4f89c2f doc: clarify story in the deploy a service guide pass 2
Focus on the migration team and platform team.
2024-09-20 11:12:41 -07:00
Jeff McCune
f15dea5ee7 doc: clarify story in the deploy a service guide
Focus on the migration team and platform team.
2024-09-20 11:02:41 -07:00
Jeff McCune
a3bbadd1f5 doc: apply manifests for deploy a service guide
This gets us through to the end with podinfo deployed.  Need to tell the
story of the migration team a bit better though, working with the
platform team to expose the service.
2024-09-20 09:42:00 -07:00
Jeff McCune
30cbb0d537 doc: add deploy a service guide
Covers wrapping a helm chart with a Holos Component.
2024-09-19 21:31:45 -07:00
314 changed files with 82130 additions and 3657 deletions

View File

@@ -5,26 +5,62 @@
"mdx"
],
"words": [
"acmesolver",
"acraccesstoken",
"acraccesstokens",
"admissionregistration",
"alertmanager",
"alertmanagers",
"anchore",
"anthos",
"apiextensions",
"apimachinery",
"apiobjects",
"apiservers",
"applicationset",
"applicationsets",
"appproject",
"appprojects",
"argoproj",
"argumentless",
"authcode",
"authorizationpolicies",
"authorizationpolicy",
"authpolicy",
"authproxy",
"authroutes",
"automount",
"automounting",
"autoscaler",
"balancereader",
"blackbox",
"buildplan",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
"cainjector",
"CAROOT",
"certificaterequest",
"certificaterequests",
"certificatesigningrequests",
"clientset",
"clsx",
"clusterexternalsecret",
"clusterexternalsecrets",
"clusterissuer",
"clusterissuers",
"clusterrole",
"clusterrolebinding",
"clustersecretstore",
"clustersecretstores",
"clusterwide",
"Cmds",
"CNCF",
"CODEOWNERS",
"configdir",
"configmap",
"configmapargs",
"connectrpc",
"cookiesecret",
"coredns",
"corev",
@@ -32,119 +68,251 @@
"crds",
"creds",
"crossplane",
"crunchydata",
"cuecontext",
"cuelang",
"customresourcedefinition",
"daemonset",
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devel",
"devicecode",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtoken",
"ecrauthorizationtokens",
"edns",
"endpointslices",
"entgo",
"envoyfilter",
"envoyfilters",
"errdetails",
"errgroup",
"etcdsnapshotfiles",
"externalsecret",
"externalsecrets",
"fctr",
"fieldmaskpb",
"fieldspec",
"flushcache",
"fullname",
"gatewayclass",
"gatewayclasses",
"gcraccesstoken",
"gcraccesstokens",
"gendoc",
"generationbehavior",
"generatorargs",
"generatoroptions",
"genproto",
"ggnpl",
"ghaction",
"githubaccesstoken",
"githubaccesstokens",
"gitops",
"GOBIN",
"godoc",
"golangci",
"gomarkdoc",
"googleapis",
"goreleaser",
"gotypesalias",
"grpcreflect",
"grpcroute",
"grpcroutes",
"grpcurl",
"healthchecks",
"healthz",
"helmchartargs",
"helmchartconfigs",
"helmcharts",
"Hiera",
"holos",
"holoslogger",
"horizontalpodautoscaler",
"horizontalpodautoscalers",
"Hostaliases",
"Hostnames",
"htpasswd",
"httpbin",
"httproute",
"httproutes",
"iampolicygenerator",
"incpatch",
"Infima",
"intstr",
"isatty",
"istiod",
"jbrx",
"jeffmccune",
"jetstack",
"jiralert",
"Jsonnet",
"kfbh",
"killall",
"kubeadm",
"kubeconfig",
"kubelet",
"kubelogin",
"kubernetesobjects",
"kubeversion",
"Kustomization",
"Kustomizations",
"kustomize",
"kustomizebuild",
"kvpairsources",
"labeldrop",
"labelmap",
"ldflags",
"leaderelection",
"ledgerwriter",
"libnss",
"limitranges",
"livez",
"loadbalancer",
"loadrestrictions",
"logfmt",
"lxnl",
"mattn",
"mccutchen",
"metav",
"mindmap",
"mktemp",
"msqbn",
"mtls",
"Multicluster",
"mutatingwebhookconfiguration",
"mutatingwebhookconfigurations",
"mvdan",
"mxcl",
"myhostname",
"myRegistrKeySecretName",
"mysecret",
"nameofclusterrole",
"nameserver",
"namespacedname",
"ndots",
"networkpolicies",
"nodename",
"nolint",
"oauthproxy",
"objectmap",
"objectmeta",
"omitempty",
"organizationconnect",
"orgid",
"otelconnect",
"outfile",
"overriden",
"Parentspanid",
"patchstrategicmerge",
"pcjc",
"peerauthentication",
"peerauthentications",
"persistentvolumeclaim",
"persistentvolumeclaims",
"persistentvolumes",
"pflag",
"pgadmin",
"pgupgrade",
"pipefail",
"PKCE",
"platformconnect",
"pluginconfig",
"pluginrestrictions",
"podcli",
"poddisruptionbudget",
"poddisruptionbudgets",
"podinfo",
"podmonitor",
"portmapping",
"postgrescluster",
"privs",
"prometheuses",
"promhttp",
"protobuf",
"protojson",
"providerconfig",
"proxyconfig",
"proxyconfigs",
"Pulumi",
"pushgateway",
"pushsecret",
"pushsecrets",
"putenv",
"qjbp",
"quickstart",
"QVRFLS",
"readyz",
"referencegrant",
"referencegrants",
"Registr",
"replacementfield",
"replicasets",
"replicationcontrollers",
"requestauthentication",
"requestauthentications",
"resourcequotas",
"retryable",
"rogpeppe",
"rolebinding",
"rootfs",
"ropc",
"sboms",
"seccomp",
"secretargs",
"SECRETKEY",
"secretstore",
"secretstores",
"serverlb",
"serverside",
"serviceaccount",
"servicebindings",
"serviceentries",
"serviceentry",
"servicemonitor",
"somevalue",
"SOMEVAR",
"sortoptions",
"spanid",
"spiffe",
"stackdriver",
"startupapicheck",
"statefulset",
"statefulsets",
"stefanprodan",
"storageclasses",
"streamwatcher",
"struct",
"structpb",
"subcharts",
"subjectaccessreviews",
"svclb",
"sysfs",
"systemconnect",
"tablewriter",
"templatable",
"testscript",
"thanos",
"Tiltfile",
"timestamppb",
"Timoni",
"tlsclientconfig",
"tokencache",
"Tokener",
"tolerations",
"Traceid",
"traefik",
"transactionhistory",
"tsdb",
"typemeta",
"udev",
"uibutton",
"Unmarshal",
"unstage",
"untar",
"upbound",
"Upsert",
"urandom",
"usecases",
@@ -152,10 +320,20 @@
"userdata",
"userservice",
"validatingwebhookconfiguration",
"validatingwebhookconfigurations",
"vaultdynamicsecret",
"vaultdynamicsecrets",
"virtualservice",
"virtualservices",
"volumeattachments",
"wasmplugin",
"wasmplugins",
"workdir",
"workloadentries",
"workloadentry",
"workloadgroup",
"workloadgroups",
"yournamespace",
"zerolog",
"zitadel",
"ztunnel"

View File

@@ -35,6 +35,9 @@ jobs:
with:
go-version: stable
- name: Setup Syft
uses: anchore/sbom-action/download-syft@1ca97d9028b51809cf6d3c934c3e160716e1b605 # v0.17.5
# Necessary to run these outside of goreleaser, otherwise
# /home/runner/_work/holos/holos/internal/frontend/node_modules/.bin/protoc-gen-connect-query is not in PATH
- name: Install Tools
@@ -54,11 +57,31 @@ jobs:
- name: Git diff
run: git diff
- name: Run GoReleaser
- uses: actions/create-github-app-token@v1
id: app-token
with:
owner: ${{ github.repository_owner }}
app-id: ${{ vars.GORELEASER_APP_ID }}
private-key: ${{ secrets.GORELEASER_APP_PRIVATE_KEY }}
- name: Run GoReleaser if tag
if: github.ref_type == 'tag'
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
version: '~> v2'
args: release --clean
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser if branch
if: github.ref_type == 'branch' && github.ref == 'refs/heads/release'
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: '~> v2'
args: release --clean --nightly
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -13,7 +13,7 @@ permissions:
jobs:
test:
runs-on: gha-rs
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -35,7 +35,7 @@ jobs:
uses: azure/setup-helm@v4
- name: Set up Kubectl
uses: azure/setup-kubectl@v3
uses: azure/setup-kubectl@v4
- name: Install Tools
run: |

View File

@@ -6,7 +6,7 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
version: 2
before:
hooks:
@@ -25,6 +25,24 @@ builds:
- amd64
- arm64
# .goreleaser.yml
nightly:
# Default: `{{ incpatch .Version }}-{{ .ShortCommit }}-nightly`.
# Templates: allowed.
version_template: "{{ .Version }}-{{ .ShortCommit }}-devel"
# Tag name to create if publish_release is enabled.
tag_name: devel
# Whether to publish a release or not.
# Only works on GitHub.
publish_release: true
# Whether to delete previous pre-releases for the same `tag_name` when
# releasing.
# This allows you to keep a single pre-release.
keep_single_release: true
signs:
- artifacts: checksum
args: ["-u", "code-signing-key@openinfrastructure.co", "--output", "${signature}", "--detach-sign", "${artifact}"]
@@ -50,3 +68,39 @@ changelog:
exclude:
- "^docs:"
- "^test:"
source:
enabled: true
name_template: '{{ .ProjectName }}_{{ .Version }}_source_code'
sboms:
- id: source
artifacts: source
documents:
- "{{ .ProjectName }}_{{ .Version }}_sbom.spdx.json"
brews:
- name: holos
repository:
owner: holos-run
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
homepage: "https://holos.run"
description: "Holos CLI"
dependencies:
- name: helm
type: optional
- name: kubectl
type: optional
install: |
bin.install "holos"
bash_output = Utils.safe_popen_read(bin/"holos", "completion", "bash")
(bash_completion/"holos").write bash_output
zsh_output = Utils.safe_popen_read(bin/"holos", "completion", "zsh")
(zsh_completion/"holos").write zsh_output
fish_output = Utils.safe_popen_read(bin/"holos", "completion", "fish")
(fish_completion/"holos.fish").write fish_output
test: |
system "#{bin}/holos version"

View File

@@ -150,6 +150,10 @@ dev-deploy: install image ## deploy to dev
website: ## Build website
./hack/build-website
.PHONY: unity
unity: ## https://cuelabs.dev/unity/
./scripts/unity
.PHONY: help
help: ## Display this help menu.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)

View File

@@ -161,3 +161,66 @@ type Platform struct {
// platform operators to define.
Domain string `cue:"string | *\"holos.localhost\""`
}
// Organization represents organizational metadata useful across the platform.
type Organization struct {
Name string
DisplayName string
Domain string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
type Projects map[core.NameLabel]Project
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[core.NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[core.NameLabel]Hostname
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
type Owner struct {
Name string
Email string
}
// Namespace represents a Kubernetes namespace.
type Namespace struct {
Name string
}
// Hostname represents the left most dns label of a domain name.
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}

View File

@@ -0,0 +1,4 @@
---
description: Simplified abstraction to generate core v1alpha3 components.
sidebar_position: 997
---

View File

@@ -0,0 +1,290 @@
// # Author API
//
// Package v1alpha4 contains ergonomic CUE definitions for Holos component
// authors. These definitions serve as adapters to produce [Core API] resources
// for the holos command line tool.
//
// [Core API]: https://holos.run/docs/api/core/v1alpha4/
package v1alpha4
import core "github.com/holos-run/holos/api/core/v1alpha4"
//go:generate ../../../hack/gendoc
// Platform assembles a Core API [Platform] in the Resource field for the holos
// render platform command. Use the Components field to register components
// with the platform using a struct. This struct is converted into a list for
// final output to holos.
//
// See related:
//
// - [Component] collection of components composing the platform.
// - [Platform] resource assembled for holos to process.
//
// [Platform]: https://holos.run/docs/api/core/v1alpha4/#Platform
// [Component]: https://holos.run/docs/api/core/v1alpha4/#Component
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// Cluster represents a cluster managed by the Platform.
type Cluster struct {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
Name string `json:"name"`
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
Primary bool `json:"primary" cue:"true | *false"`
}
// Fleet represents a named collection of similarly configured Clusters. Useful
// to segregate workload clusters from their management cluster.
type Fleet struct {
Name string `json:"name"`
// Clusters represents a mapping of Clusters by their name.
Clusters map[string]Cluster `json:"clusters" cue:"{[Name=_]: name: Name}"`
}
// StandardFleets represents the standard set of Clusters in a Platform
// segmented into Fleets by their purpose. The management Fleet contains a
// single Cluster, for example a GKE autopilot cluster with no workloads
// deployed for reliability and cost efficiency. The workload Fleet contains
// all other Clusters which contain workloads and sync Secrets from the
// management cluster.
type StandardFleets struct {
// Workload represents a Fleet of zero or more workload Clusters.
Workload Fleet `json:"workload" cue:"{name: \"workload\"}"`
// Management represents a Fleet with one Cluster named management.
Management Fleet `json:"management" cue:"{name: \"management\"}"`
}
// ArgoConfig represents the ArgoCD GitOps configuration associated with a
// [BuildPlan]. Useful to define once at the root of the Platform configuration
// and reuse across all components.
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
type ArgoConfig struct {
// Enabled causes holos to render an Application resource when true.
Enabled bool `cue:"true | *false"`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// Root represents the path from the git repository root to the WriteTo output
// directory, the behavior of the holos render component --write-to flag and
// the Core API Component WriteTo field. Used as a prefix for the
// Application.spec.source.path field.
Root string `cue:"string | *\"deploy\""`
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
// AppProject represents the ArgoCD Project to associate the Application with.
AppProject string `cue:"string | *\"default\""`
}
// Organization represents organizational metadata useful across the platform.
type Organization struct {
Name string
DisplayName string
Domain string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
// Kubernetes provides a [BuildPlan] via the Output field which contains inline
// API Objects provided directly from CUE in the Resources field of
// [ComponentConfig].
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Helm provides a [BuildPlan] via the Output field which generates manifests
// from a helm chart with optional mix-in resources provided directly from CUE
// in the Resources field.
//
// This definition is a convenient way to produce a [BuildPlan] composed of
// three [Resources] generators with one [Kustomize] transformer.
//
// See related:
//
// - [ComponentConfig]
// - [Chart]
// - [Values]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Chart]: https://holos.run/docs/api/core/v1alpha4/#Chart
// [Values]: https://holos.run/docs/api/core/v1alpha4/#Values
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize provides a [BuildPlan] via the Output field which generates
// manifests from a kustomize kustomization with optional mix-in resources
// provided directly from CUE in the Resources field.
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// ComponentConfig represents the configuration common to all kinds of
// component.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// See the following resources for additional details:
//
// - [Resources]
// - [ArgoConfig]
// - [KustomizeConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Resources]: https://holos.run/docs/api/core/v1alpha4/#Resources
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Component represents the path to the component producing the BuildPlan.
Component string
// Cluster represents the name of the cluster this BuildPlan is for.
Cluster string
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// ArgoConfig represents the ArgoCD GitOps configuration for this BuildPlan.
ArgoConfig ArgoConfig
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
// Namespace manages the metadata.namespace field on all resources except the
// ArgoCD Application.
Namespace string `json:",omitempty"`
// KustomizeConfig represents the configuration for kustomize.
KustomizeConfig KustomizeConfig
}
// KustomizeConfig represents the configuration for kustomize post processing.
// The Files field is used to mixing in static manifest files from the component
// directory. The Resources field is used for mixing in manifests from network
// locations urls.
//
// See related:
//
// - [ComponentConfig]
// - [Kustomization]
//
// [Kustomization]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
type Projects map[NameLabel]Project
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[NameLabel]Hostname
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
type Owner struct {
Name string
Email string
}
// Namespace represents a Kubernetes namespace.
type Namespace struct {
Name string
}
// Hostname represents the left most dns label of a domain name.
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}
// NameLabel signals the common use case of converting a struct to a list where
// the name field of each value unifies with the field name of the outer struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,4 @@
---
description: Simplified abstraction to generate core v1alpha4 build plans.
sidebar_position: 996
---

View File

@@ -0,0 +1,145 @@
// Package author contains a standard set of schemas for component authors to
// generate common [core] BuildPlans.
//
// Holos values stability, flexibility, and composition. This package
// intentionally defines only the minimal necessary set of structures.
// Component authors are encouraged to define their own structures building on
// our example [topics].
//
// The Holos Maintainers may add definitions to this package if the community
// identifies nearly all users must define the exact same structure. Otherwise,
// definitions should be added as a customizable example in [topics].
//
// For example, structures representing a cluster and environment almost always
// need to be defined. Their definition varies from one organization to the
// next. Therefore, customizable definitions for a cluster and environment are
// best maintained in [topics], not standardized in this package.
//
// [core]: https://holos.run/docs/api/core/
// [topics]: https://holos.run/docs/topics/
package author
import core "github.com/holos-run/holos/api/core/v1alpha5"
//go:generate ../../../hack/gendoc
// Platform assembles a core Platform in the Resource field for the holos render
// platform command. Use the Components field to register components with the
// platform.
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// ComponentConfig represents the configuration common to all kinds of
// components for use with the holos render component command. All component
// kinds may be transformed with [kustomize] configured with the
// [KustomizeConfig] field.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Path represents the path to the component producing the BuildPlan.
Path string
// Parameters are useful to reuse a component with various parameters.
// Injected as CUE @tag variables. Parameters with a "holos_" prefix are
// reserved for use by the Holos Authors.
Parameters map[string]string
// OutputBaseDir represents the output base directory used when assembling
// artifacts. Useful to organize components by clusters or other parameters.
// For example, holos writes resource manifests to
// {WriteTo}/{OutputBaseDir}/components/{Name}/{Name}.gen.yaml
OutputBaseDir string `cue:"string | *\"\""`
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// KustomizeConfig represents the configuration kustomize.
KustomizeConfig KustomizeConfig
// Artifacts represents additional artifacts to mix in. Useful for adding
// GitOps resources. Each Artifact is unified without modification into the
// BuildPlan.
Artifacts map[NameLabel]core.Artifact
}
// Helm assembles a BuildPlan rendering a helm chart. Useful to mix in
// additional resources from CUE and transform the helm output with kustomize.
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `cue:"true | *false"`
// Namespace sets the helm chart namespace flag if provided.
Namespace string `json:",omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:",omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:",omitempty"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kubernetes assembles a BuildPlan containing inline resources exported from
// CUE.
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize assembles a BuildPlan rendering manifests from a [kustomize]
// kustomization.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// KustomizeConfig represents the configuration for [kustomize] post processing.
// Use the Files field to mix in plain manifest files located in the component
// directory. Use the Resources field to mix in manifests from network urls.
//
// [kustomize]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// CommonLabels represents common labels added without including selectors.
CommonLabels map[string]string
}
// NameLabel represents the common use case of converting a struct to a list
// where the name field of each value unifies with the field name of the outer
// struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,5 @@
---
title: Author Schemas
description: Standardized schemas for component authors.
sidebar_position: 200
---

View File

@@ -0,0 +1,4 @@
---
description: Core v1alpha4 schema for advanced use cases.
sidebar_position: 998
---

View File

@@ -14,6 +14,12 @@ import "google.golang.org/protobuf/types/known/structpb"
// output.
type InternalLabel string
// NameLabel is a unique identifier useful to convert a CUE struct to a list
// when the values have a Name field with a default value. This type is
// intended to indicate the common use case of converting a struct to a list
// where the Name field of the value aligns with the struct field name.
type NameLabel string
// Kind is a kubernetes api object kind. Defined as a type for clarity and type
// checking.
type Kind string

View File

@@ -0,0 +1,4 @@
---
description: Core v1alpha3 schema for advanced use cases.
sidebar_position: 997
---

View File

@@ -0,0 +1,4 @@
---
description: Core schema for holos to render a component BuildPlan.
sidebar_position: 100
---

402
api/core/v1alpha4/types.go Normal file
View File

@@ -0,0 +1,402 @@
// # Core API
//
// Package v1alpha4 contains the core API contract between the holos cli and CUE
// configuration code. Platform designers, operators, and software developers
// use this API to write configuration in CUE which holos loads. The Core API
// is declarative. Each resource represents a desired state necessary for holos
// to fully render Kubernetes manifests into plain files.
//
// The following resources provide important context for the Core API. The
// [Author API] is intended for component authors as a convenient adapter for
// the Core API resources Holos expects.
//
// 1. [Technical Overview]
// 2. [Quickstart]
// 3. [Author API]
//
// # Platform
//
// [Platform] defines the complete configuration of a platform. A platform
// represents a [Component] collection.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
//
// # Component
//
// A [Component] is the combination of CUE code along one path relative to the
// platform root directory plus data injected from the [PlatformSpec] via CUE tags.
// The platform configuration root is the directory containing cue.mod.
//
// A [Component] always produces exactly one [BuildPlan].
//
// # BuildPlan
//
// A [BuildPlan] contains an [Artifact] collection. A BuildPlan often produces
// two artifacts, one containing the fully rendered Kubernetes API resources,
// the other containing an additional resource to manage the former with GitOps.
// For example, a BuildPlan for a podinfo component produces a manifest
// containing a Deployment and a Service, along with a second manifest
// containing an ArgoCD Application.
//
// Inspect a BuildPlan resource holos render component would process by executing:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// # Artifact
//
// An [Artifact] is one fully rendered manifest file produced from the final
// [Transformer] in a sequence of transformers. An Artifact may also be
// produced directly from a [Generator], but this use case is uncommon.
//
// # Transformer
//
// A [Transformer] takes multiple inputs from prior [Generator] or [Transformer]
// outputs, then transforms the data into one output. [Kustomize] is the most
// commonly used transformer, though a simple [Join] is also supported.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// # Generators
//
// A [Generator] generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
// [Author API]: https://holos.run/docs/api/author/
// [Quickstart]: https://holos.run/docs/quickstart/
// [Technical Overview]: https://holos.run/docs/technical-overview/
package v1alpha4
//go:generate ../../../hack/gendoc
// BuildPlan represents a build plan for holos to execute. Each [Platform]
// component produces exactly one BuildPlan.
//
// One or more [Artifact] files are produced by a BuildPlan, representing the
// fully rendered manifests for the Kubernetes API Server.
//
// # Example BuildPlan
//
// Command:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// Output:
//
// kind: BuildPlan
// apiVersion: v1alpha4
// metadata:
// name: dev-namespaces
// spec:
// component: projects/platform/components/namespaces
// artifacts:
// - artifact: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// generators:
// - kind: Resources
// output: resources.gen.yaml
// resources:
// Namespace:
// dev-jeff:
// metadata:
// name: dev-jeff
// labels:
// kubernetes.io/metadata.name: dev-jeff
// kind: Namespace
// apiVersion: v1
// transformers:
// - kind: Kustomize
// inputs:
// - resources.gen.yaml
// output: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// kustomize:
// kustomization:
// commonLabels:
// holos.run/component.name: dev-namespaces
// resources:
// - resources.gen.yaml
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha4\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Component represents the component that produced the build plan.
// Represented as a path relative to the platform root.
Component string `json:"component"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
}
// Generator generates an intermediate manifest for a [Artifact].
//
// Each Generator in a [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// Refer to [Resources], [Helm], and [File].
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents a kubernetes resources [Generator] from CUE.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Transformer transforms [Generator] manifests within a [Artifact].
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContent represents file contents.
type FileContent string
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// NameLabel is a unique identifier useful to convert a CUE struct to a list
// when the values have a Name field with a default value. NameLabel indicates
// the common use case of converting a struct to a list where the Name field of
// the value aligns with the outer struct field name.
//
// For example:
//
// Outer: [NAME=_]: Name: NAME
type NameLabel string
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
// primary use case is to collect the cluster names, cluster types, platform
// model, and holos components to build into one resource.
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha4\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec represents the specification.
Spec PlatformSpec `json:"spec"`
}
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
}
// PlatformSpec represents the specification of a [Platform]. Think of a
// platform spec as a [Component] collection for multiple kubernetes clusters
// combined with the user-specified Platform Model.
type PlatformSpec struct {
// Components represents a list of holos components to manage.
Components []Component `json:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan].
// Component carries information injected from holos render platform to holos
// render component to produce each [BuildPlan].
//
// All of these fields are passed to the holos render component command using
// flags, which in turn are injected to CUE using tags. For clarity, CUE field
// and tag names should match the struct json tag names below.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_name" to set the BuildPlan metadata.name field. Necessary for clear
// user feedback during platform rendering.
Name string `json:"name"`
// Component represents the path of the component relative to the platform
// root. Injected as the tag variable "holos_component".
Component string `json:"component"`
// Cluster is the cluster name to provide when rendering the component.
// Injected as the tag variable "holos_cluster".
Cluster string `json:"cluster"`
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
// Injected as the tag "holos_model".
Model map[string]any `json:"model,omitempty"`
// Tags represents cue @tag variables injected into the holos render component
// command from the holos render platform command. Tags with a "holos_"
// prefix are reserved for use by the Holos Authors.
Tags map[string]string `json:"tags,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
}
// Tags represents standardized fields injected into the component [BuildPlan]
// from the [Platform].
//
// Note, tags should have a reasonable default value to easily use cue eval and
// cue export without needing to make a bunch of decisions about tag values.
//
// Example:
//
// import core "github.com/holos-run/holos/api/core/v1alpha4"
// _Tags: core.#Tags & {
// cluster: _ @tag(cluster, type=string)
// environment: _ @tag(environment, type=string)
// component: _ @tag(component, type=string)
// name: _ @tag(name, type=string)
// }
type Tags struct {
// Name represents the BuildPlan metadata.name field injected from the Platform.
Name string `json:"name" cue:"string | *\"no-name\""`
// Cluster represents the cluster name injected from
Cluster string `json:"cluster" cue:"string | *\"no-cluster\""`
// Environment represents the build plan environment.
Environment string `json:"environment" cue:"string | *\"no-environment\""`
// Component represents the path of the component relative to the platform root.
Component string `json:"component" cue:"string | *\"no-component\""`
}

View File

@@ -0,0 +1,5 @@
---
title: Core Schemas
description: BuildPlan defines the holos rendering pipeline.
sidebar_position: 100
---

279
api/core/v1alpha5/types.go Normal file
View File

@@ -0,0 +1,279 @@
// Package core contains schemas for a [Platform] and [BuildPlan]. Holos takes
// a [Platform] as input, then iterates over each [Component] to produce a
// [BuildPlan]. Holos processes the [BuildPlan] to produce fully rendered
// manifests, each an [Artifact].
package core
//go:generate ../../../hack/gendoc
// BuildPlan represents an implementation of the [rendered manifest pattern].
// Holos processes a BuildPlan to produce one or more [Artifact] output files.
// BuildPlan artifact files usually contain Kubernetes manifests, but they may
// have any content.
//
// A BuildPlan usually produces two artifacts. One artifact contains a manifest
// of resources. A second artifact contains a GitOps resource to manage the
// first, usually an ArgoCD Application resource.
//
// Holos uses CUE to construct a BuildPlan. A future enhancement will support
// user defined executables providing a BuildPlan to Holos in the style of an
// [external credential provider].
//
// [rendered manifest pattern]: https://akuity.io/blog/the-rendered-manifests-pattern
// [external credential provider]: https://github.com/kubernetes/enhancements/blob/313ad8b59c80819659e1fbf0f165230f633f2b22/keps/sig-auth/541-external-credential-providers/README.md
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
// APIVersion represents the versioned schema of the resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec specifies the desired state of the resource.
Spec BuildPlanSpec `json:"spec"`
// Source reflects the origin of the BuildPlan.
Source BuildPlanSource `json:"source,omitempty"`
}
// BuildPlanSpec represents the specification of the [BuildPlan].
type BuildPlanSpec struct {
// Artifacts represents the artifacts for holos to build.
Artifacts []Artifact `json:"artifacts"`
// Disabled causes the holos cli to disregard the build plan.
Disabled bool `json:"disabled,omitempty"`
}
// BuildPlanSource reflects the origin of a [BuildPlan]. Useful to save a build
// plan to a file, then re-generate it without needing to process a [Platform]
// component collection.
type BuildPlanSource struct {
// Component reflects the component that produced the build plan.
Component Component `json:"component,omitempty"`
}
// Artifact represents one fully rendered manifest produced by a [Transformer]
// sequence, which transforms a [Generator] collection. A [BuildPlan] produces
// an [Artifact] collection.
//
// Each Artifact produces one manifest file artifact. Generator Output values
// are used as Transformer Inputs. The Output field of the final [Transformer]
// should have the same value as the Artifact field.
//
// When there is more than one [Generator] there must be at least one
// [Transformer] to combine outputs into one Artifact. If there is a single
// Generator, it may directly produce the Artifact output.
//
// An Artifact is processed concurrently with other artifacts in the same
// [BuildPlan]. An Artifact should not use an output from another Artifact as
// an input. Each [Generator] may also run concurrently. Each [Transformer] is
// executed sequentially starting after all generators have completed.
//
// Output fields are write-once. It is an error for multiple Generators or
// Transformers to produce the same Output value within the context of a
// [BuildPlan].
type Artifact struct {
Artifact FilePath `json:"artifact,omitempty"`
Generators []Generator `json:"generators,omitempty"`
Transformers []Transformer `json:"transformers,omitempty"`
Skip bool `json:"skip,omitempty"`
}
// Generator generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// Each Generator in an [Artifact] must have a distinct Output value for a
// [Transformer] to reference.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
type Generator struct {
// Kind represents the kind of generator. Must be Resources, Helm, or File.
Kind string `json:"kind" cue:"\"Resources\" | \"Helm\" | \"File\""`
// Output represents a file for a Transformer or Artifact to consume.
Output FilePath `json:"output"`
// Resources generator. Ignored unless kind is Resources. Resources are
// stored as a two level struct. The top level key is the Kind of resource,
// e.g. Namespace or Deployment. The second level key is an arbitrary
// InternalLabel. The third level is a map[string]any representing the
// Resource.
Resources Resources `json:"resources,omitempty"`
// Helm generator. Ignored unless kind is Helm.
Helm Helm `json:"helm,omitempty"`
// File generator. Ignored unless kind is File.
File File `json:"file,omitempty"`
}
// Resource represents one kubernetes api object.
type Resource map[string]any
// Resources represents Kubernetes resources. Most commonly used to mix
// resources into the [BuildPlan] generated from CUE, but may be generated from
// elsewhere.
type Resources map[Kind]map[InternalLabel]Resource
// File represents a simple single file copy [Generator]. Useful with a
// [Kustomize] [Transformer] to process plain manifest files stored in the
// component directory. Multiple File generators may be used to transform
// multiple resources.
type File struct {
// Source represents a file sub-path relative to the component path.
Source FilePath `json:"source"`
}
// Helm represents a [Chart] manifest [Generator].
type Helm struct {
// Chart represents a helm chart to manage.
Chart Chart `json:"chart"`
// Values represents values for holos to marshal into values.yaml when
// rendering the chart.
Values Values `json:"values"`
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool `json:"enableHooks,omitempty"`
// Namespace represents the helm namespace flag
Namespace string `json:"namespace,omitempty"`
// APIVersions represents the helm template --api-versions flag
APIVersions []string `json:"apiVersions,omitempty"`
// KubeVersion represents the helm template --kube-version flag
KubeVersion string `json:"kubeVersion,omitempty"`
}
// Values represents [Helm] Chart values generated from CUE.
type Values map[string]any
// Chart represents a [Helm] Chart.
type Chart struct {
// Name represents the chart name.
Name string `json:"name"`
// Version represents the chart version.
Version string `json:"version"`
// Release represents the chart release when executing helm template.
Release string `json:"release"`
// Repository represents the repository to fetch the chart from.
Repository Repository `json:"repository,omitempty"`
}
// Repository represents a [Helm] [Chart] repository.
type Repository struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Transformer combines multiple inputs from prior [Generator] or [Transformer]
// outputs into one output. [Kustomize] is the most commonly used transformer.
// A simple [Join] is also supported for use with plain manifest files.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
type Transformer struct {
// Kind represents the kind of transformer. Must be Kustomize, or Join.
Kind string `json:"kind" cue:"\"Kustomize\" | \"Join\""`
// Inputs represents the files to transform. The Output of prior Generators
// and Transformers.
Inputs []FilePath `json:"inputs"`
// Output represents a file for a subsequent Transformer or Artifact to
// consume.
Output FilePath `json:"output"`
// Kustomize transformer. Ignored unless kind is Kustomize.
Kustomize Kustomize `json:"kustomize,omitempty"`
// Join transformer. Ignored unless kind is Join.
Join Join `json:"join,omitempty"`
}
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
}
// Kustomize represents a kustomization [Transformer].
type Kustomize struct {
// Kustomization represents the decoded kustomization.yaml file
Kustomization Kustomization `json:"kustomization"`
// Files holds file contents for kustomize, e.g. patch files.
Files FileContentMap `json:"files,omitempty"`
}
// Kustomization represents a kustomization.yaml file for use with the
// [Kustomize] [Transformer]. Untyped to avoid tightly coupling holos to
// kubectl versions which was a problem for the Flux maintainers. Type checking
// is expected to happen in CUE against the kubectl version the user prefers.
type Kustomization map[string]any
// FileContent represents file contents.
type FileContent string
// FileContentMap represents a mapping of file paths to file contents.
type FileContentMap map[FilePath]FileContent
// FilePath represents a file path.
type FilePath string
// InternalLabel is an arbitrary unique identifier internal to holos itself.
// The holos cli is expected to never write a InternalLabel value to rendered
// output files, therefore use a InternalLabel when the identifier must be
// unique and internal. Defined as a type for clarity and type checking.
type InternalLabel string
// Kind is a discriminator. Defined as a type for clarity and type checking.
type Kind string
// Metadata represents data about the resource such as the Name.
type Metadata struct {
// Name represents the resource name.
Name string `json:"name"`
}
// Platform represents a platform to manage. A Platform specifies a [Component]
// collection and integrates the components together into a holistic platform.
// Holos iterates over the [Component] collection producing a [BuildPlan] for
// each, which holos then executes to render manifests.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
type Platform struct {
// Kind is a string value representing the resource.
Kind string `json:"kind" cue:"\"Platform\""`
// APIVersion represents the versioned schema of this resource.
APIVersion string `json:"apiVersion" cue:"string | *\"v1alpha5\""`
// Metadata represents data about the resource such as the Name.
Metadata Metadata `json:"metadata"`
// Spec represents the platform specification.
Spec PlatformSpec `json:"spec"`
}
// PlatformSpec represents the platform specification.
type PlatformSpec struct {
// Components represents a collection of holos components to manage.
Components []Component `json:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
// from a path containing parameterized CUE configuration.
type Component struct {
// Name represents the name of the component. Injected as the tag variable
// "holos_component_name".
Name string `json:"name"`
// Path represents the path of the component relative to the platform root.
// Injected as the tag variable "holos_component_path".
Path string `json:"path"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
// Parameters represent user defined input variables to produce various
// [BuildPlan] resources from one component path. Injected as CUE @tag
// variables. Parameters with a "holos_" prefix are reserved for use by the
// Holos Authors. Multiple environments are a prime example of an input
// parameter that should always be user defined, never defined by Holos.
Parameters map[string]string `json:"parameters,omitempty"`
}

View File

@@ -2,8 +2,10 @@ package main
import (
"os"
"path/filepath"
"testing"
cue "cuelang.org/go/cmd/cue/cmd"
"github.com/holos-run/holos/internal/cli"
"github.com/rogpeppe/go-internal/testscript"
)
@@ -11,11 +13,42 @@ import (
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cli.MakeMain(),
"cue": cue.Main,
}))
}
func TestGetSecrets(t *testing.T) {
testscript.Run(t, testscript.Params{
Dir: "testdata",
})
func TestGuides_v1alpha4(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha4", "guides")))
}
func TestGuides_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "guides")))
}
func TestSchemas_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "schemas")))
}
func TestIssues_v1alpha5(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha5", "issues")))
}
func TestCLI(t *testing.T) {
testscript.Run(t, params("cli"))
}
func params(dir string) testscript.Params {
return testscript.Params{
Dir: filepath.Join("tests", dir),
RequireExplicitExec: true,
RequireUniqueNames: os.Getenv("HOLOS_WORKDIR_ROOT") == "",
WorkdirRoot: os.Getenv("HOLOS_WORKDIR_ROOT"),
UpdateScripts: os.Getenv("HOLOS_UPDATE_SCRIPTS") != "",
Setup: func(env *testscript.Env) error {
// Just like cmd/cue/cmd.TestScript, set up separate cache and config dirs per test.
env.Setenv("CUE_CACHE_DIR", filepath.Join(env.WorkDir, "tmp/cachedir"))
configDir := filepath.Join(env.WorkDir, "tmp/configdir")
env.Setenv("CUE_CONFIG_DIR", configDir)
return nil
},
}
}

View File

@@ -1,34 +0,0 @@
# Want support for intermediary constraints
exec holos build ./foo/... --log-level debug
stdout '^bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- foo/constraints.cue --
package holos
metadata: name: "jeff"
-- foo/bar/bar.cue --
package holos
spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
apiObjectMap: foo: bar: "bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b"
}
]
-- schema.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#KubernetesObjects: {
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
apiObjectMap: {...}
}
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"

View File

@@ -1,20 +0,0 @@
# Want cue errors to show files and lines
! exec holos build .
stderr 'apiObjectMap.foo.bar: cannot convert incomplete value'
stderr '/component.cue:\d+:\d+$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: foo: bar: _baz}]
_baz: string

View File

@@ -1,61 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,62 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
stderr 'skipping helm: no chart name specified'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,25 +0,0 @@
# Want api object kind and name in errors
! exec holos build .
stderr 'apiObjects.secretstore.default.foo: field not allowed'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
metadata: name: string
}
apiObjects: {
secretstore: {
default: #SecretStore & { foo: "not allowed" }
}
}

View File

@@ -1,289 +0,0 @@
# Want helm errors to show up
! exec holos build .
stderr 'Error: execution error at \(zitadel/templates/secret_zitadel-masterkey.yaml:2:4\): Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- zitadel.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [_HelmChart]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
_HelmChart: {
apiVersion: "holos.run/v1alpha1"
kind: "HelmChart"
metadata: name: "zitadel"
namespace: "zitadel"
chart: {
name: "zitadel"
version: "7.9.0"
release: name
repository: {
name: "zitadel"
url: "https://charts.zitadel.com"
}
}
}
-- vendor/zitadel/templates/secret_zitadel-masterkey.yaml --
{{- if (or (and .Values.zitadel.masterkey .Values.zitadel.masterkeySecretName) (and (not .Values.zitadel.masterkey) (not .Values.zitadel.masterkeySecretName)) ) }}
{{- fail "Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName" }}
{{- end }}
{{- if .Values.zitadel.masterkey -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: zitadel-masterkey
{{- with .Values.zitadel.masterkeyAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "zitadel.labels" . | nindent 4 }}
stringData:
masterkey: {{ .Values.zitadel.masterkey }}
{{- end -}}
-- vendor/zitadel/Chart.yaml --
apiVersion: v2
appVersion: v2.46.0
description: A Helm chart for ZITADEL
icon: https://zitadel.com/zitadel-logo-dark.svg
kubeVersion: '>= 1.21.0-0'
maintainers:
- email: support@zitadel.com
name: zitadel
url: https://zitadel.com
name: zitadel
type: application
version: 7.9.0
-- vendor/zitadel/values.yaml --
# Default values for zitadel.
zitadel:
# The ZITADEL config under configmapConfig is written to a Kubernetes ConfigMap
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
configmapConfig:
ExternalSecure: true
Machine:
Identification:
Hostname:
Enabled: true
Webhook:
Enabled: false
# The ZITADEL config under secretConfig is written to a Kubernetes Secret
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
secretConfig:
# Annotations set on secretConfig secret
secretConfigAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# Reference the name of a secret that contains ZITADEL configuration.
configSecretName:
# The key under which the ZITADEL configuration is located in the secret.
configSecretKey: config-yaml
# ZITADEL uses the masterkey for symmetric encryption.
# You can generate it for example with tr -dc A-Za-z0-9 </dev/urandom | head -c 32
masterkey: ""
# Reference the name of the secret that contains the masterkey. The key should be named "masterkey".
# Note: Either zitadel.masterkey or zitadel.masterkeySecretName must be set
masterkeySecretName: ""
# Annotations set on masterkey secret
masterkeyAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The CA Certificate needed for establishing secure database connections
dbSslCaCrt: ""
# The Secret containing the CA certificate at key ca.crt needed for establishing secure database connections
dbSslCaCrtSecret: ""
# The db admins secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslAdminCrtSecret: ""
# The db users secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslUserCrtSecret: ""
# Generate a self-signed certificate using an init container
# This will also mount the generated files to /etc/tls/ so that you can reference them in the pod.
# E.G. KeyPath: /etc/tls/tls.key CertPath: /etc/tls/tls.crt
# By default, the SAN DNS names include, localhost, the POD IP address and the POD name. You may include one more by using additionalDnsName like "my.zitadel.fqdn".
selfSignedCert:
enabled: false
additionalDnsName:
replicaCount: 3
image:
repository: ghcr.io/zitadel/zitadel
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
chownImage:
repository: alpine
pullPolicy: IfNotPresent
tag: "3.19"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
# Annotations to add to the deployment
annotations: {}
# Annotations to add to the configMap
configMap:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podAdditionalLabels: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
securityContext: {}
# Additional environment variables
env:
[]
# - name: ZITADEL_DATABASE_POSTGRES_HOST
# valueFrom:
# secretKeyRef:
# name: postgres-pguser-postgres
# key: host
service:
type: ClusterIP
# If service type is "ClusterIP", this can optionally be set to a fixed IP address.
clusterIP: ""
port: 8080
protocol: http2
annotations: {}
scheme: HTTP
ingress:
enabled: false
className: ""
annotations: {}
hosts:
- host: localhost
paths:
- path: /
pathType: Prefix
tls: []
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
topologySpreadConstraints: []
initJob:
# Once ZITADEL is installed, the initJob can be disabled.
enabled: true
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "1"
resources: {}
backoffLimit: 5
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
# Available init commands :
# "": initialize ZITADEL instance (without skip anything)
# database: initialize only the database
# grant: set ALL grant to user
# user: initialize only the database user
# zitadel: initialize ZITADEL internals (skip "create user" and "create database")
command: ""
setupJob:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "2"
resources: {}
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
additionalArgs:
- "--init-projections=true"
machinekeyWriter:
image:
repository: bitnami/kubectl
tag: ""
resources: {}
readinessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
livenessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
startupProbe:
enabled: true
periodSeconds: 1
failureThreshold: 30
metrics:
enabled: false
serviceMonitor:
# If true, the chart creates a ServiceMonitor that is compatible with Prometheus Operator
# https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#monitoring.coreos.com/v1.ServiceMonitor.
# The Prometheus community Helm chart installs this operator
# https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#kube-prometheus-stack
enabled: false
honorLabels: false
honorTimestamps: true
pdb:
enabled: false
# these values are used for the PDB and are mutally exclusive
minAvailable: 1
# maxUnavailable: 1
annotations: {}

View File

@@ -1,39 +0,0 @@
# Kustomize is a supported holos component kind
exec holos render component --cluster-name=mycluster . --log-level=debug
# Want generated output
cmp want.yaml deploy/clusters/mycluster/components/kstest/kstest.gen.yaml
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KustomizeBuildList: [{metadata: name: "kstest"}]
-- kustomization.yaml --
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mynamespace
resources:
- serviceaccount.yaml
-- serviceaccount.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
-- want.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
namespace: mynamespace

View File

@@ -1,17 +0,0 @@
# https://github.com/holos-run/holos/issues/72
# Want holos to fail on unknown fields to catch typos and aid refactors
! exec holos build .
stderr 'unknown field \\"TypoKubernetesObjectsList\\"'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: TypoKubernetesObjectsList: []

View File

@@ -0,0 +1,2 @@
# https://github.com/holos-run/holos/issues/334
exec holos

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,38 @@
# https://github.com/holos-run/holos/issues/332
env HOME=$WORK
# Mock with a stub helm command
env PATH=$WORK/bin:$PATH
chmod 755 bin/helm
# Initialize the platform
exec holos init platform v1alpha5 --force
# when helm update returns an error
! exec holos render platform ./platform
# holos should log the helm error to stderr
stderr 'Error: chart "podinfo" matching 0.0.0 not found in podinfo index'
-- bin/helm --
#! /bin/bash
echo 'Error: chart "podinfo" matching 0.0.0 not found in podinfo index' >&2
exit 2
-- platform/podinfo.cue --
package holos
Platform: Components: podinfo: {
name: "podinfo"
path: "components/podinfo"
}
-- components/podinfo/podinfo.cue --
package holos
// Produce a helm chart build plan.
holos: HelmChart.BuildPlan
HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "0.0.0"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}

View File

@@ -0,0 +1,144 @@
# https://github.com/holos-run/holos/issues/330
exec holos init platform v1alpha5 --force
exec helm template ./components/capabilities/vendor/0.1.0/capabilities
cmp stdout want/helm-template.yaml
exec holos render platform ./platform
# When no capabilities are specified
cmp deploy/components/capabilities/capabilities.gen.yaml want/when-no-capabilities-specified.yaml
# With APIVersions specified
cmp deploy/components/specified/specified.gen.yaml want/with-capabilities-specified.yaml
# With KubeVersion specified
cmp deploy/components/kubeversion1/kubeversion1.gen.yaml want/with-kubeversion-specified.yaml
# With both APIVersions and KubeVersion specified
cmp deploy/components/kubeversion2/kubeversion2.gen.yaml want/with-both-specified.yaml
-- want/with-both-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-kubeversion-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.20.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/when-no-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1beta1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- want/with-capabilities-specified.yaml --
apiVersion: v1
kind: Service
metadata:
annotations:
kubeVersion: v1.31.0
name: has-foo-v1
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
-- platform/capabilities.cue --
package holos
import "encoding/json"
Platform: Components: capabilities: {
name: "capabilities"
path: "components/capabilities"
}
Platform: Components: specified: {
name: "specified"
path: "components/capabilities"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
Platform: Components: kubeversion1: {
name: "kubeversion1"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
}
Platform: Components: kubeversion2: {
name: "kubeversion2"
path: "components/capabilities"
parameters: kubeVersion: "v1.20.0"
parameters: apiVersions: json.Marshal(["foo/v1","bar/v1"])
}
-- components/capabilities/capabilities.cue --
package holos
import "encoding/json"
holos: Component.BuildPlan
Component: #Helm & {
Name: string @tag(holos_component_name, type=string)
Chart: name: "capabilities"
Chart: version: "0.1.0"
_APIVersions: string | *"[]" @tag(apiVersions, type=string)
APIVersions: json.Unmarshal(_APIVersions)
KubeVersion: string | *"v1.31.0" @tag(kubeVersion, type=string)
}
-- components/capabilities/vendor/0.1.0/capabilities/Chart.yaml --
apiVersion: v2
name: capabilities
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: "1.16.0"
-- components/capabilities/vendor/0.1.0/capabilities/templates/service.yaml --
apiVersion: v1
kind: Service
metadata:
{{- if .Capabilities.APIVersions.Has "foo/v1" }}
name: has-foo-v1
{{- else }}
name: has-foo-v1beta1
{{- end }}
annotations:
kubeVersion: {{ .Capabilities.KubeVersion }}
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
-- want/helm-template.yaml --
---
# Source: capabilities/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: has-foo-v1beta1
annotations:
kubeVersion: v1.31.0
spec:
ports:
- port: 80
targetPort: http
protocol: TCP
name: http

View File

@@ -0,0 +1,52 @@
# author.#Kubernetes
# Start in an empty directory.
cd $WORK
# Generate the directory structure we're going to work in.
exec holos generate platform v1alpha5 --force
# Platforms are empty by default.
exec holos render platform ./platform
stderr -count=1 '^rendered platform'
# When author.#Kubernetes is empty
exec holos cue export --expression holos --out=yaml ./components/empty
cmp stdout want.txt
-- components/empty/empty.cue --
package holos
Kubernetes: #Kubernetes & {}
holos: Kubernetes.BuildPlan
-- want.txt --
kind: BuildPlan
apiVersion: v1alpha5
metadata:
name: no-name
spec:
artifacts:
- artifact: components/no-name/no-name.gen.yaml
generators:
- kind: Resources
output: resources.gen.yaml
resources: {}
transformers:
- kind: Kustomize
inputs:
- resources.gen.yaml
output: components/no-name/no-name.gen.yaml
kustomize:
kustomization:
labels:
- includeSelectors: false
pairs: {}
resources:
- resources.gen.yaml
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
source:
component:
name: no-name
path: no-path
parameters: {}

View File

@@ -2,8 +2,12 @@ import DocCardList from '@theme/DocCardList';
# Guides
## Start with the [Quickstart] guide
## Technical Overview
Please see the [Technical Overview] to learn about Holos. If you're ready to
drive in and try Holos, please work through the following guides.
## Bank of Holos
The guides are organized as a progression. We'll use Holos to manage a
fictional bank's platform, the Bank of Holos in each of the guides. In doing so
we'll take the time to explain the foundational concepts of Holos.
@@ -22,3 +26,4 @@ deployed services safely and consistently.
[Quickstart]: /docs/quickstart/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Change a Service]: /docs/guides/change-a-service/
[Technical Overview]: /docs/technical-overview/

View File

@@ -0,0 +1,735 @@
---
description: Change a service on your platform.
slug: /guides/change-a-service
sidebar_position: 300
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Change a Service
In this guide, we'll explore how Holos supports the frontend development team at [Bank of Holos] in reconfiguring an already deployed service. Along the way, we'll demonstrate how simple configuration changes are made safer with type checking, and how rendering the complete platform provides clear visibility into those changes.
This guide builds on the concepts covered in the [Quickstart] and [Deploy a Service] guides.
## What you'll need {#requirements}
Like our other guides, this guide is intended to be useful without needing to
run each command. If you'd like to apply the manifests to a real Cluster,
complete the [Local Cluster Guide](/docs/guides/local-cluster) before this
guide.
You'll need the following tools installed to run the commands in this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap Helm charts.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Holos
Components that render with Kustomize.
## Fork the Git Repository
If you haven't already done so, [fork the Bank of
Holos](https://github.com/holos-run/bank-of-holos/fork) then clone the
repository to your local machine.
<Tabs groupId="git-clone">
<TabItem value="command" label="Command">
```bash
# Change YourName
git clone https://github.com/YourName/bank-of-holos
cd bank-of-holos
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Cloning into 'bank-of-holos'...
remote: Enumerating objects: 1177, done.
remote: Counting objects: 100% (1177/1177), done.
remote: Compressing objects: 100% (558/558), done.
remote: Total 1177 (delta 394), reused 1084 (delta 303), pack-reused 0 (from 0)
Receiving objects: 100% (1177/1177), 2.89 MiB | 6.07 MiB/s, done.
Resolving deltas: 100% (394/394), done.
```
</TabItem>
</Tabs>
Run the rest of the commands in this guide from the root of the repository.
If you plan to apply the changes we make, you can delete and re-create your
local platform synced to the start of this guide.
```bash
./scripts/reset-cluster
./scripts/apply
```
## Rename the Bank
Let's imagine the bank recently re-branded from The Bank of Holos to The
Holistic Bank. The software development team responsible for the front end
website needs to update the branding accordingly.
Let's explore how Holos catches errors early, before they land in production,
then guides the team to the best place to make a change.
The bank front end web service is managed by the
`projects/bank-of-holos/frontend/components/bank-frontend/` component which
refers to the organization display name in `schema.gen.cue`.
<Tabs groupId="F5B546EB-566F-4B83-84C3-C55B40F55555">
<TabItem value="schema.cue" label="schema.cue">
```cue showLineNumbers
package holos
import api "github.com/holos-run/holos/api/author/v1alpha4"
// Define the default organization name.
_Organization: api.#OrganizationStrict & {
DisplayName: string | *"Bank of Holos"
Name: string | *"bank-of-holos"
Domain: string | *"holos.localhost"
}
// Projects represents a way to organize components into projects with owners.
// https://holos.run/docs/api/author/v1alpha4/#Projects
_Projects: api.#Projects
// ArgoConfig represents the configuration of ArgoCD Application resources for
// each component.
// https://holos.run/docs/api/author/v1alpha4/#ArgoConfig
_ArgoConfig: api.#ArgoConfig
#ComponentConfig: api.#ComponentConfig & {
Name: _Tags.name
Component: _Tags.component
Cluster: _Tags.cluster
ArgoConfig: _ArgoConfig & {
if _Tags.project != "no-project" {
AppProject: _Tags.project
}
}
Resources: #Resources
// Mix in project labels if the project is defined by the platform.
if _Tags.project != "no-project" {
CommonLabels: _Projects[_Tags.project].CommonLabels
}
}
// https://holos.run/docs/api/author/v1alpha4/#Kubernetes
#Kubernetes: close({
#ComponentConfig
api.#Kubernetes
})
// https://holos.run/docs/api/author/v1alpha4/#Kustomize
#Kustomize: close({
#ComponentConfig
api.#Kustomize
})
// https://holos.run/docs/api/author/v1alpha4/#Helm
#Helm: close({
#ComponentConfig
api.#Helm
})
```
</TabItem>
<TabItem value="projects/bank-of-holos/frontend/components/bank-frontend/bank-frontend.cue" label="projects/bank-of-holos/frontend/components/bank-frontend/bank-frontend.cue">
```cue showLineNumbers
package holos
// Produce a kubernetes objects build plan.
(#Kubernetes & Objects).BuildPlan
let Objects = {
Name: "bank-frontend"
Namespace: _BankOfHolos.Frontend.Namespace
// Ensure resources go in the correct namespace
Resources: [_]: [_]: metadata: namespace: Namespace
// https://github.com/GoogleCloudPlatform/bank-of-anthos/blob/release/v0.6.5/kubernetes-manifests/frontend.yaml
Resources: {
Service: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
_ports: http: {
name: "http"
port: 80
targetPort: 8080
protocol: "TCP"
}
ports: [for x in _ports {x}]
}
}
Deployment: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: matchLabels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
template: {
metadata: labels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
securityContext: {
seccompProfile: type: "RuntimeDefault"
fsGroup: 1000
runAsGroup: 1000
runAsNonRoot: true
runAsUser: 1000
}
serviceAccountName: "bank-of-holos"
terminationGracePeriodSeconds: 5
containers: [{
env: [{
name: "BANK_NAME"
value: _Organization.DisplayName
}, {
name: "ENV_PLATFORM"
value: "local"
}, {
name: "VERSION"
value: "v0.6.5"
}, {
name: "PORT"
value: "8080"
}, {
name: "ENABLE_TRACING"
value: "false"
}, {
name: "SCHEME"
value: "https"
}, {
name: "LOG_LEVEL"
value: "info"
}, {
name: "DEFAULT_USERNAME"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_USERNAME"
name: "demo-data-config"
}
}, {
name: "DEFAULT_PASSWORD"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_PASSWORD"
name: "demo-data-config"
}
}, {
name: "REGISTERED_OAUTH_CLIENT_ID"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_CLIENT_ID"
name: "oauth-config"
optional: true
}
}, {
name: "ALLOWED_OAUTH_REDIRECT_URI"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_REDIRECT_URI"
name: "oauth-config"
optional: true
}
}]
envFrom: [{
configMapRef: name: "environment-config"
}, {
configMapRef: name: "service-api-config"
}]
image: "us-central1-docker.pkg.dev/bank-of-anthos-ci/bank-of-anthos/frontend:v0.6.5@sha256:d72050f70d12383e4434ad04d189b681dc625f696087ddf0b5df641645c9dafa"
livenessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 60
periodSeconds: 15
timeoutSeconds: 30
}
name: "front"
readinessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 10
}
resources: {
limits: {
cpu: "250m"
memory: "128Mi"
}
requests: {
cpu: "100m"
memory: "64Mi"
}
}
securityContext: {
allowPrivilegeEscalation: false
capabilities: drop: ["all"]
privileged: false
readOnlyRootFilesystem: true
}
volumeMounts: [{
mountPath: "/tmp"
name: "tmp"
}, {
mountPath: "/tmp/.ssh"
name: "publickey"
readOnly: true
}]
}]
volumes: [
{
emptyDir: {}
name: "tmp"
},
{
name: "publickey"
secret: {
items: [{key: "jwtRS256.key.pub", path: "publickey"}]
secretName: "jwt-key"
}
},
]
}
}
}
}
// Allow HTTPRoutes in the ingress gateway namespace to reference Services
// in this namespace.
ReferenceGrant: grant: _ReferenceGrant & {
metadata: namespace: Namespace
}
// Include shared resources
_BankOfHolos.Resources
}
}
```
</TabItem>
</Tabs>
Line 7 of the `schema.cue` file defines the _default_ value for
`_Organization.DisplayName` by using `string | *"..."`. In CUE, the `*`
asterisk character denotes a [default value].
Line 78 of the `bank-frontend.cue` file refers to `_Organization.DisplayName` to
configure the front end web container.
Let's change the name of the bank by defining a new value for
`_Organization.DisplayName` at the root of the configuration. Create
`projects/organization.cue` with the following content.
<Tabs groupId="B386181F-EBE7-469D-8CB5-37631067669B">
<TabItem value="projects/organization.cue" label="projects/organization.cue">
```cue showLineNumbers
package holos
_Organization: DisplayName: "The Holistic-Bank"
```
</TabItem>
</Tabs>
Let's render the platform and see if this changes the name.
<Tabs groupId="A014333C-3271-4C22-87E6-2B7BF898EA3E">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
could not run: could not marshal json projects/platform/components/istio/cni: cue: marshal error: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/builder.go:63
_Organization.DisplayName: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors)
could not run: could not marshal json projects/platform/components/argocd/crds: cue: marshal error: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/builder.go:63
_Organization.DisplayName: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors)
could not run: could not render component: exit status 1 at builder/v1alpha4/builder.go:95
```
</TabItem>
</Tabs>
:::warning Whoops
The development team defined a value that isn't allowed by the
configuration.
:::
Someone else in the organization placed a [constraint] on the
configuration to ensure the display name contains only letters, numbers, and
spaces. This constraint is expressed as a [regular expression].
:::tip
CUE provides clear visibility where to start looking to resolve conflicts. Each
file and line number listed is a place the `#Organization.DisplayName` field is
defined.
:::
Let's try again, this time replacing the hyphen with a space.
<Tabs groupId="F93B34FA-C0C6-4793-A32F-DAD094403208">
<TabItem value="projects/organization.cue" label="projects/organization.cue">
```cue showLineNumbers
package holos
_Organization: DisplayName: "The Holistic Bank"
```
</TabItem>
</Tabs>
<Tabs groupId="5FD68778-476A-4F82-8817-71CEE205216E">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered bank-ledger-db for cluster workload in 139.863625ms
rendered bank-accounts-db for cluster workload in 151.74875ms
rendered bank-balance-reader for cluster workload in 154.356083ms
rendered bank-ledger-writer for cluster workload in 161.209541ms
rendered bank-userservice for cluster workload in 163.373417ms
rendered bank-backend-config for cluster workload in 179.271208ms
rendered bank-secrets for cluster workload in 204.35625ms
rendered gateway for cluster workload in 118.707583ms
rendered httproutes for cluster workload in 140.981541ms
rendered bank-transaction-history for cluster workload in 156.066875ms
rendered bank-frontend for cluster workload in 300.102292ms
rendered bank-contacts for cluster workload in 159.89625ms
rendered cni for cluster workload in 150.754458ms
rendered istiod for cluster workload in 222.922625ms
rendered app-projects for cluster workload in 118.422792ms
rendered ztunnel for cluster workload in 142.840625ms
rendered cert-manager for cluster workload in 190.938834ms
rendered base for cluster workload in 340.679416ms
rendered local-ca for cluster workload in 107.120334ms
rendered external-secrets for cluster workload in 145.020834ms
rendered argocd for cluster workload in 299.690917ms
rendered namespaces for cluster workload in 115.862334ms
rendered gateway-api for cluster workload in 225.783833ms
rendered external-secrets-crds for cluster workload in 339.741166ms
rendered crds for cluster workload in 421.849041ms
rendered platform in 718.015959ms
```
</TabItem>
</Tabs>
:::tip Success
Great, the platform rendered. We know the display name is valid according to
the constraints.
:::
Let's see if the new display name value updated the configuration for the bank
frontend.
<Tabs groupId="6C068651-2061-4262-BE1E-7BB3E7EB66CB">
<TabItem value="command" label="Command">
```bash
git status
```
</TabItem>
<TabItem value="output" label="Output">
```txt
On branch main
Your branch and 'jeffmccune/main' have diverged,
and have 2 and 4 different commits each, respectively.
(use "git pull" to merge the remote branch into yours)
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
// highlight-next-line
modified: deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
modified: deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
Untracked files:
(use "git add <file>..." to include in what will be committed)
projects/organization.cue
no changes added to commit (use "git add" and/or "git commit -a")
```
</TabItem>
</Tabs>
<Tabs groupId="4A20831E-461B-4EDE-8F6E-E73C3AEC12DB">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml b/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
index 7914756..250c660 100644
--- a/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
+++ b/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
@@ -9,7 +9,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
@@ -26,7 +26,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
@@ -43,7 +43,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
@@ -60,7 +60,7 @@ spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
- description: Holos managed AppProject for Bank of Holos
+ description: Holos managed AppProject for The Holistic Bank
destinations:
- namespace: '*'
server: '*'
diff --git a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
index dae6f93..d41516b 100644
--- a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
+++ b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
@@ -71,7 +71,7 @@ spec:
containers:
- env:
- name: BANK_NAME
- value: Bank of Holos
+ value: The Holistic Bank
- name: ENV_PLATFORM
value: local
- name: VERSION
```
</TabItem>
</Tabs>
:::danger
The new display name changed the frontend container, but it _also_ affected the
app-projects component owned by the platform team.
:::
Submitting a pull request would trigger a code review from the platform
engineering team who manages the app-projects component. Let's see how to
narrow the change down to limit the scope to the bank's user facing services.
All of these services are managed under `projects/bank-of-holos/` Move the
`organization.cue` file into this folder to limit the scope of configuration to
the the components contained within.
```bash
mv projects/organization.cue projects/bank-of-holos/
```
Render the platform and let's see what changed.
<Tabs groupId="0FFEC244-B59B-4136-9C82-837985DC2AB8">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered bank-ledger-db for cluster workload in 163.814917ms
rendered bank-accounts-db for cluster workload in 163.960208ms
rendered bank-userservice for cluster workload in 164.1625ms
rendered bank-ledger-writer for cluster workload in 169.185291ms
rendered bank-balance-reader for cluster workload in 174.5455ms
rendered bank-backend-config for cluster workload in 178.092125ms
rendered bank-secrets for cluster workload in 202.305334ms
rendered gateway for cluster workload in 122.81725ms
rendered httproutes for cluster workload in 134.121084ms
rendered bank-contacts for cluster workload in 146.4185ms
rendered bank-frontend for cluster workload in 311.35425ms
rendered bank-transaction-history for cluster workload in 160.103ms
rendered cni for cluster workload in 145.762083ms
rendered istiod for cluster workload in 216.0065ms
rendered app-projects for cluster workload in 117.684333ms
rendered ztunnel for cluster workload in 144.555292ms
rendered cert-manager for cluster workload in 178.247917ms
rendered base for cluster workload in 336.679ms
rendered external-secrets for cluster workload in 142.21825ms
rendered local-ca for cluster workload in 101.249ms
rendered argocd for cluster workload in 280.54525ms
rendered namespaces for cluster workload in 106.822042ms
rendered gateway-api for cluster workload in 200.459791ms
rendered external-secrets-crds for cluster workload in 470.125833ms
rendered crds for cluster workload in 844.388666ms
rendered platform in 1.154937084s
```
</TabItem>
</Tabs>
<Tabs groupId="DE4FEEE5-FC53-48A6-BC6F-D0EA1DBFD00C">
<TabItem value="command" label="Command">
```bash
git diff
```
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
index dae6f93..d41516b 100644
--- a/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
+++ b/deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
@@ -71,7 +71,7 @@ spec:
containers:
- env:
- name: BANK_NAME
- value: Bank of Holos
+ value: The Holistic Bank
- name: ENV_PLATFORM
value: local
- name: VERSION
```
</TabItem>
</Tabs>
:::tip Success
Great! This time, the only manifest affected is our `bank-frontend.gen.yaml`.
:::
The `BANK_NAME` environment variable will change as we expect, and only the dev
teams managing the bank services components are affected by the change.
Let's commit and push this change and see if it works.
<Tabs groupId="435D9C60-F841-4CF1-A947-506422E6BAC9">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m 'frontend: rename bank to The Holistic Bank'
git push
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main fda74ec] frontend: rename bank to The Holistic Bank
2 files changed, 4 insertions(+), 1 deletion(-)
create mode 100644 projects/bank-of-holos/organization.cue
```
</TabItem>
</Tabs>
Now that we've pushed the change, let's apply the change to the platform.
## Apply the Change
Once we've pushed the change, navigate to the [bank-frontend GitOps
Application](https://argocd.holos.localhost/applications/argocd/bank-frontend?view=tree&resource=).
We can see the Deployment needs to sync to the desired state we just pushed.
![bank-frontend out of sync](./img/change-a-service-out-of-sync.png)
Clicking on the frontend Deployment, we see the diff with the change we expect.
![bank-frontend diff](./img/change-a-service-diff.png)
Sync the change, ArgoCD applies the desired configuration state to the cluster
and Kubernetes handles rolling out the updated Deployment resource.
![bank-frontend progressing](./img/change-a-service-progressing.png)
Soon, the deployment finishes and the component is in sync again.
![bank-frontend in sync](./img/change-a-service-in-sync.png)
Finally, let's see if the name actually changed on the website. Navigate to
https://bank.holos.localhost/.
![bank-frontend login page](./img/change-a-service-login-page.png)
:::tip Success
We successfully made our change and successfully applied the changed
configuration to the platform.
:::
Thanks for taking the time to work through this guide which covered:
- How multiple teams could be impacted by defining configuration at the
`projects/` path.
- How to scope our change to only affect components within the
`projects/bank-of-holos/` path, eliminating the impact on other teams.
- How CUE can [constrain] values in Holos, increasing safety.
- How to handle a [default value] in CUE.
- How CUE surfaces the file and line number of _every_ place to look for where a
value is defined, making it faster and easier to troubleshoot problems.
[Quickstart]: /docs/quickstart/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Change a Service]: /docs/guides/change-a-service/
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize
[ComponentFields]: /docs/api/author/v1alpha3/#ComponentFields
[platform-files]: /docs/quickstart/#how-platform-rendering-works
[AppProject]: https://argo-cd.readthedocs.io/en/stable/user-guide/projects/
[unification operator]: https://cuelang.org/docs/reference/spec/#unification
[code-owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[Kustomization API]: https://github.com/kubernetes-sigs/kustomize/blob/release-kustomize-v5.2/api/types/kustomization.go#L34
[cue import]: https://cuelang.org/docs/reference/command/cue-help-import/
[cue get go]: https://cuelang.org/docs/concept/how-cue-works-with-go/
[timoni-crds]: https://timoni.sh/cue/module/custom-resources/
[HTTPRoute]: https://gateway-api.sigs.k8s.io/api-types/httproute/?h=filter
[Ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
[hidden field]: https://cuelang.org/docs/tour/references/hidden/
[comprehension]: https://cuelang.org/docs/reference/spec/#comprehensions
[code owners]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[ReferenceGrant]: https://gateway-api.sigs.k8s.io/api-types/referencegrant/
[Local Cluster Guide]: /docs/guides/local-cluster
[Bank of Holos]: https://github.com/holos-run/bank-of-holos
[default value]: https://cuelang.org/docs/tour/types/defaults/
[constrain]: https://cuelang.org/docs/tour/basics/constraints/
[constraint]: https://cuelang.org/docs/tour/basics/constraints/
[regular expression]: https://cuelang.org/docs/tour/expressions/regexp/

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

Before

Width:  |  Height:  |  Size: 690 KiB

After

Width:  |  Height:  |  Size: 690 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 997 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1009 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 617 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 706 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 794 KiB

File diff suppressed because it is too large Load Diff

View File

Before

Width:  |  Height:  |  Size: 934 KiB

After

Width:  |  Height:  |  Size: 934 KiB

View File

Before

Width:  |  Height:  |  Size: 703 KiB

After

Width:  |  Height:  |  Size: 703 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 728 KiB

After

Width:  |  Height:  |  Size: 728 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 854 KiB

After

Width:  |  Height:  |  Size: 854 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 624 KiB

After

Width:  |  Height:  |  Size: 624 KiB

View File

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 116 KiB

View File

@@ -0,0 +1,52 @@
---
description: Introduction
---
# Introduction
Welcome to Holos. Holos is an open source tool to manage software development
platforms safely, easily, and consistently. We built Holos to help engineering
teams work more efficiently together by empowering them to build golden paths
and paved roads for other teams to leverage for quicker delivery.
## Documentation
- [Guides] are organized into example use-cases of how Holos helps engineering
teams at the fictional Bank of Holos deliver business value on the bank's
platform.
- The [API Reference] is a technical reference for when you're writing CUE code to define your platform.
## Backstory
At [Open Infrastructure Services], we've each helped dozens of companies build and operate their software development platforms. During the U.S. presidential election just before the pandemic, our second-largest client, Twitter, experienced a global outage that lasted nearly a full day. We were managing their production configuration system, allowing the core infrastructure team to focus on business-critical objectives. This gave us a front-row seat to the incident.
A close friend and engineer on the team made a trivial one-line change to the firewall configuration. Less than 30 minutes later, everything was down. That change, which passed code review, caused the host firewall to revert to its default state on hundreds of thousands of servers, blocking all connections globally—except for SSH, thankfully. Even a Presidential candidate complained loudly.
This incident forced us to reconsider key issues with Twitter's platform:
1. **Lack of Visibility** - Engineers couldn't foresee the impact of even a small change, making it difficult to assess risks.
2. **Large Blast Radius** - Small changes affected the entire global fleet in under 30 minutes. There was no way to limit the impact of a single change.
3. **Incomplete Tooling** - The right processes were in place, but the tooling didn't fully support them. The change was tested and reviewed, but critical information wasn't surfaced in time.
Over the next few years, we built features to address these issues. Meanwhile, I began exploring how these solutions could work in the Kubernetes and cloud-native space.
As Google Cloud partners, we worked with large customers to understand how they built their platforms on Kubernetes. During the pandemic, we built a platform using CNCF projects like ArgoCD, Prometheus Stack, Istio, Cert Manager, and External Secrets Operator, integrating them into a cohesive platform. We started with upstream recommendations—primarily Helm charts—and wrote scripts to integrate each piece into the platform. For example, we passed Helm outputs to Kustomize to add labels or fix bugs, and wrote umbrella charts to add Ingress, HTTPRoute, and ExternalSecret resources.
These scripts served as necessary glue to hold everything together but became difficult to manage across multiple environments, regions, and cloud providers. YAML templates and nested loops created friction, making them hard to troubleshoot. The scripts themselves made it difficult to see what was happening and to fix issues affecting the entire platform.
Still, the scripts had a key advantage: they produced fully rendered manifests in plain text, committed to version control, and applied via ArgoCD. This clarity made troubleshooting easier and reduced errors in production.
Despite the makeshift nature of the scripts, I kept thinking about the "[Why are we templating YAML]?" post on Hacker News. I wanted to replace our scripts and charts with something more robust and easier to maintain—something that addressed Twitter's issues head-on.
I rewrote our scripts and charts using CUE and Go, replacing the glue layer. The result is **Holos**—a tool designed to complement Helm, Kustomize, and Jsonnet, making it easier and safer to define golden paths and paved roads without bespoke scripts or templates.
Thanks for reading. Take Holos for a spin on your local machine with our [Quickstart] guide.
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[CUE]: https://cuelang.org/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story

View File

@@ -42,9 +42,9 @@ graph TB
Kustomize[<a href="#component">Kustomize</a>]
CUE[<a href="#component">CUE</a>]
Cluster --> Platform
Fleet --> Cluster
Component --> Fleet
Fleet --> Platform
Cluster --> Fleet
Component --> Cluster
Helm --> Component
Kustomize --> Component
CUE --> Component
@@ -324,28 +324,28 @@ ArgoCD, or Flux.
```mermaid
---
title: Figure 2 - Render Pipeline
title: Figure 1 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/core/v1alpha2#PlatformSpec">PlatformSpec</a>]
BP[<a href="/docs/api/core/v1alpha2#BuildPlan">BuildPlan</a>]
HC[<a href="/docs/api/core/v1alpha2#HolosComponent">Components</a>]
PS[<a href="/docs/api/author/v1alpha3/#Platform">Platform</a>]
HC[<a href="/docs/api/author/v1alpha3/#ComponentFields">Components</a>]
BP[<a href="/docs/api/core/v1alpha3#BuildPlan">BuildPlan</a>]
H[<a href="/docs/api/core/v1alpha2#HelmChart">HelmChart</a>]
K[<a href="/docs/api/core/v1alpha2#KustomizeBuild">KustomizeBuild</a>]
O[<a href="/docs/api/core/v1alpha2#KubernetesObjects">KubernetesObjects</a>]
H[<a href="/docs/api/author/v1alpha3/#Helm">Helm</a>]
K[<a href="/docs/api/author/v1alpha3/#Kustomize">Kustomize</a>]
O[<a href="/docs/api/author/v1alpha3/#Kubernetes">Kubernetes</a>]
P[<a href="/docs/api/core/v1alpha2#Kustomize">Kustomize</a>]
P[<a href="/docs/api/core/v1alpha3#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
C[Kube API Server]
PS --> BP --> HC
HC --> H --> P
HC --> K --> P
HC --> O --> P
PS --> HC --> BP
BP --> H --> P
BP --> K --> P
BP --> O --> P
P --> Y --> FS
P --> G --> FS

Some files were not shown because too many files have changed in this diff Show More