Compare commits

..

77 Commits

Author SHA1 Message Date
Gary Larizza
488f7ae985 Start Hello Holos
This is a very, very minimal framework using the Tokyo docs as a guide.
This code is going to have to change when `v1alpha5` is published, so
I'm just dropping it in here as a placeholder.

Like the Tokyo docs, I was intening to setup the code and commands at the
start, and then have a "Breaking it down" section later explaining all
the moving parts.
2024-11-05 16:27:10 -08:00
Gary Larizza
4817491aab Update groupId for uniqueness 2024-11-05 16:26:40 -08:00
Gary Larizza
30ce72a9b7 Add the Local Cluster guide 2024-11-05 14:50:16 -08:00
Gary Larizza
958f65ddcf Setup page first draft 2024-11-05 14:47:12 -08:00
Gary Larizza
a1c9111a05 Overview initial draft 2024-11-05 13:30:35 -08:00
Jeff McCune
ee30c52673 docs: generate version specific api docs (#303)
Without this patch each version of the core and author schemas are
duplicated into each docs version.  This is unnecessary and difficult to
maintain now that we have docusaurus versioned docs enabled.

This patch updates the schema generation script to check if the docs
version has been released, and if so write into a markdown file in the
versioned docs folder.  If not, the version is written into the next
version folder.

This patch also updates some, but not all, document links to the md or
mdx relative file paths.  This is necessary to generate the correct
versioned links.

A nice outcome of this change is that technical docs no longer need to
link to version specific pages.  For example, `[Core Schema]:
./api/core.md` will always refer to the correct auto generated docs
associated with the docs version.
2024-11-05 07:20:53 -08:00
Jeff McCune
117a00334f docs: restructure docs into tutorial and topics (#301)
The docs for v1alpha4 have the right information, but in the wrong
places.  The most important bits are tucked away in the Core API docs.
One of our first users entirely missed the `holos generate platform`
command mentioned in the Helm guide.

We'll fix this by organizing the docs into two distinct categories.
First, a tutorial written as a series progressively building up the
minimum knowledge to use holos effectively and gain the benefits.  Think
of it as a tour of the essential bits.

The second category are focused topics which stand alone.  They're the
things most people using holos will need to know eventually, but aren't
essential for everyone to know.  For example, Clusters and Fleets will
move from the Author API to stand alone examples of how to implement
these features if necessary.

Then there's a Glossary which serves as the place to describe our
concepts and domain specific language.

Finally there's the API documentation which should be cut down to the
specific version.  The next release version will be v1alpha5.

Attribution: We're copying the Tokio docs structure, it's concise and a
similar size and complexity to our own project.

The Go docs are also an inspiration, but the project is much larger so
not directly comparable.  The organization of https://go.dev/doc/ feels
complete at first glance, despite the size and age of the project.  The
site also makes clear who each section is for without needing to come
right out and say it. Getting started, Using and understanding Go,
Writing modules, using databases, etc...
2024-11-04 20:25:04 -08:00
Jeff McCune
1e03debfac tests: add make unity target
For https://cuelabs.dev/unity/
2024-11-04 19:08:44 -08:00
Jeff McCune
72137b2fa9 docs: upgrade docusaurus to 3.6.0
npm i @docusaurus/core@latest @docusaurus/plugin-client-redirects@latest \
  @docusaurus/preset-classic@latest @docusaurus/theme-mermaid@latest \
  @docusaurus/module-type-aliases@latest @docusaurus/tsconfig@latest \
  @docusaurus/types@latest
2024-11-04 06:47:48 -08:00
Jeff McCune
5abf967116 docs: npm run docusaurus docs:version v1alpha4 (#299)
Tag version v1alpha4 so we can start working on v1alpha5 as the next
version in main.
2024-11-04 06:43:15 -08:00
Jeff McCune
5d882f465d website: fix resources.yaml tab in helm guide (#293)
We switched from using a kustomize remote base to a local file so the
tests don't need to make a network round trip to github.  It's also
better practice to use local files for this sort of thing.

In doing so I botched the location of the file, putting it in the
platform registration section.  This patch claifies how `resources.yaml`
is linked to `httpbin.cue` through the `KustomizeConfig: Files:
"resources.yaml": _` field.
2024-11-03 10:57:52 -08:00
Jeff McCune
45bdaac833 main: cue v0.10.1 and add e2e test for helm guide (#293)
Previously there was no test coverage of the
https://holos.run/docs/guides/helm/ guide.  This patch uses Roger's
testscript package, which the CUE folks also use to add comprehensive
test coverage of each step in the guide.  Ideally we would execute these
commands directly from the guide itself, but for now we'll duplicate the
commands into the test script.  This could be enhanced by generating the
test script from the document itself in some way.

When updating the script, use the `holos txtar` command to embed entire
helm charts into the test script.  It's not super fast, but it's better
than network access and it's not terribly slow either.  A few seconds to
unpack.

---

txtar: quote files for testscript unquote

For the helm guide test script we want to include the entire helm chart
which may have files that need to be quoted.  This patch changes the
default behavior of the holos txtar command to quote files if necessary
and list them in an unquote script command in the comment of the
archive.

The purpose is for testscript authors to copy and paste the entire thing
into a test script and include the unquote command at the top.

---

This change also updates CUE to v0.10.1
2024-11-03 10:27:46 -08:00
Jeff McCune
7ae1f990ef website: update quickstart diagram to match helm
Avoid confusion, got a question about this in discord.
2024-11-03 08:53:43 -08:00
Jeff McCune
b526fd1669 testdata: clean up old v1alpha1 tests (#292)
No longer necessary now that we're on v1alpha4.  Test coverage for
v1alpha4 and the user facing guides will be added back soon for use both
in the holos repo and in Unity.
2024-11-01 15:22:13 -07:00
Jeff McCune
5e07655f35 website: fix port in helm guide
Should be 9115 not 6115.
2024-11-01 06:39:48 -07:00
Jeff McCune
6fb6afe8d5 v0.97.3 2024-10-31 21:04:22 -07:00
Jeff McCune
d6f89052d9 website: update helm guide to apply patches (#291)
Updated the helm guide to apply patches while still showing the diff in
the documentation markdown.  The only gotcha is it creates orig files.
2024-10-31 20:54:57 -07:00
Jeff McCune
e4aa7f5994 website: update change-a-service to use hidden fields (#291)
Use _Foo instead of #Foo to hold concrete values.
2024-10-31 20:25:40 -07:00
Jeff McCune
6e4c65cb6c website: update deploy-a-service to use hidden fields (#291)
Use _Foo instead of #Foo to hold concrete values.
2024-10-31 20:13:17 -07:00
Jeff McCune
4f091677e2 website: update quickstart for v1alpha4 hidden fields (#291) 2024-10-31 16:35:41 -07:00
Jeff McCune
0c05df1162 website: update technical overview with consistent fields (#291) 2024-10-31 11:30:20 -07:00
Jeff McCune
64a745fd34 v1alpha4: use hidden fields consistently (#291)
Previously it wasn't clear for users if platform wide structs should be
definitions or hidden fields in CUE.  They should be hidden fields when
they contain data and definitions when they define a schema.

This patch updates the generate platform v1alpha4 subcommand to use the
correct field names consistently for clarity.
2024-10-31 10:45:47 -07:00
Jeff McCune
490f91f580 cli: hide unsupported commands (#289)
Use a simple feature flag system that checks env vars if a feature is
enabled.
2024-10-31 10:04:01 -07:00
Jeff McCune
79b065cda8 website: add open graph image for helm guide try 6 2024-10-30 12:23:18 -07:00
Jeff McCune
0fa6047552 website: add open graph image for helm guide try 5 2024-10-30 12:20:19 -07:00
Jeff McCune
11ecc0cc3a website: add open graph image for helm guide try 4 2024-10-30 12:04:39 -07:00
Jeff McCune
a62e4ba117 website: add open graph image for helm guide try 3 2024-10-30 11:56:01 -07:00
Jeff McCune
07fe667f30 website: add open graph image for helm guide try 2 2024-10-30 11:40:39 -07:00
Jeff McCune
3ad994cbb9 website: add open graph image for helm guide 2024-10-30 11:26:54 -07:00
Jeff McCune
b3d9bd32af website: add why cue for configuration blog post
This is going to be one of the first questions we get.
2024-10-28 21:30:11 -07:00
Jeff McCune
d398b49d7f website: fix head title tag try 2
The open graph title was still showing up poorly, docusaurus generates
it with the Holos | Holos repetition, so we need to override it.
2024-10-28 14:37:43 -07:00
Jeff McCune
12179a6991 website: fix head title tag and social card
Generate the social card manually from https://www.opengraph.xyz/
Override the page title tag, otherwise it shows up as "Announcing Holos
| Holos" in social links, which is weird.
2024-10-28 14:07:40 -07:00
Jeff McCune
fee472bb66 website: add stock social card for annoucement 2024-10-28 13:31:10 -07:00
Jeff McCune
c6a13059f3 v0.97.1 2024-10-28 11:12:55 -07:00
Jeff McCune
ff3eb896f3 webite: put ois logo back
Until we get a better logo.
2024-10-28 10:46:59 -07:00
Jeff McCune
70f70ae6b9 website: fix launch announcement 2024-10-28 10:45:43 -07:00
Jeff McCune
2580ec1c5f website: fix order of api references
The api references are in reverse order and don't have good descriptions
in the index listings.  This patch adds front matter to each generated
document to order them correctly and add a nice description.
2024-10-27 20:43:54 -07:00
Jeff McCune
4fa99e0faa website: add helm prometheus blackbox httpbin guide
The purpose of this guide is to demonstrate how holos offers value
above and beyond helm and kustomize alone.
2024-10-27 19:48:42 -07:00
Jeff McCune
7341d25483 website: add at proto did for bsky 2024-10-25 14:26:42 -07:00
Jeff McCune
3074b3a241 website: add discord invite link 2024-10-24 10:27:37 -07:00
Jeff McCune
9a5e7869c6 v1alpha4: omit the platform model if empty
The platform model distracts from getting started:

  cue export --out yaml ./platform

  kind: Platform
  apiVersion: v1alpha4
  metadata:
    name: default
  spec:
    components:
      - name: prometheus
        component: projects/platform/components/prometheus
        cluster: local
        model: {}

With this patch it's absent by default.
2024-10-23 13:46:58 -07:00
Jeff McCune
1064ceba31 v1alpha4: manage a single workload cluster named local by default
Manage a single cluster by default after generating the platform.
Assume it's a local cluster for use with the guides.
2024-10-23 13:42:46 -07:00
Jeff McCune
4bccaa3710 v1alpha4: _Platform not #Platform for component registration
`_Platform` is a hidden field representing the platform components,
`#Platform` defines the schema of the hidden field.
2024-10-23 13:33:28 -07:00
Jeff McCune
95efae1343 docs: update rendered manifest figure in technical overview
It's too wide with KubeAPI, take it out.
2024-10-20 10:11:57 -07:00
Jeff McCune
ba88125877 v1alpha4: enable config map generator for Kustomize
Without this patch it's difficult to mix in a plain file as a config
map.  This is necessary for the use case of using a Job to generate a
secret in-cluster.  We want a plain shell script to be carried through
and transformed into the job.

We already have the KustomizeConfig fields to support this, they just
weren't wired up to the #Kustomization component kind.

I didn't check if it's wired up to Helm and Kustomize for expedience.
They may be missing there as well.
2024-10-19 10:57:37 -07:00
Jeff McCune
d12c1a0c11 Merge pull request #284 from holos-run/gl/deploy-a-service-v4alpha-update
Update deploy-a-service guide for Author API v1alpha4
2024-10-18 20:35:51 -07:00
Jeff McCune
d56d3400a7 docs: replace tabs in deploy-a-service guide 2024-10-18 20:33:54 -07:00
Gary Larizza
4f0f9dced5 Update deploy-a-service guide for Author API v1alpha4
PROBLEM:

Version v1alpha4 of the Author API has been updated with backwards
incompatible changes, and the deploy-a-service guide uses code from
version v1alpha3.

SOLUTION:

Update any code, links, and data that is out of date, and then run
through the guide to make sure it works locally.

OUTCOME:

The instructions in the deploy-a-service guide will work successfully
with version v1alpha4 of the Author API.
2024-10-18 15:21:56 -07:00
Jeff McCune
6bf0cb8d8e v1alpha4: v0.97.0 2024-10-17 07:50:49 -07:00
Jeff McCune
766c8912b7 Merge pull request #281 from holos-run/jeff/280-authorapi-v1alpha4
v1alpha4 Author API
2024-10-17 07:17:14 -07:00
Jeff McCune
be1dee5f1c v1alpha4: update technical overview guide (#280)
Update the guide and the bank of holos repository to use v1alpha4
instead of v1alpha3.
2024-10-17 07:13:51 -07:00
Jeff McCune
6ad56525ac v1alpha4: refactor --tag to --inject and remove environment (#276)
Cue uses --inject, -t as the flags to set variables for fields tagged
using @tag(var,type=string).

We used --tag, which is different and requires a mental mapping.  Let's
use the same flag and also pass it multiple times like they require so
we can copy and paste the command line output from the debug logs into a
cue export command to see what's going on.

This patch deprecates the --cluster-name flag, use --inject
holos_cluster=mycluster instead.

This patch also removes the environment field from the Component core
API, leaving this to the user namespace to define via tags.  We don't
want to be too opinionated on how users manage their platform, baking
environment into the schema is a slippery slope toward those kinds of
opinions.

Closes: #276
2024-10-16 22:07:47 -07:00
Jeff McCune
791ec5ee71 v1alpha4: refactor core.Component Tags to map[string]string (#280)
Previously it was a []string slice that must be formatted as key=value.
This is more difficult to work with than a map[string]string.
2024-10-16 20:10:14 -07:00
Jeff McCune
638ac7473c fixup 2024-10-16 20:05:05 -07:00
Jeff McCune
ee24b5ce13 fixup 2024-10-16 20:04:18 -07:00
Jeff McCune
fa2fdbe4e8 fixup 2024-10-16 20:00:53 -07:00
Jeff McCune
63e1df1d4c v1alpha4: add common labels to projects schema (#280)
Now that we have CommonLabels as part of the ComponentConfig for all
components, it makes sense to also mix in CommonLabels for a Project.

Common labes are key aspect of the Technical Overview document.
2024-10-16 17:40:40 -07:00
Jeff McCune
2ad0c2a93e render: refactor tm to typeMeta
Gary and I noted tm wasn't clear when I was showing him code.
2024-10-16 17:11:09 -07:00
Jeff McCune
3a6a04f318 v1alpha4: add projects to author api (#280)
Projects are a key element of the Technical Overview guide, so we need
the schema for them in the Author API.
2024-10-16 12:29:35 -07:00
Jeff McCune
8afeece890 v1alpha4: embed ComponentConfig in Helm, Kustomize, Kubernetes (#280)
For the Author API, it would be nice to define a schema for the fields
common to all component kinds.  Users could then configure all kinds by
unifying the schema into their own platform tree.

This makes a clear use case to extract the common fields back into an
embedded struct like we did in v1alpha3.  I removed the embedded struct
in v1alpha4 because it wasn't clear why it should be separate, but now
the use case is clear, to configure all component kinds.
2024-10-16 12:16:48 -07:00
Jeff McCune
bc9c43a0b9 fix argocd application project 2024-10-15 20:52:56 -07:00
Jeff McCune
5a98c77e4c add argocd.argoproj.io/instance label to resources
But not the ArgoCD Application resource.
2024-10-15 20:41:39 -07:00
Jeff McCune
b3f7de39ec v1alpha4: feedback in case of chart cache dead lock (#280)
Without this patch holos render platform may hang until the overall
context timeout is reached.  This is a problem because the user has no
idea why it's hung.

This patch adds a warning at the 5 second and another at the 10 second
mark indicating the lock may be deadlocked.  The user can then remove
the directory.
2024-10-15 16:43:36 -07:00
Jeff McCune
ca4ecf1b28 v1alpha4: KustomizeConfig Resources and Files (#280)
The Kustomize build plan kind needs to support both copying files from
the component directory and pulling resources from https URL's.  Without
this patch this support is missing from the Author API

With this patch the Kustomize build plan kind has a KustomizeConfig
field with two structs, Files and Resources.  The kustomization
resources list is built up from both of these.

Two transformers are used so we don't affect the GitOps transfomer which
really only needs CommonLabels.

I decided to keep this field exclusive to the Kustomize kind, but it
could replace the Kustomization field of the other kinds as well.
2024-10-15 16:11:05 -07:00
Jeff McCune
9ce28660ce v1alpha4: intermediate kustomization (#280)
Without this patch the user facing API doesn't have a way to kustomize
the output of all the build plan kinds.  This patch ensures the
Kustomization field is present on all of Helm, Kustomize, and
Kubernetes.

This field is inteded for patches and transforms.  The second
kustomization in the transformer sequence is intended for common labels
and annotations, managed by a corresponding field instead of a full on
Kustomization resource.
2024-10-15 15:02:15 -07:00
Jeff McCune
728e8ba06e v1alpha4: default helm chart release to chart name (#280) 2024-10-15 14:47:42 -07:00
Jeff McCune
e4b07dad6d v1alpha4: helm enable hooks default false (#280)
Fix:

could not run: could not marshal json projects/platform/components/cert-manager: cue: marshal error: spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON at internal/builder/builder.go:63
spec.artifacts.0.generators.0.helm.enableHooks: cannot convert incomplete value "bool" to JSON:
    /Users/jeff/Holos/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha4/types_go_gen.cue:235:16
could not run: could not render component: exit status 1 at builder/v1alpha4/builder.go:94
2024-10-15 14:43:05 -07:00
Jeff McCune
b7c0bba2b9 v1alpha4: add resources to schema.cue (#280)
Without this patch kustomize cannot execute because resources from CUE
are incomplete.
2024-10-15 14:41:49 -07:00
Jeff McCune
847ab8441c v1alpha4: add default chart name (#280)
Defaults to the build plan name.
2024-10-15 14:40:36 -07:00
Jeff McCune
5f72af3d53 v1alpha4: link helm to the chart struct (#280) 2024-10-15 14:32:16 -07:00
Jeff McCune
33eed43fd1 v1alpha4: surface kustomize stderr output (#280)
Without this patch kustomize errors aren't surfaced when executing holos
render platform.

This patch gives a fighting chance to the user to figure out what's
going on.  The stderr is copied, logged, and surfaced up to the parent
holos render platform command.
2024-10-15 14:32:11 -07:00
Jeff McCune
d2fbbdd1cc logger: log the process pid (#280)
Useful when troubleshooting concurrent processes.
2024-10-15 14:31:42 -07:00
Jeff McCune
e42da118dc v1alpha4: add Helm and Kustomize to author api (#280)
Previously the #Helm and #Kustomize build plan helpers were not defined
in the v1alpha4 Author API.  We need this definition to update the
Quickstart guide for v1alpha4 from v1alpha3.

This patch defines the #Helm and #Kustomize helpers in the Author API
similar to how #Kubernetes is defined.
2024-10-15 10:32:16 -07:00
Jeff McCune
7d36567dcf v1alpha4: define Kubernetes in author api (#280)
Previously #Kubernetes was defined in the platform code.  This is a
problem because every platform engineer would need to copy and paste
this code.

This patch moves the #Kubernetes helper into the cue.mod directory so it
can be imported and used ergonomically.
2024-10-14 20:45:04 -07:00
Jeff McCune
bee698bebe v1alpha4: add platform to the author api (#280)
This patch gets the Author API rendering the namespaces component in the
Bank of Holos guide.  It's not the final form of the API yet, we still
need to decide how best to expose the Kubernetes, Helm, and Kustomize
definitions.

I'm thinking we abstract away the transformers and generators within the
Author API Kubernetes definition.
2024-10-14 17:19:49 -07:00
Jeff McCune
58df0626d0 v1alpha4: plumb --write-to flag from platform (#280)
Without this patch the --write-to flag can't be controlled from the
PlatformSpec in the CoreAPI.  We need to surface this for the ArgoConfig
struct in the AuthorAPI.

That is to say, in v1alpha3 the --write-to flag was previously assumed
to be deploy/ in ArgoConfig using the DeployFiles functionality.  We no
longer have DeployFiles in Core API v1alpha4, all artifacts are instead
written relative to the --write-to flag.  Still, we need to expose this
flag in the PlatformSpec so users can use something other than the
deploy directory.
2024-10-14 15:16:36 -07:00
Jeff McCune
c817a24704 v1alpha4: improve core api documentation 2024-10-10 20:37:38 -07:00
270 changed files with 41209 additions and 3909 deletions

View File

@@ -5,36 +5,60 @@
"mdx"
],
"words": [
"acmesolver",
"acraccesstokens",
"admissionregistration",
"alertmanager",
"alertmanagers",
"anthos",
"apiextensions",
"apimachinery",
"apiobjects",
"apiservers",
"applicationset",
"applicationsets",
"appproject",
"appprojects",
"argoproj",
"argumentless",
"authcode",
"authorizationpolicies",
"authorizationpolicy",
"authpolicy",
"authproxy",
"authroutes",
"automount",
"automounting",
"autoscaler",
"balancereader",
"blackbox",
"buildplan",
"builtinpluginloadingoptions",
"cachedir",
"cadvisor",
"cainjector",
"CAROOT",
"certificaterequest",
"certificaterequests",
"certificatesigningrequests",
"clientset",
"clsx",
"clusterexternalsecret",
"clusterexternalsecrets",
"clusterissuer",
"clusterissuers",
"clusterrole",
"clusterrolebinding",
"clustersecretstore",
"clustersecretstores",
"clusterwide",
"Cmds",
"CNCF",
"CODEOWNERS",
"configdir",
"configmap",
"configmapargs",
"connectrpc",
"cookiesecret",
"coredns",
"corev",
@@ -42,111 +66,191 @@
"crds",
"creds",
"crossplane",
"crunchydata",
"cuecontext",
"cuelang",
"customresourcedefinition",
"daemonset",
"deploymentruntimeconfig",
"destinationrule",
"destinationrules",
"devicecode",
"dnsmasq",
"dscacheutil",
"ecrauthorizationtokens",
"edns",
"endpointslices",
"entgo",
"envoyfilter",
"envoyfilters",
"errdetails",
"errgroup",
"etcdsnapshotfiles",
"externalsecret",
"externalsecrets",
"fctr",
"fieldmaskpb",
"fieldspec",
"flushcache",
"fullname",
"gatewayclass",
"gatewayclasses",
"gcraccesstokens",
"gendoc",
"generationbehavior",
"generatorargs",
"generatoroptions",
"genproto",
"ggnpl",
"ghaction",
"githubaccesstokens",
"gitops",
"GOBIN",
"godoc",
"golangci",
"gomarkdoc",
"googleapis",
"goreleaser",
"gotypesalias",
"grpcreflect",
"grpcroute",
"grpcroutes",
"grpcurl",
"healthchecks",
"healthz",
"helmchartargs",
"helmchartconfigs",
"helmcharts",
"Hiera",
"holos",
"holoslogger",
"horizontalpodautoscaler",
"horizontalpodautoscalers",
"Hostaliases",
"Hostnames",
"htpasswd",
"httpbin",
"httproute",
"httproutes",
"iampolicygenerator",
"Infima",
"intstr",
"isatty",
"istiod",
"jbrx",
"jeffmccune",
"jetstack",
"jiralert",
"Jsonnet",
"kfbh",
"killall",
"kubeadm",
"kubeconfig",
"kubelet",
"kubelogin",
"kubernetesobjects",
"Kustomization",
"Kustomizations",
"kustomize",
"kustomizebuild",
"kvpairsources",
"labeldrop",
"labelmap",
"ldflags",
"leaderelection",
"ledgerwriter",
"libnss",
"limitranges",
"livez",
"loadbalancer",
"loadrestrictions",
"logfmt",
"mattn",
"mccutchen",
"metav",
"mindmap",
"mktemp",
"msqbn",
"mtls",
"Multicluster",
"mutatingwebhookconfiguration",
"mutatingwebhookconfigurations",
"mvdan",
"mxcl",
"myhostname",
"myRegistrKeySecretName",
"mysecret",
"nameofclusterrole",
"nameserver",
"namespacedname",
"ndots",
"networkpolicies",
"nodename",
"nolint",
"oauthproxy",
"objectmap",
"objectmeta",
"organizationconnect",
"orgid",
"otelconnect",
"outfile",
"overriden",
"Parentspanid",
"patchstrategicmerge",
"pcjc",
"peerauthentication",
"peerauthentications",
"persistentvolumeclaim",
"persistentvolumeclaims",
"persistentvolumes",
"pflag",
"pgadmin",
"pgupgrade",
"pipefail",
"PKCE",
"platformconnect",
"pluginconfig",
"pluginrestrictions",
"podcli",
"poddisruptionbudget",
"poddisruptionbudgets",
"podinfo",
"podmonitor",
"portmapping",
"postgrescluster",
"privs",
"prometheuses",
"promhttp",
"protobuf",
"protojson",
"providerconfig",
"proxyconfig",
"proxyconfigs",
"Pulumi",
"pushgateway",
"pushsecret",
"pushsecrets",
"putenv",
"qjbp",
"quickstart",
"QVRFLS",
"readyz",
"referencegrant",
"referencegrants",
"Registr",
"replacementfield",
"replicasets",
"replicationcontrollers",
"requestauthentication",
"requestauthentications",
"resourcequotas",
"retryable",
"rogpeppe",
"rolebinding",
"rootfs",
"ropc",
"seccomp",
"secretargs",
"SECRETKEY",
"secretstore",
"secretstores",
@@ -155,29 +259,48 @@
"serviceaccount",
"servicebindings",
"serviceentries",
"serviceentry",
"servicemonitor",
"somevalue",
"SOMEVAR",
"sortoptions",
"spanid",
"spiffe",
"stackdriver",
"startupapicheck",
"statefulset",
"statefulsets",
"stefanprodan",
"storageclasses",
"streamwatcher",
"struct",
"structpb",
"subcharts",
"subjectaccessreviews",
"svclb",
"sysfs",
"systemconnect",
"tablewriter",
"templatable",
"testscript",
"thanos",
"Tiltfile",
"timestamppb",
"Timoni",
"tlsclientconfig",
"tokencache",
"Tokener",
"tolerations",
"Traceid",
"traefik",
"transactionhistory",
"tsdb",
"typemeta",
"udev",
"uibutton",
"unstage",
"untar",
"upbound",
"Upsert",
"urandom",
"usecases",
@@ -185,11 +308,19 @@
"userdata",
"userservice",
"validatingwebhookconfiguration",
"validatingwebhookconfigurations",
"vaultdynamicsecrets",
"virtualservice",
"virtualservices",
"volumeattachments",
"wasmplugin",
"wasmplugins",
"workdir",
"workloadentries",
"workloadentry",
"workloadgroup",
"workloadgroups",
"yournamespace",
"zerolog",
"zitadel",
"ztunnel"

View File

@@ -13,7 +13,7 @@ permissions:
jobs:
test:
runs-on: gha-rs
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -35,7 +35,7 @@ jobs:
uses: azure/setup-helm@v4
- name: Set up Kubectl
uses: azure/setup-kubectl@v3
uses: azure/setup-kubectl@v4
- name: Install Tools
run: |

View File

@@ -150,6 +150,10 @@ dev-deploy: install image ## deploy to dev
website: ## Build website
./hack/build-website
.PHONY: unity
unity: ## https://cuelabs.dev/unity/
./scripts/unity
.PHONY: help
help: ## Display this help menu.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-20s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)

View File

@@ -0,0 +1,4 @@
---
description: Simplified abstraction to generate core v1alpha3 components.
sidebar_position: 997
---

View File

@@ -0,0 +1,290 @@
// # Author API
//
// Package v1alpha4 contains ergonomic CUE definitions for Holos component
// authors. These definitions serve as adapters to produce [Core API] resources
// for the holos command line tool.
//
// [Core API]: https://holos.run/docs/api/core/v1alpha4/
package v1alpha4
import core "github.com/holos-run/holos/api/core/v1alpha4"
//go:generate ../../../hack/gendoc
// Platform assembles a Core API [Platform] in the Resource field for the holos
// render platform command. Use the Components field to register components
// with the platform using a struct. This struct is converted into a list for
// final output to holos.
//
// See related:
//
// - [Component] collection of components composing the platform.
// - [Platform] resource assembled for holos to process.
//
// [Platform]: https://holos.run/docs/api/core/v1alpha4/#Platform
// [Component]: https://holos.run/docs/api/core/v1alpha4/#Component
type Platform struct {
Name string
Components map[NameLabel]core.Component
Resource core.Platform
}
// Cluster represents a cluster managed by the Platform.
type Cluster struct {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
Name string `json:"name"`
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
Primary bool `json:"primary" cue:"true | *false"`
}
// Fleet represents a named collection of similarly configured Clusters. Useful
// to segregate workload clusters from their management cluster.
type Fleet struct {
Name string `json:"name"`
// Clusters represents a mapping of Clusters by their name.
Clusters map[string]Cluster `json:"clusters" cue:"{[Name=_]: name: Name}"`
}
// StandardFleets represents the standard set of Clusters in a Platform
// segmented into Fleets by their purpose. The management Fleet contains a
// single Cluster, for example a GKE autopilot cluster with no workloads
// deployed for reliability and cost efficiency. The workload Fleet contains
// all other Clusters which contain workloads and sync Secrets from the
// management cluster.
type StandardFleets struct {
// Workload represents a Fleet of zero or more workload Clusters.
Workload Fleet `json:"workload" cue:"{name: \"workload\"}"`
// Management represents a Fleet with one Cluster named management.
Management Fleet `json:"management" cue:"{name: \"management\"}"`
}
// ArgoConfig represents the ArgoCD GitOps configuration associated with a
// [BuildPlan]. Useful to define once at the root of the Platform configuration
// and reuse across all components.
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
type ArgoConfig struct {
// Enabled causes holos to render an Application resource when true.
Enabled bool `cue:"true | *false"`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// Root represents the path from the git repository root to the WriteTo output
// directory, the behavior of the holos render component --write-to flag and
// the Core API Component WriteTo field. Used as a prefix for the
// Application.spec.source.path field.
Root string `cue:"string | *\"deploy\""`
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
// AppProject represents the ArgoCD Project to associate the Application with.
AppProject string `cue:"string | *\"default\""`
}
// Organization represents organizational metadata useful across the platform.
type Organization struct {
Name string
DisplayName string
Domain string
}
// OrganizationStrict represents organizational metadata useful across the
// platform. This is an example of using CUE regular expressions to constrain
// and validate configuration.
type OrganizationStrict struct {
Organization `json:",inline"`
// Name represents the organization name as a resource name. Must be 63
// characters or less. Must start with a letter. May contain non-repeating
// hyphens, letters, and numbers. Must end with a letter or number.
Name string `cue:"=~ \"^[a-z][0-9a-z-]{1,61}[0-9a-z]$\" & !~ \"--\""`
// DisplayName represents the human readable organization name.
DisplayName string `cue:"=~ \"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$\" & !~ \" \""`
}
// Kubernetes provides a [BuildPlan] via the Output field which contains inline
// API Objects provided directly from CUE in the Resources field of
// [ComponentConfig].
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
type Kubernetes struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Helm provides a [BuildPlan] via the Output field which generates manifests
// from a helm chart with optional mix-in resources provided directly from CUE
// in the Resources field.
//
// This definition is a convenient way to produce a [BuildPlan] composed of
// three [Resources] generators with one [Kustomize] transformer.
//
// See related:
//
// - [ComponentConfig]
// - [Chart]
// - [Values]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Chart]: https://holos.run/docs/api/core/v1alpha4/#Chart
// [Values]: https://holos.run/docs/api/core/v1alpha4/#Values
type Helm struct {
ComponentConfig `json:",inline"`
// Chart represents a Helm chart.
Chart core.Chart
// Values represents data to marshal into a values.yaml for helm.
Values core.Values
// EnableHooks enables helm hooks when executing the `helm template` command.
EnableHooks bool
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// Kustomize provides a [BuildPlan] via the Output field which generates
// manifests from a kustomize kustomization with optional mix-in resources
// provided directly from CUE in the Resources field.
//
// See related:
//
// - [ComponentConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#buildplan
type Kustomize struct {
ComponentConfig `json:",inline"`
// BuildPlan represents the derived BuildPlan produced for the holos render
// component command.
BuildPlan core.BuildPlan
}
// ComponentConfig represents the configuration common to all kinds of
// component.
//
// - [Helm] charts.
// - [Kubernetes] resources generated from CUE.
// - [Kustomize] bases.
//
// See the following resources for additional details:
//
// - [Resources]
// - [ArgoConfig]
// - [KustomizeConfig]
// - [BuildPlan]
//
// [BuildPlan]: https://holos.run/docs/api/core/v1alpha4/#BuildPlan
// [Resources]: https://holos.run/docs/api/core/v1alpha4/#Resources
type ComponentConfig struct {
// Name represents the BuildPlan metadata.name field. Used to construct the
// fully rendered manifest file path.
Name string
// Component represents the path to the component producing the BuildPlan.
Component string
// Cluster represents the name of the cluster this BuildPlan is for.
Cluster string
// Resources represents kubernetes resources mixed into the rendered manifest.
Resources core.Resources
// ArgoConfig represents the ArgoCD GitOps configuration for this BuildPlan.
ArgoConfig ArgoConfig
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
// Namespace manages the metadata.namespace field on all resources except the
// ArgoCD Application.
Namespace string `json:",omitempty"`
// KustomizeConfig represents the configuration for kustomize.
KustomizeConfig KustomizeConfig
}
// KustomizeConfig represents the configuration for kustomize post processing.
// The Files field is used to mixing in static manifest files from the component
// directory. The Resources field is used for mixing in manifests from network
// locations urls.
//
// See related:
//
// - [ComponentConfig]
// - [Kustomization]
//
// [Kustomization]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/
type KustomizeConfig struct {
// Kustomization represents the kustomization used to transform resources.
// Note the resources field is internally managed from the Files and Resources fields.
Kustomization map[string]any `json:",omitempty"`
// Files represents files to copy from the component directory for kustomization.
Files map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
// Resources represents additional entries to included in the resources list.
Resources map[string]struct{ Source string } `cue:"{[NAME=_]: Source: NAME}"`
}
// Projects represents projects managed by the platform team for use by other
// teams using the platform.
type Projects map[NameLabel]Project
// Project represents logical grouping of components owned by one or more teams.
// Useful for the platform team to manage resources for project teams to use.
type Project struct {
// Name represents project name.
Name string
// Owner represents the team who own this project.
Owner Owner
// Namespaces represents the namespaces assigned to this project.
Namespaces map[NameLabel]Namespace
// Hostnames represents the host names to expose for this project.
Hostnames map[NameLabel]Hostname
// CommonLabels represents common labels to manage on all rendered manifests.
CommonLabels map[string]string
}
// Owner represents the owner of a resource. For example, the name and email
// address of an engineering team.
type Owner struct {
Name string
Email string
}
// Namespace represents a Kubernetes namespace.
type Namespace struct {
Name string
}
// Hostname represents the left most dns label of a domain name.
type Hostname struct {
// Name represents the subdomain to expose, e.g. "www"
Name string
// Namespace represents the namespace metadata.name field of backend object
// reference.
Namespace string
// Service represents the Service metadata.name field of backend object
// reference.
Service string
// Port represents the Service port of the backend object reference.
Port int
}
// NameLabel signals the common use case of converting a struct to a list where
// the name field of each value unifies with the field name of the outer struct.
//
// For example:
//
// S: [NameLabel=string]: name: NameLabel
// S: jeff: _
// S: gary: _
// S: nate: _
// L: [for x in S {x}]
// // L is [{name: "jeff"}, {name: "gary"}, {name: "nate"}]
type NameLabel string

View File

@@ -0,0 +1,4 @@
---
description: Simplified abstraction to generate core v1alpha4 build plans.
sidebar_position: 996
---

View File

@@ -0,0 +1,4 @@
---
description: Core v1alpha4 schema for advanced use cases.
sidebar_position: 998
---

View File

@@ -0,0 +1,4 @@
---
description: Core v1alpha3 schema for advanced use cases.
sidebar_position: 997
---

View File

@@ -0,0 +1,4 @@
---
description: Core schema for holos to render a component BuildPlan.
sidebar_position: 100
---

View File

@@ -1,17 +1,80 @@
// # Core API
//
// Package v1alpha4 contains the core API contract between the holos cli and CUE
// configuration code. Platform designers, operators, and software developers
// use this API to write configuration in CUE which `holos` loads. The overall
// shape of the API defines imperative actions `holos` should carry out to
// render the complete yaml that represents a Platform.
// use this API to write configuration in CUE which holos loads. The Core API
// is declarative. Each resource represents a desired state necessary for holos
// to fully render Kubernetes manifests into plain files.
//
// [Platform] defines the complete configuration of a platform. With the holos
// reference platform this takes the shape of one management cluster and at
// least two workload clusters.
// The following resources provide important context for the Core API. The
// [Author API] is intended for component authors as a convenient adapter for
// the Core API resources Holos expects.
//
// Each holos component path, e.g. `components/namespaces` produces exactly one
// [BuildPlan] which produces an [Artifact] collection. An [Artifact] is a
// fully rendered manifest produced from a [Transformer] sequence, which
// transforms a [Generator] collection.
// 1. [Technical Overview]
// 2. [Quickstart]
// 3. [Author API]
//
// # Platform
//
// [Platform] defines the complete configuration of a platform. A platform
// represents a [Component] collection.
//
// Inspect a Platform resource holos would process by executing:
//
// cue export --out yaml ./platform
//
// # Component
//
// A [Component] is the combination of CUE code along one path relative to the
// platform root directory plus data injected from the [PlatformSpec] via CUE tags.
// The platform configuration root is the directory containing cue.mod.
//
// A [Component] always produces exactly one [BuildPlan].
//
// # BuildPlan
//
// A [BuildPlan] contains an [Artifact] collection. A BuildPlan often produces
// two artifacts, one containing the fully rendered Kubernetes API resources,
// the other containing an additional resource to manage the former with GitOps.
// For example, a BuildPlan for a podinfo component produces a manifest
// containing a Deployment and a Service, along with a second manifest
// containing an ArgoCD Application.
//
// Inspect a BuildPlan resource holos render component would process by executing:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// # Artifact
//
// An [Artifact] is one fully rendered manifest file produced from the final
// [Transformer] in a sequence of transformers. An Artifact may also be
// produced directly from a [Generator], but this use case is uncommon.
//
// # Transformer
//
// A [Transformer] takes multiple inputs from prior [Generator] or [Transformer]
// outputs, then transforms the data into one output. [Kustomize] is the most
// commonly used transformer, though a simple [Join] is also supported.
//
// 1. [Kustomize] - Patch and transform the output from prior generators or
// transformers. See [Introduction to Kustomize].
// 2. [Join] - Concatenate multiple prior outputs into one output.
//
// # Generators
//
// A [Generator] generates Kubernetes resources. [Helm] and [Resources] are the
// most commonly used, often paired together to mix-in resources to an
// unmodified Helm chart. A simple [File] generator is also available for use
// with the [Kustomize] transformer.
//
// 1. [Resources] - Generates resources from CUE code.
// 2. [Helm] - Generates rendered yaml from a [Chart].
// 3. [File] - Generates data by reading a file from the component directory.
//
// [Introduction to Kustomize]: https://kubectl.docs.kubernetes.io/guides/config_management/introduction/
// [Author API]: https://holos.run/docs/api/author/
// [Quickstart]: https://holos.run/docs/quickstart/
// [Technical Overview]: https://holos.run/docs/technical-overview/
package v1alpha4
//go:generate ../../../hack/gendoc
@@ -21,6 +84,46 @@ package v1alpha4
//
// One or more [Artifact] files are produced by a BuildPlan, representing the
// fully rendered manifests for the Kubernetes API Server.
//
// # Example BuildPlan
//
// Command:
//
// cue export --out yaml ./projects/platform/components/namespaces
//
// Output:
//
// kind: BuildPlan
// apiVersion: v1alpha4
// metadata:
// name: dev-namespaces
// spec:
// component: projects/platform/components/namespaces
// artifacts:
// - artifact: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// generators:
// - kind: Resources
// output: resources.gen.yaml
// resources:
// Namespace:
// dev-jeff:
// metadata:
// name: dev-jeff
// labels:
// kubernetes.io/metadata.name: dev-jeff
// kind: Namespace
// apiVersion: v1
// transformers:
// - kind: Kustomize
// inputs:
// - resources.gen.yaml
// output: clusters/no-cluster/components/dev-namespaces/dev-namespaces.gen.yaml
// kustomize:
// kustomization:
// commonLabels:
// holos.run/component.name: dev-namespaces
// resources:
// - resources.gen.yaml
type BuildPlan struct {
// Kind represents the type of the resource.
Kind string `json:"kind" cue:"\"BuildPlan\""`
@@ -158,9 +261,12 @@ type Transformer struct {
Join Join `json:"join,omitempty"`
}
// Join represents a [Join](https://pkg.go.dev/strings#Join) [Transformer].
// Useful for the common case of combining the output of [Helm] and [Resources]
// [Generator] into one [Artifact] when [Kustomize] is otherwise unnecessary.
// Join represents a [Transformer] using [bytes.Join] to concatenate multiple
// inputs into one output with a separator. Useful for combining output from
// [Helm] and [Resources] together into one [Artifact] when [Kustomize] is
// otherwise unnecessary.
//
// [bytes.Join]: https://pkg.go.dev/bytes#Join
type Join struct {
Separator string `json:"separator" cue:"string | *\"---\\n\""`
}
@@ -238,28 +344,59 @@ type PlatformSpec struct {
Components []Component `json:"components"`
}
// Component represents the complete context necessary to produce a [BuildPlan]
// from a [Platform] component.
// Component represents the complete context necessary to produce a [BuildPlan].
// Component carries information injected from holos render platform to holos
// render component to produce each [BuildPlan].
//
// All of these fields are passed to the holos render component command using
// flags, which in turn are injected to CUE using tags. Field names should be
// used consistently through the platform rendering process for readability.
// flags, which in turn are injected to CUE using tags. For clarity, CUE field
// and tag names should match the struct json tag names below.
type Component struct {
// Name represents the name of the component, injected as a tag to set the
// BuildPlan metadata.name field. Necessary for clear user feedback during
// platform rendering.
// Name represents the name of the component. Injected as the tag variable
// "holos_name" to set the BuildPlan metadata.name field. Necessary for clear
// user feedback during platform rendering.
Name string `json:"name"`
// Component represents the path of the component relative to the platform root.
// Component represents the path of the component relative to the platform
// root. Injected as the tag variable "holos_component".
Component string `json:"component"`
// Cluster is the cluster name to provide when rendering the component.
// Injected as the tag variable "holos_cluster".
Cluster string `json:"cluster"`
// Environment for example, dev, test, stage, prod
Environment string `json:"environment,omitempty"`
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model map[string]any `json:"model"`
// Tags represents cue tags to inject when rendering the component. The json
// struct tag names of other fields in this struct are reserved tag names not
// to be used in the tags collection.
Tags []string `json:"tags,omitempty"`
// Injected as the tag "holos_model".
Model map[string]any `json:"model,omitempty"`
// Tags represents cue @tag variables injected into the holos render component
// command from the holos render platform command. Tags with a "holos_"
// prefix are reserved for use by the Holos Authors.
Tags map[string]string `json:"tags,omitempty"`
// WriteTo represents the holos render component --write-to flag. If empty,
// the default value for the --write-to flag is used.
WriteTo string `json:"writeTo,omitempty"`
}
// Tags represents standardized fields injected into the component [BuildPlan]
// from the [Platform].
//
// Note, tags should have a reasonable default value to easily use cue eval and
// cue export without needing to make a bunch of decisions about tag values.
//
// Example:
//
// import core "github.com/holos-run/holos/api/core/v1alpha4"
// _Tags: core.#Tags & {
// cluster: _ @tag(cluster, type=string)
// environment: _ @tag(environment, type=string)
// component: _ @tag(component, type=string)
// name: _ @tag(name, type=string)
// }
type Tags struct {
// Name represents the BuildPlan metadata.name field injected from the Platform.
Name string `json:"name" cue:"string | *\"no-name\""`
// Cluster represents the cluster name injected from
Cluster string `json:"cluster" cue:"string | *\"no-cluster\""`
// Environment represents the build plan environment.
Environment string `json:"environment" cue:"string | *\"no-environment\""`
// Component represents the path of the component relative to the platform root.
Component string `json:"component" cue:"string | *\"no-component\""`
}

View File

@@ -2,8 +2,10 @@ package main
import (
"os"
"path/filepath"
"testing"
cue "cuelang.org/go/cmd/cue/cmd"
"github.com/holos-run/holos/internal/cli"
"github.com/rogpeppe/go-internal/testscript"
)
@@ -11,11 +13,29 @@ import (
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cli.MakeMain(),
"cue": cue.Main,
}))
}
func TestGetSecrets(t *testing.T) {
testscript.Run(t, testscript.Params{
Dir: "testdata",
})
func TestGuides(t *testing.T) {
testscript.Run(t, params(filepath.Join("v1alpha4", "guides")))
}
func TestCLI(t *testing.T) {
testscript.Run(t, params("cli"))
}
func params(dir string) testscript.Params {
return testscript.Params{
Dir: filepath.Join("tests", dir),
RequireExplicitExec: true,
RequireUniqueNames: true,
Setup: func(env *testscript.Env) error {
// Just like cmd/cue/cmd.TestScript, set up separate cache and config dirs per test.
env.Setenv("CUE_CACHE_DIR", filepath.Join(env.WorkDir, "tmp/cachedir"))
configDir := filepath.Join(env.WorkDir, "tmp/configdir")
env.Setenv("CUE_CONFIG_DIR", configDir)
return nil
},
}
}

View File

@@ -1,34 +0,0 @@
# Want support for intermediary constraints
exec holos build ./foo/... --log-level debug
stdout '^bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- foo/constraints.cue --
package holos
metadata: name: "jeff"
-- foo/bar/bar.cue --
package holos
spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
apiObjectMap: foo: bar: "bf2bc7f9-9ba0-4f9e-9bd2-9a205627eb0b"
}
]
-- schema.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#KubernetesObjects: {
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
apiObjectMap: {...}
}
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"

View File

@@ -1,20 +0,0 @@
# Want cue errors to show files and lines
! exec holos build .
stderr 'apiObjectMap.foo.bar: cannot convert incomplete value'
stderr '/component.cue:\d+:\d+$'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: foo: bar: _baz}]
_baz: string

View File

@@ -1,61 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KubernetesObjectsList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,62 +0,0 @@
# Want kube api objects in the apiObjects output.
exec holos build .
stdout '^kind: SecretStore$'
stdout '# Source: CUE apiObjects.SecretStore.default'
stderr 'skipping helm: no chart name specified'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [{apiObjectMap: #APIObjects.apiObjectMap}]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
kind: string
metadata: name: string
}
#APIObjects: {
apiObjects: {
SecretStore: {
default: #SecretStore & { metadata: name: "default" }
}
}
}
-- schema.cue --
package holos
// #APIObjects is the output type for api objects produced by cue. A map is used to aid debugging and clarity.
import "encoding/yaml"
#APIObjects: {
// apiObjects holds each the api objects produced by cue.
apiObjects: {
[Kind=_]: {
[Name=_]: {
kind: Kind
metadata: name: Name
}
}
}
// apiObjectsContent holds the marshalled representation of apiObjects
apiObjectMap: {
for kind, v in apiObjects {
"\(kind)": {
for name, obj in v {
"\(name)": yaml.Marshal(obj)
}
}
}
}
}

View File

@@ -1,25 +0,0 @@
# Want api object kind and name in errors
! exec holos build .
stderr 'apiObjects.secretstore.default.foo: field not allowed'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "KubernetesObjects"
cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
#SecretStore: {
metadata: name: string
}
apiObjects: {
secretstore: {
default: #SecretStore & { foo: "not allowed" }
}
}

View File

@@ -1,289 +0,0 @@
# Want helm errors to show up
! exec holos build .
stderr 'Error: execution error at \(zitadel/templates/secret_zitadel-masterkey.yaml:2:4\): Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- zitadel.cue --
package holos
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: HelmChartList: [_HelmChart]
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
_HelmChart: {
apiVersion: "holos.run/v1alpha1"
kind: "HelmChart"
metadata: name: "zitadel"
namespace: "zitadel"
chart: {
name: "zitadel"
version: "7.9.0"
release: name
repository: {
name: "zitadel"
url: "https://charts.zitadel.com"
}
}
}
-- vendor/zitadel/templates/secret_zitadel-masterkey.yaml --
{{- if (or (and .Values.zitadel.masterkey .Values.zitadel.masterkeySecretName) (and (not .Values.zitadel.masterkey) (not .Values.zitadel.masterkeySecretName)) ) }}
{{- fail "Either set .Values.zitadel.masterkey xor .Values.zitadel.masterkeySecretName" }}
{{- end }}
{{- if .Values.zitadel.masterkey -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: zitadel-masterkey
{{- with .Values.zitadel.masterkeyAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "zitadel.labels" . | nindent 4 }}
stringData:
masterkey: {{ .Values.zitadel.masterkey }}
{{- end -}}
-- vendor/zitadel/Chart.yaml --
apiVersion: v2
appVersion: v2.46.0
description: A Helm chart for ZITADEL
icon: https://zitadel.com/zitadel-logo-dark.svg
kubeVersion: '>= 1.21.0-0'
maintainers:
- email: support@zitadel.com
name: zitadel
url: https://zitadel.com
name: zitadel
type: application
version: 7.9.0
-- vendor/zitadel/values.yaml --
# Default values for zitadel.
zitadel:
# The ZITADEL config under configmapConfig is written to a Kubernetes ConfigMap
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
configmapConfig:
ExternalSecure: true
Machine:
Identification:
Hostname:
Enabled: true
Webhook:
Enabled: false
# The ZITADEL config under secretConfig is written to a Kubernetes Secret
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
secretConfig:
# Annotations set on secretConfig secret
secretConfigAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# Reference the name of a secret that contains ZITADEL configuration.
configSecretName:
# The key under which the ZITADEL configuration is located in the secret.
configSecretKey: config-yaml
# ZITADEL uses the masterkey for symmetric encryption.
# You can generate it for example with tr -dc A-Za-z0-9 </dev/urandom | head -c 32
masterkey: ""
# Reference the name of the secret that contains the masterkey. The key should be named "masterkey".
# Note: Either zitadel.masterkey or zitadel.masterkeySecretName must be set
masterkeySecretName: ""
# Annotations set on masterkey secret
masterkeyAnnotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The CA Certificate needed for establishing secure database connections
dbSslCaCrt: ""
# The Secret containing the CA certificate at key ca.crt needed for establishing secure database connections
dbSslCaCrtSecret: ""
# The db admins secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslAdminCrtSecret: ""
# The db users secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslUserCrtSecret: ""
# Generate a self-signed certificate using an init container
# This will also mount the generated files to /etc/tls/ so that you can reference them in the pod.
# E.G. KeyPath: /etc/tls/tls.key CertPath: /etc/tls/tls.crt
# By default, the SAN DNS names include, localhost, the POD IP address and the POD name. You may include one more by using additionalDnsName like "my.zitadel.fqdn".
selfSignedCert:
enabled: false
additionalDnsName:
replicaCount: 3
image:
repository: ghcr.io/zitadel/zitadel
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
chownImage:
repository: alpine
pullPolicy: IfNotPresent
tag: "3.19"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
# Annotations to add to the deployment
annotations: {}
# Annotations to add to the configMap
configMap:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "0"
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podAdditionalLabels: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
securityContext: {}
# Additional environment variables
env:
[]
# - name: ZITADEL_DATABASE_POSTGRES_HOST
# valueFrom:
# secretKeyRef:
# name: postgres-pguser-postgres
# key: host
service:
type: ClusterIP
# If service type is "ClusterIP", this can optionally be set to a fixed IP address.
clusterIP: ""
port: 8080
protocol: http2
annotations: {}
scheme: HTTP
ingress:
enabled: false
className: ""
annotations: {}
hosts:
- host: localhost
paths:
- path: /
pathType: Prefix
tls: []
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
topologySpreadConstraints: []
initJob:
# Once ZITADEL is installed, the initJob can be disabled.
enabled: true
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "1"
resources: {}
backoffLimit: 5
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
# Available init commands :
# "": initialize ZITADEL instance (without skip anything)
# database: initialize only the database
# grant: set ALL grant to user
# user: initialize only the database user
# zitadel: initialize ZITADEL internals (skip "create user" and "create database")
command: ""
setupJob:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "2"
resources: {}
activeDeadlineSeconds: 300
extraContainers: []
podAnnotations: {}
additionalArgs:
- "--init-projections=true"
machinekeyWriter:
image:
repository: bitnami/kubectl
tag: ""
resources: {}
readinessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
livenessProbe:
enabled: true
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 3
startupProbe:
enabled: true
periodSeconds: 1
failureThreshold: 30
metrics:
enabled: false
serviceMonitor:
# If true, the chart creates a ServiceMonitor that is compatible with Prometheus Operator
# https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#monitoring.coreos.com/v1.ServiceMonitor.
# The Prometheus community Helm chart installs this operator
# https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#kube-prometheus-stack
enabled: false
honorLabels: false
honorTimestamps: true
pdb:
enabled: false
# these values are used for the PDB and are mutally exclusive
minAvailable: 1
# maxUnavailable: 1
annotations: {}

View File

@@ -1,39 +0,0 @@
# Kustomize is a supported holos component kind
exec holos render component --cluster-name=mycluster . --log-level=debug
# Want generated output
cmp want.yaml deploy/clusters/mycluster/components/kstest/kstest.gen.yaml
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: KustomizeBuildList: [{metadata: name: "kstest"}]
-- kustomization.yaml --
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mynamespace
resources:
- serviceaccount.yaml
-- serviceaccount.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
-- want.yaml --
apiVersion: v1
kind: ServiceAccount
metadata:
name: test
namespace: mynamespace

View File

@@ -1,17 +0,0 @@
# https://github.com/holos-run/holos/issues/72
# Want holos to fail on unknown fields to catch typos and aid refactors
! exec holos build .
stderr 'unknown field \\"TypoKubernetesObjectsList\\"'
-- platform.config.json --
{}
-- cue.mod --
package holos
-- component.cue --
package holos
_cluster: string @tag(cluster, string)
_platform_config: string @tag(platform_config, string)
apiVersion: "holos.run/v1alpha1"
kind: "BuildPlan"
spec: components: TypoKubernetesObjectsList: []

File diff suppressed because it is too large Load Diff

View File

@@ -80,33 +80,62 @@ The bank front end web service is managed by the
refers to the organization display name in `schema.gen.cue`.
<Tabs groupId="F5B546EB-566F-4B83-84C3-C55B40F55555">
<TabItem value="schema.gen.cue" label="schema.gen.cue">
<TabItem value="schema.cue" label="schema.cue">
```cue showLineNumbers
package holos
import api "github.com/holos-run/holos/api/author/v1alpha3"
import api "github.com/holos-run/holos/api/author/v1alpha4"
// Define the default organization name
// highlight-next-line
#Organization: DisplayName: string | *"Bank of Holos"
#Organization: Name: string | *"bank-of-holos"
#Organization: api.#OrganizationStrict
#Platform: api.#Platform
#Fleets: api.#StandardFleets
_ComponentConfig: {
Resources: #Resources
ArgoConfig: #ArgoConfig
// Define the default organization name.
_Organization: api.#OrganizationStrict & {
DisplayName: string | *"Bank of Holos"
Name: string | *"bank-of-holos"
Domain: string | *"holos.localhost"
}
#Helm: api.#Helm & _ComponentConfig
#Kustomize: api.#Kustomize & _ComponentConfig
#Kubernetes: api.#Kubernetes & _ComponentConfig
// Projects represents a way to organize components into projects with owners.
// https://holos.run/docs/api/author/v1alpha4/#Projects
_Projects: api.#Projects
#ArgoConfig: api.#ArgoConfig & {
ClusterName: _ClusterName
// ArgoConfig represents the configuration of ArgoCD Application resources for
// each component.
// https://holos.run/docs/api/author/v1alpha4/#ArgoConfig
_ArgoConfig: api.#ArgoConfig
#ComponentConfig: api.#ComponentConfig & {
Name: _Tags.name
Component: _Tags.component
Cluster: _Tags.cluster
ArgoConfig: _ArgoConfig & {
if _Tags.project != "no-project" {
AppProject: _Tags.project
}
}
Resources: #Resources
// Mix in project labels if the project is defined by the platform.
if _Tags.project != "no-project" {
CommonLabels: _Projects[_Tags.project].CommonLabels
}
}
// https://holos.run/docs/api/author/v1alpha4/#Kubernetes
#Kubernetes: close({
#ComponentConfig
api.#Kubernetes
})
// https://holos.run/docs/api/author/v1alpha4/#Kustomize
#Kustomize: close({
#ComponentConfig
api.#Kustomize
})
// https://holos.run/docs/api/author/v1alpha4/#Helm
#Helm: close({
#ComponentConfig
api.#Helm
})
```
</TabItem>
<TabItem value="projects/bank-of-holos/frontend/components/bank-frontend/bank-frontend.cue" label="projects/bank-of-holos/frontend/components/bank-frontend/bank-frontend.cue">
@@ -117,215 +146,214 @@ package holos
(#Kubernetes & Objects).BuildPlan
let Objects = {
Name: "bank-frontend"
Namespace: #BankOfHolos.Frontend.Namespace
Name: "bank-frontend"
Namespace: _BankOfHolos.Frontend.Namespace
// Ensure resources go in the correct namespace
Resources: [_]: [_]: metadata: namespace: Namespace
// Ensure resources go in the correct namespace
Resources: [_]: [_]: metadata: namespace: Namespace
// https://github.com/GoogleCloudPlatform/bank-of-anthos/blob/release/v0.6.5/kubernetes-manifests/frontend.yaml
Resources: {
Service: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
_ports: http: {
name: "http"
port: 80
targetPort: 8080
protocol: "TCP"
}
ports: [for x in _ports {x}]
}
}
// https://github.com/GoogleCloudPlatform/bank-of-anthos/blob/release/v0.6.5/kubernetes-manifests/frontend.yaml
Resources: {
Service: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
_ports: http: {
name: "http"
port: 80
targetPort: 8080
protocol: "TCP"
}
ports: [for x in _ports {x}]
}
}
Deployment: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: matchLabels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
template: {
metadata: labels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
securityContext: {
seccompProfile: type: "RuntimeDefault"
fsGroup: 1000
runAsGroup: 1000
runAsNonRoot: true
runAsUser: 1000
}
serviceAccountName: "bank-of-holos"
terminationGracePeriodSeconds: 5
containers: [{
env: [{
name: "BANK_NAME"
// highlight-next-line
value: #Organization.DisplayName
}, {
name: "ENV_PLATFORM"
value: "local"
}, {
name: "VERSION"
value: "v0.6.5"
}, {
name: "PORT"
value: "8080"
}, {
name: "ENABLE_TRACING"
value: "false"
}, {
name: "SCHEME"
value: "https"
}, {
name: "LOG_LEVEL"
value: "info"
}, {
name: "DEFAULT_USERNAME"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_USERNAME"
name: "demo-data-config"
}
}, {
name: "DEFAULT_PASSWORD"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_PASSWORD"
name: "demo-data-config"
}
}, {
name: "REGISTERED_OAUTH_CLIENT_ID"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_CLIENT_ID"
name: "oauth-config"
optional: true
}
}, {
name: "ALLOWED_OAUTH_REDIRECT_URI"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_REDIRECT_URI"
name: "oauth-config"
optional: true
}
}]
envFrom: [{
configMapRef: name: "environment-config"
}, {
configMapRef: name: "service-api-config"
}]
image: "us-central1-docker.pkg.dev/bank-of-anthos-ci/bank-of-anthos/frontend:v0.6.5@sha256:d72050f70d12383e4434ad04d189b681dc625f696087ddf0b5df641645c9dafa"
livenessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 60
periodSeconds: 15
timeoutSeconds: 30
}
name: "front"
readinessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 10
}
resources: {
limits: {
cpu: "250m"
memory: "128Mi"
}
requests: {
cpu: "100m"
memory: "64Mi"
}
}
securityContext: {
allowPrivilegeEscalation: false
capabilities: drop: ["all"]
privileged: false
readOnlyRootFilesystem: true
}
volumeMounts: [{
mountPath: "/tmp"
name: "tmp"
}, {
mountPath: "/tmp/.ssh"
name: "publickey"
readOnly: true
}]
}]
volumes: [
{
emptyDir: {}
name: "tmp"
},
{
name: "publickey"
secret: {
items: [{key: "jwtRS256.key.pub", path: "publickey"}]
secretName: "jwt-key"
}
},
]
}
}
}
}
Deployment: frontend: {
metadata: name: "frontend"
metadata: labels: {
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
selector: matchLabels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
template: {
metadata: labels: {
app: "frontend"
application: "bank-of-holos"
environment: "development"
team: "frontend"
tier: "web"
}
spec: {
securityContext: {
seccompProfile: type: "RuntimeDefault"
fsGroup: 1000
runAsGroup: 1000
runAsNonRoot: true
runAsUser: 1000
}
serviceAccountName: "bank-of-holos"
terminationGracePeriodSeconds: 5
containers: [{
env: [{
name: "BANK_NAME"
value: _Organization.DisplayName
}, {
name: "ENV_PLATFORM"
value: "local"
}, {
name: "VERSION"
value: "v0.6.5"
}, {
name: "PORT"
value: "8080"
}, {
name: "ENABLE_TRACING"
value: "false"
}, {
name: "SCHEME"
value: "https"
}, {
name: "LOG_LEVEL"
value: "info"
}, {
name: "DEFAULT_USERNAME"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_USERNAME"
name: "demo-data-config"
}
}, {
name: "DEFAULT_PASSWORD"
valueFrom: configMapKeyRef: {
key: "DEMO_LOGIN_PASSWORD"
name: "demo-data-config"
}
}, {
name: "REGISTERED_OAUTH_CLIENT_ID"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_CLIENT_ID"
name: "oauth-config"
optional: true
}
}, {
name: "ALLOWED_OAUTH_REDIRECT_URI"
valueFrom: configMapKeyRef: {
key: "DEMO_OAUTH_REDIRECT_URI"
name: "oauth-config"
optional: true
}
}]
envFrom: [{
configMapRef: name: "environment-config"
}, {
configMapRef: name: "service-api-config"
}]
image: "us-central1-docker.pkg.dev/bank-of-anthos-ci/bank-of-anthos/frontend:v0.6.5@sha256:d72050f70d12383e4434ad04d189b681dc625f696087ddf0b5df641645c9dafa"
livenessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 60
periodSeconds: 15
timeoutSeconds: 30
}
name: "front"
readinessProbe: {
httpGet: {
path: "/ready"
port: 8080
}
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 10
}
resources: {
limits: {
cpu: "250m"
memory: "128Mi"
}
requests: {
cpu: "100m"
memory: "64Mi"
}
}
securityContext: {
allowPrivilegeEscalation: false
capabilities: drop: ["all"]
privileged: false
readOnlyRootFilesystem: true
}
volumeMounts: [{
mountPath: "/tmp"
name: "tmp"
}, {
mountPath: "/tmp/.ssh"
name: "publickey"
readOnly: true
}]
}]
volumes: [
{
emptyDir: {}
name: "tmp"
},
{
name: "publickey"
secret: {
items: [{key: "jwtRS256.key.pub", path: "publickey"}]
secretName: "jwt-key"
}
},
]
}
}
}
}
// Allow HTTPRoutes in the ingress gateway namespace to reference Services
// in this namespace.
ReferenceGrant: grant: #ReferenceGrant & {
metadata: namespace: Namespace
}
// Allow HTTPRoutes in the ingress gateway namespace to reference Services
// in this namespace.
ReferenceGrant: grant: _ReferenceGrant & {
metadata: namespace: Namespace
}
// Include shared resources
#BankOfHolos.Resources
}
// Include shared resources
_BankOfHolos.Resources
}
}
```
</TabItem>
</Tabs>
Line 6 of the `schema.gen.cue` file defines the _default_ value for
`#Organization.DisplayName` by using `string | *"..."`. In CUE, the `*`
Line 7 of the `schema.cue` file defines the _default_ value for
`_Organization.DisplayName` by using `string | *"..."`. In CUE, the `*`
asterisk character denotes a [default value].
Line 78 of the `bank-frontend.cue` file refers to `#Organization.DisplayName` to
Line 78 of the `bank-frontend.cue` file refers to `_Organization.DisplayName` to
configure the front end web container.
Let's change the name of the bank by defining a new value for
`#Organization.DisplayName` at the root of the configuration. Create
`_Organization.DisplayName` at the root of the configuration. Create
`projects/organization.cue` with the following content.
<Tabs groupId="B386181F-EBE7-469D-8CB5-37631067669B">
@@ -333,7 +361,7 @@ Let's change the name of the bank by defining a new value for
```cue showLineNumbers
package holos
#Organization: DisplayName: "The Holistic-Bank"
_Organization: DisplayName: "The Holistic-Bank"
```
</TabItem>
</Tabs>
@@ -348,18 +376,11 @@ holos render platform ./platform
</TabItem>
<TabItem value="output" label="Output">
```txt
#Organization.DisplayName: 2 errors in empty disjunction:
#Organization.DisplayName: conflicting values "Bank of Holos" and "The Holistic-Bank":
/bank-of-holos/projects/organization.cue:3:29
/bank-of-holos/schema.gen.cue:6:39
// highlight-next-line
#Organization.DisplayName: invalid value "The Holistic-Bank" (out of bound =~"^[0-9A-Za-z][0-9A-Za-z ]{2,61}[0-9A-Za-z]$"):
/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/author/v1alpha3/definitions_go_gen.cue:203:25
/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/author/v1alpha3/definitions_go_gen.cue:188:15
/bank-of-holos/cue.mod/gen/github.com/holos-run/holos/api/author/v1alpha3/definitions_go_gen.cue:203:15
/bank-of-holos/projects/organization.cue:3:29
/bank-of-holos/schema.gen.cue:6:29
could not run: could not render component: exit status 1 at internal/render/platform.go:50
could not run: could not marshal json projects/platform/components/istio/cni: cue: marshal error: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/builder.go:63
_Organization.DisplayName: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors)
could not run: could not marshal json projects/platform/components/argocd/crds: cue: marshal error: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors) at internal/builder/builder.go:63
_Organization.DisplayName: _Organization.DisplayName: 2 errors in empty disjunction: (and 2 more errors)
could not run: could not render component: exit status 1 at builder/v1alpha4/builder.go:95
```
</TabItem>
</Tabs>
@@ -386,7 +407,7 @@ Let's try again, this time replacing the hyphen with a space.
```cue showLineNumbers
package holos
#Organization: DisplayName: "The Holistic Bank"
_Organization: DisplayName: "The Holistic Bank"
```
</TabItem>
</Tabs>

View File

@@ -94,8 +94,8 @@ an ArgoCD Application or Flux Kustomization.
consistently add common labels.
:::tip
[ComponentFields] in the Author API describes the fields common to all kinds of
component.
[ComponentConfig] in the Author API describes the fields common to all kinds of
components.
:::
We'll start with a [Helm] component to deploy the service, then compare it to a
@@ -130,23 +130,23 @@ package holos
// Platform wide definitions
// highlight-next-line
#Migration: Namespace: "migration"
_Migration: Namespace: "migration"
// Register namespaces
// highlight-next-line
#Namespaces: (#Migration.Namespace): _
_Namespaces: (_Migration.Namespace): _
// Register projects
// highlight-next-line
#AppProjects: migration: _
_AppProjects: migration: _
```
</TabItem>
</Tabs>
Each of the highlighted lines has a specific purpose.
- Line 4 defines the `#Migration` CUE struct. The team that currently owns the
migration project defines this struct.
- Line 4 defines the `_Migration` hidden field. The team owning the migration
project manages this struct.
- Line 7 registers the namespace with the `namespaces` component owned by the
platform team. The `_` value indicates the value is defined elsewhere in CUE.
In this case, the platform team defines what a Namespace is.
@@ -208,9 +208,9 @@ Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
// highlight-next-line
modified: deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
modified: deploy/clusters/local/components/app-projects/app-projects.gen.yaml
// highlight-next-line
modified: deploy/clusters/workload/components/namespaces/namespaces.gen.yaml
modified: deploy/clusters/local/components/namespaces/namespaces.gen.yaml
Untracked files:
(use "git add <file>..." to include in what will be committed)
@@ -230,15 +230,14 @@ git diff deploy
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml b/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
diff --git a/deploy/clusters/local/components/app-projects/app-projects.gen.yaml b/deploy/clusters/local/components/app-projects/app-projects.gen.yaml
index bdc8371..42cb01a 100644
--- a/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
+++ b/deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
--- a/deploy/clusters/local/components/app-projects/app-projects.gen.yaml
+++ b/deploy/clusters/local/components/app-projects/app-projects.gen.yaml
@@ -50,6 +50,23 @@ spec:
sourceRepos:
- '*'
---
+# Source: CUE apiObjects.AppProject.migration
+apiVersion: argoproj.io/v1alpha1
+kind: AppProject
+metadata:
@@ -255,19 +254,17 @@ index bdc8371..42cb01a 100644
+ sourceRepos:
+ - '*'
+---
# Source: CUE apiObjects.AppProject.platform
apiVersion: argoproj.io/v1alpha1
kind: AppProject
diff --git a/deploy/clusters/workload/components/namespaces/namespaces.gen.yaml b/deploy/clusters/workload/components/namespaces/namespaces.gen.yaml
diff --git a/deploy/clusters/local/components/namespaces/namespaces.gen.yaml b/deploy/clusters/local/components/namespaces/namespaces.gen.yaml
index de96ab9..7ddd870 100644
--- a/deploy/clusters/workload/components/namespaces/namespaces.gen.yaml
+++ b/deploy/clusters/workload/components/namespaces/namespaces.gen.yaml
--- a/deploy/clusters/local/components/namespaces/namespaces.gen.yaml
+++ b/deploy/clusters/local/components/namespaces/namespaces.gen.yaml
@@ -62,3 +62,11 @@ metadata:
kubernetes.io/metadata.name: istio-system
kind: Namespace
apiVersion: v1
+---
+# Source: CUE apiObjects.Namespace.migration
+metadata:
+ name: migration
+ labels:
@@ -351,33 +348,33 @@ import ks "sigs.k8s.io/kustomize/api/types"
// Produce a helm chart build plan.
// highlight-next-line
(#Helm & Chart).BuildPlan
_Helm.BuildPlan
let Chart = {
// highlight-next-line
Name: "podinfo"
Version: "6.6.2"
// highlight-next-line
Namespace: #Migration.Namespace
_Helm: #Helm & {
// highlight-next-line
Name: "podinfo"
// highlight-next-line
Namespace: _Migration.Namespace
// Necessary to ensure the resources go to the correct namespace.
// highlight-next-line
EnableKustomizePostProcessor: true
// highlight-next-line
KustomizeFiles: "kustomization.yaml": ks.#Kustomization & {
namespace: Namespace
}
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
Repo: name: "podinfo"
Repo: url: "https://stefanprodan.github.io/podinfo"
KustomizeConfig: Kustomization: ks.#Kustomization & {
// highlight-next-line
namespace: Namespace
}
// Allow the platform team to route traffic into our namespace.
// highlight-next-line
Resources: ReferenceGrant: grant: #ReferenceGrant & {
metadata: namespace: Namespace
}
// Allow the platform team to route traffic into our namespace.
Resources: ReferenceGrant: grant: _ReferenceGrant & {
// highlight-next-line
metadata: namespace: Namespace
}
}
```
</TabItem>
</Tabs>
@@ -396,19 +393,19 @@ Name as the sub-directory name when it writes the rendered manifest into
`deploy/`. Normally this name also matches the directory and file name of the
component, `podinfo/podinfo.cue`, but `holos` doesn't enforce this convention.
**Line 11**: We use the same namespace we registered with the `namespaces`
**Line 10**: We use the same namespace we registered with the `namespaces`
component as the value we pass to Helm. This is a good example of Holos
offering safety and consistency with CUE, if we change the value of
`#Migration.Namespace`, multiple components stay consistent.
`_Migration.Namespace`, multiple components stay consistent.
**Lines 14-15**: Unfortunately, the Helm chart doesn't set the
**Lines 21**: Unfortunately, the Helm chart doesn't set the
`metadata.namespace` field for the resources it generates, which creates a
security problem. The resources will be created in the wrong namespace. We
don't want to modify the upstream chart because it creates a maintenance burden.
We solve the problem by having Holos post-process the Helm output with
Kustomize. This ensures all resources go into the correct namespace.
**Lines 23**: The migration team grants the platform team permission to route
**Lines 26**: The migration team grants the platform team permission to route
traffic into the `migration` Namespace using a [ReferenceGrant].
:::note
@@ -459,11 +456,12 @@ following content.
package holos
// Manage on workload clusters only
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name)/podinfo": {
path: "projects/migration/components/podinfo"
cluster: Cluster.name
}
for Cluster in _Fleets.workload.clusters {
_Platform: Components: "\(Cluster.name)/podinfo": {
name: "podinfo"
component: "projects/migration/components/podinfo"
cluster: Cluster.name
}
}
```
</TabItem>
@@ -488,7 +486,7 @@ package holos
// Assign ArgoCD Applications to the migration AppProject
// highlight-next-line
#ArgoConfig: AppProject: #AppProjects.migration.metadata.name
_ArgoConfig: AppProject: _AppProjects.migration.metadata.name
```
</TabItem>
</Tabs>
@@ -497,7 +495,7 @@ This file provides consistency and safety in a number of ways:
1. All components under `projects/migration/` will automatically have their
ArgoCD Application assigned to the migration `AppProject`.
2. `holos render platform` errors out if `#AppProjects.migration` is not
2. `holos render platform` errors out if `_AppProjects.migration` is not
defined, we defined it in `projects/migration.cue`
3. The platform team is responsible for managing the `AppProject` resource
itself, the team doing the migration refers to the `metadata.name` field defined
@@ -557,14 +555,14 @@ git status
</TabItem>
<TabItem value="output" label="Output">
```txt
On branch main
On branch jeff/291-consistent-fields
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
new file: deploy/clusters/workload/components/podinfo/podinfo.gen.yaml
new file: deploy/clusters/workload/gitops/podinfo.application.gen.yaml
new file: platform/migration-podinfo.cue
new file: projects/migration/app-project.cue
new file: projects/migration/components/podinfo/podinfo.cue
new file: deploy/clusters/local/components/podinfo/podinfo.gen.yaml
new file: deploy/clusters/local/gitops/podinfo.gen.yaml
new file: platform/migration-podinfo.cue
new file: projects/migration/app-project.cue
new file: projects/migration/components/podinfo/podinfo.cue
```
</TabItem>
</Tabs>
@@ -574,25 +572,26 @@ in a new manifest for the Helm output along with an ArgoCD Application for
GitOps. Here's what they look like:
<Tabs groupId="0F2B3066-B57F-466E-A27F-A603C1803E11">
<TabItem value="deploy/clusters/workload/gitops/podinfo.application.gen.yaml" label="podinfo.application.gen.yaml">
<TabItem value="deploy/clusters/local/gitops/podinfo.gen.yaml" label="podinfo.gen.yaml">
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
labels:
holos.run/component.name: podinfo
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
# highlight-next-line
project: migration
source:
path: ./deploy/clusters/workload/components/podinfo
path: deploy/clusters/local/components/podinfo
repoURL: https://github.com/holos-run/bank-of-holos
targetRevision: main
```
</TabItem>
<TabItem value="deploy/clusters/workload/components/podinfo/podinfo.gen.yaml" label="podinfo.gen.yaml">
<TabItem value="deploy/clusters/local/components/podinfo/podinfo.gen.yaml" label="podinfo.gen.yaml">
```yaml showLineNumbers
apiVersion: v1
kind: Service
@@ -601,9 +600,10 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
argocd.argoproj.io/instance: podinfo
helm.sh/chart: podinfo-6.6.2
holos.run/component.name: podinfo
name: podinfo
# highlight-next-line
namespace: migration
spec:
ports:
@@ -617,6 +617,8 @@ spec:
targetPort: grpc
selector:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
holos.run/component.name: podinfo
type: ClusterIP
---
apiVersion: apps/v1
@@ -626,15 +628,18 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
argocd.argoproj.io/instance: podinfo
helm.sh/chart: podinfo-6.6.2
holos.run/component.name: podinfo
name: podinfo
# highlight-next-line
namespace: migration
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
holos.run/component.name: podinfo
strategy:
rollingUpdate:
maxUnavailable: 1
@@ -646,6 +651,8 @@ spec:
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
holos.run/component.name: podinfo
spec:
containers:
- command:
@@ -710,6 +717,23 @@ spec:
volumes:
- emptyDir: {}
name: data
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
labels:
argocd.argoproj.io/instance: podinfo
holos.run/component.name: podinfo
name: istio-ingress
namespace: migration
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: istio-ingress
to:
- group: ""
kind: Service
```
</TabItem>
</Tabs>
@@ -726,10 +750,10 @@ git commit -m 'register the migration project podinfo component with the platfor
</TabItem>
<TabItem value="output" label="Output">
```txt
[main] register the migration project podinfo component with the platform
5 files changed, 167 insertions(+)
create mode 100644 deploy/clusters/workload/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/workload/gitops/podinfo.application.gen.yaml
[main 31197e2] register the migration project podinfo component with the platform
5 files changed, 205 insertions(+)
create mode 100644 deploy/clusters/local/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/local/gitops/podinfo.gen.yaml
create mode 100644 platform/migration-podinfo.cue
create mode 100644 projects/migration/app-project.cue
create mode 100644 projects/migration/components/podinfo/podinfo.cue
@@ -773,31 +797,29 @@ platform as a whole. Bank of Holos uses [HTTPRoute] routes from the new Gateway
API. The company the bank acquired uses older Ingress resources from earlier
Kubernetes versions.
The platform team at the bank defines a `#HTTPRoutes` struct for other teams at
the bank to register with. The `#HTTPRoutes` struct is similar to the
`#Namespaces` and `#AppProjects` structs we've already seen.
The platform team at the bank defines a `_HTTPRoutes` struct for other teams at
the bank to register with. The `_HTTPRoutes` struct is similar to the
`_Namespaces` and `_AppProjects` structs we've already seen.
As a member of the migration team, we'll add the file
`projects/migration-routes.cue` to expose the service we're migrating.
Go ahead and create this file with the following content.
Go ahead and create this file (if it hasn't been created previously) with the following content.
<Tabs groupId="6F9044EC-1737-4926-BD07-455536BA6573">
<TabItem value="projects/migration-routes.cue" label="projects/migration-routes.cue">
```cue showLineNumbers
package holos
let Podinfo = {
podinfo: {
port: 9898
namespace: #Migration.Namespace
}
}
// Route migration-podinfo.example.com to port 9898 of Service podinfo in the
// migration namespace.
// highlight-next-line
#HTTPRoutes: "migration-podinfo": _backendRefs: Podinfo
_HTTPRoutes: "migration-podinfo": _backendRefs: {
podinfo: {
port: 9898
namespace: _Migration.Namespace
}
}
```
</TabItem>
<TabItem value="projects/httproutes.cue" label="projects/httproutes.cue">
@@ -806,44 +828,45 @@ package holos
import v1 "gateway.networking.k8s.io/httproute/v1"
// #HTTPRoutes defines managed HTTPRoute resources for the platform. These
// resources are managed in the istio-ingress namespace. Other components
// define the routes they need close to the root of configuration.
// Struct containing HTTPRoute configurations. These resources are managed in
// the istio-ingress namespace. Other components define the routes they need
// close to the root of configuration.
_HTTPRoutes: #HTTPRoutes
// #HTTPRoutes defines the schema of managed HTTPRoute resources for the
// platform.
#HTTPRoutes: {
// For the guides, we simplify this down to a flat namespace.
// highlight-next-line
[Name=string]: v1.#HTTPRoute & {
let HOST = Name + "." + #Platform.Domain
// For the guides, we simplify this down to a flat namespace.
[Name=string]: v1.#HTTPRoute & {
let HOST = Name + "." + _Organization.Domain
// highlight-next-line
_backendRefs: [NAME=string]: {
name: NAME
namespace: string
port: number | *80
}
_backendRefs: [NAME=string]: {
name: NAME
namespace: string
port: number | *80
}
metadata: name: Name
metadata: namespace: #Istio.Gateway.Namespace
metadata: labels: app: Name
spec: hostnames: [HOST]
spec: parentRefs: [{
name: "default"
namespace: metadata.namespace
}]
spec: rules: [
{
matches: [{path: {type: "PathPrefix", value: "/"}}]
// highlight-next-line
backendRefs: [for x in _backendRefs {x}]
},
]
}
metadata: name: Name
metadata: namespace: _Istio.Gateway.Namespace
metadata: labels: app: Name
spec: hostnames: [HOST]
spec: parentRefs: [{
name: "default"
namespace: metadata.namespace
}]
spec: rules: [
{
matches: [{path: {type: "PathPrefix", value: "/"}}]
backendRefs: [for x in _backendRefs {x}]
},
]
}
}
```
</TabItem>
</Tabs>
In this file we're adding a field to the `#HTTPRoutes` struct the platform team
In this file we're adding a field to the `_HTTPRoutes` struct the platform team
defined for us.
You might be wondering how we knew all of these fields to put into this file.
@@ -857,8 +880,9 @@ The most important things the migration team takes away from this file are:
1. The platform team requires a `gateway.networking.k8s.io/httproute/v1`
`HTTPRoute`.
2. Line 13 uses a [hidden field] so we can provide backend references as a struct instead of a list.
3. Line 30 uses a [comprehension] to convert the struct to a list.
2. Line 17 uses a [hidden field] so we can provide backend references as a
struct instead of a list.
3. Line 34 uses a [comprehension] to convert the struct to a list.
We can look up the spec for the fields we need to provide in the Gateway API
reference documentation for [HTTPRoute].
@@ -924,39 +948,39 @@ git diff
</TabItem>
<TabItem value="output" label="Output">
```diff
diff --git a/deploy/clusters/workload/components/httproutes/httproutes.gen.yaml b/deploy/clusters/workload/components/httproutes/httproutes.gen.yaml
index 4b476da..a150015 100644
--- a/deploy/clusters/workload/components/httproutes/httproutes.gen.yaml
+++ b/deploy/clusters/workload/components/httproutes/httproutes.gen.yaml
@@ -46,3 +46,27 @@ spec:
- name: frontend
namespace: bank-frontend
port: 80
diff --git a/deploy/clusters/local/components/httproutes/httproutes.gen.yaml b/deploy/clusters/local/components/httproutes/httproutes.gen.yaml
index 06f7c91..349e070 100644
--- a/deploy/clusters/local/components/httproutes/httproutes.gen.yaml
+++ b/deploy/clusters/local/components/httproutes/httproutes.gen.yaml
@@ -47,3 +47,28 @@ spec:
- path:
type: PathPrefix
value: /
+---
+# Source: CUE apiObjects.HTTPRoute.migration-podinfo
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ name: migration-podinfo
+ namespace: istio-ingress
+ labels:
+ app: migration-podinfo
+ argocd.argoproj.io/instance: httproutes
+ holos.run/component.name: httproutes
+ name: migration-podinfo
+ namespace: istio-ingress
+spec:
+ hostnames:
+ - migration-podinfo.holos.localhost
+ - migration-podinfo.holos.localhost
+ parentRefs:
+ - name: default
+ namespace: istio-ingress
+ - name: default
+ namespace: istio-ingress
+ rules:
+ - matches:
+ - path:
+ type: PathPrefix
+ value: /
+ backendRefs:
+ - name: podinfo
+ port: 9898
+ namespace: migration
+ - backendRefs:
+ - name: podinfo
+ namespace: migration
+ port: 9898
+ matches:
+ - path:
+ type: PathPrefix
+ value: /
```
</TabItem>
</Tabs>
@@ -1039,7 +1063,7 @@ in the `bank-of-holos` repository after resetting your cluster following the
</TabItem>
<TabItem value="output" label="Output">
```txt
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/namespaces/namespaces.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/namespaces/namespaces.gen.yaml
namespace/argocd serverside-applied
namespace/bank-backend serverside-applied
namespace/bank-frontend serverside-applied
@@ -1049,17 +1073,17 @@ namespace/external-secrets serverside-applied
namespace/istio-ingress serverside-applied
namespace/istio-system serverside-applied
namespace/migration serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/argocd-crds/argocd-crds.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/argocd-crds/argocd-crds.gen.yaml
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/gateway-api/gateway-api.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/gateway-api/gateway-api.gen.yaml
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/external-secrets-crds/external-secrets-crds.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/external-secrets-crds/external-secrets-crds.gen.yaml
customresourcedefinition.apiextensions.k8s.io/acraccesstokens.generators.external-secrets.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/clusterexternalsecrets.external-secrets.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/clustersecretstores.external-secrets.io serverside-applied
@@ -1121,7 +1145,7 @@ customresourcedefinition.apiextensions.k8s.io/wasmplugins.extensions.istio.io co
customresourcedefinition.apiextensions.k8s.io/webhooks.generators.external-secrets.io condition met
customresourcedefinition.apiextensions.k8s.io/workloadentries.networking.istio.io condition met
customresourcedefinition.apiextensions.k8s.io/workloadgroups.networking.istio.io condition met
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/external-secrets/external-secrets.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/external-secrets/external-secrets.gen.yaml
serviceaccount/external-secrets-cert-controller serverside-applied
serviceaccount/external-secrets serverside-applied
serviceaccount/external-secrets-webhook serverside-applied
@@ -1141,7 +1165,7 @@ deployment.apps/external-secrets serverside-applied
deployment.apps/external-secrets-webhook serverside-applied
validatingwebhookconfiguration.admissionregistration.k8s.io/secretstore-validate serverside-applied
validatingwebhookconfiguration.admissionregistration.k8s.io/externalsecret-validate serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/cert-manager/cert-manager.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/cert-manager/cert-manager.gen.yaml
serviceaccount/cert-manager-cainjector serverside-applied
serviceaccount/cert-manager serverside-applied
serviceaccount/cert-manager-webhook serverside-applied
@@ -1187,11 +1211,11 @@ deployment.apps/cert-manager serverside-applied
deployment.apps/cert-manager-webhook serverside-applied
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook serverside-applied
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/local-ca/local-ca.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/local-ca/local-ca.gen.yaml
clusterissuer.cert-manager.io/local-ca serverside-applied
+ kubectl wait --for=condition=Ready clusterissuer/local-ca --timeout=30s
clusterissuer.cert-manager.io/local-ca condition met
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/argocd/argocd.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/argocd/argocd.gen.yaml
serviceaccount/argocd-application-controller serverside-applied
serviceaccount/argocd-applicationset-controller serverside-applied
serviceaccount/argocd-notifications-controller serverside-applied
@@ -1238,13 +1262,13 @@ deployment.apps/argocd-server serverside-applied
statefulset.apps/argocd-application-controller serverside-applied
job.batch/argocd-redis-secret-init serverside-applied
referencegrant.gateway.networking.k8s.io/istio-ingress serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/app-projects/app-projects.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/app-projects/app-projects.gen.yaml
appproject.argoproj.io/bank-backend serverside-applied
appproject.argoproj.io/bank-frontend serverside-applied
appproject.argoproj.io/bank-security serverside-applied
appproject.argoproj.io/migration serverside-applied
appproject.argoproj.io/platform serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/istio-base/istio-base.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/istio-base/istio-base.gen.yaml
customresourcedefinition.apiextensions.k8s.io/authorizationpolicies.security.istio.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/destinationrules.networking.istio.io serverside-applied
customresourcedefinition.apiextensions.k8s.io/envoyfilters.networking.istio.io serverside-applied
@@ -1261,7 +1285,7 @@ customresourcedefinition.apiextensions.k8s.io/workloadentries.networking.istio.i
customresourcedefinition.apiextensions.k8s.io/workloadgroups.networking.istio.io serverside-applied
serviceaccount/istio-reader-service-account serverside-applied
validatingwebhookconfiguration.admissionregistration.k8s.io/istiod-default-validator serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/istiod/istiod.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/istiod/istiod.gen.yaml
serviceaccount/istiod serverside-applied
role.rbac.authorization.k8s.io/istiod serverside-applied
clusterrole.rbac.authorization.k8s.io/istio-reader-clusterrole-istio-system serverside-applied
@@ -1279,7 +1303,7 @@ poddisruptionbudget.policy/istiod serverside-applied
horizontalpodautoscaler.autoscaling/istiod serverside-applied
mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector serverside-applied
validatingwebhookconfiguration.admissionregistration.k8s.io/istio-validator-istio-system serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/istio-cni/istio-cni.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/istio-cni/istio-cni.gen.yaml
serviceaccount/istio-cni serverside-applied
configmap/istio-cni-config serverside-applied
clusterrole.rbac.authorization.k8s.io/istio-cni serverside-applied
@@ -1291,20 +1315,20 @@ clusterrolebinding.rbac.authorization.k8s.io/istio-cni-ambient serverside-applie
daemonset.apps/istio-cni-node serverside-applied
+ kubectl wait --for=condition=Ready pod -l k8s-app=istio-cni-node --timeout=300s -n istio-system
pod/istio-cni-node-7kfbh condition met
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/istio-ztunnel/istio-ztunnel.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/istio-ztunnel/istio-ztunnel.gen.yaml
serviceaccount/ztunnel serverside-applied
daemonset.apps/ztunnel serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/istio-gateway/istio-gateway.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/istio-gateway/istio-gateway.gen.yaml
certificate.cert-manager.io/gateway-cert serverside-applied
gateway.gateway.networking.k8s.io/default serverside-applied
serviceaccount/default-istio serverside-applied
+ kubectl wait --for=condition=Ready pod -l istio.io/gateway-name=default --timeout=300s -n istio-ingress
pod/default-istio-54598d985b-69wmr condition met
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/httproutes/httproutes.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/httproutes/httproutes.gen.yaml
httproute.gateway.networking.k8s.io/argocd serverside-applied
httproute.gateway.networking.k8s.io/bank serverside-applied
httproute.gateway.networking.k8s.io/migration-podinfo serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-secrets/bank-secrets.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-secrets/bank-secrets.gen.yaml
configmap/jwt-key-writer serverside-applied
job.batch/jwt-key-writer serverside-applied
role.rbac.authorization.k8s.io/jwt-key-reader serverside-applied
@@ -1314,7 +1338,7 @@ rolebinding.rbac.authorization.k8s.io/jwt-key-writer serverside-applied
serviceaccount/jwt-key-writer serverside-applied
+ kubectl wait --for=condition=complete job.batch/jwt-key-writer -n bank-security --timeout=300s
job.batch/jwt-key-writer condition met
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-backend-config/bank-backend-config.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-backend-config/bank-backend-config.gen.yaml
configmap/demo-data-config serverside-applied
configmap/environment-config serverside-applied
configmap/service-api-config serverside-applied
@@ -1322,30 +1346,30 @@ externalsecret.external-secrets.io/jwt-key serverside-applied
referencegrant.gateway.networking.k8s.io/istio-ingress serverside-applied
secretstore.external-secrets.io/bank-security serverside-applied
serviceaccount/bank-of-holos serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-accounts-db/bank-accounts-db.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-accounts-db/bank-accounts-db.gen.yaml
configmap/accounts-db-config serverside-applied
service/accounts-db serverside-applied
statefulset.apps/accounts-db serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-ledger-db/bank-ledger-db.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-ledger-db/bank-ledger-db.gen.yaml
configmap/ledger-db-config serverside-applied
service/ledger-db serverside-applied
statefulset.apps/ledger-db serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-contacts/bank-contacts.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-contacts/bank-contacts.gen.yaml
deployment.apps/contacts serverside-applied
service/contacts serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-balance-reader/bank-balance-reader.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-balance-reader/bank-balance-reader.gen.yaml
deployment.apps/balancereader serverside-applied
service/balancereader serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-userservice/bank-userservice.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-userservice/bank-userservice.gen.yaml
deployment.apps/userservice serverside-applied
service/userservice serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-ledger-writer/bank-ledger-writer.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-ledger-writer/bank-ledger-writer.gen.yaml
deployment.apps/ledgerwriter serverside-applied
service/ledgerwriter serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-transaction-history/bank-transaction-history.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-transaction-history/bank-transaction-history.gen.yaml
deployment.apps/transactionhistory serverside-applied
service/transactionhistory serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/components/bank-frontend/bank-frontend.gen.yaml
+ kubectl apply --server-side=true -f deploy/clusters/local/components/bank-frontend/bank-frontend.gen.yaml
configmap/demo-data-config serverside-applied
configmap/environment-config serverside-applied
configmap/service-api-config serverside-applied
@@ -1355,7 +1379,7 @@ referencegrant.gateway.networking.k8s.io/istio-ingress serverside-applied
secretstore.external-secrets.io/bank-security serverside-applied
service/frontend serverside-applied
serviceaccount/bank-of-holos serverside-applied
+ kubectl apply --server-side=true -f deploy/clusters/workload/gitops
+ kubectl apply --server-side=true -f deploy/clusters/local/gitops
application.argoproj.io/app-projects serverside-applied
application.argoproj.io/argocd-crds serverside-applied
application.argoproj.io/argocd serverside-applied
@@ -1428,10 +1452,10 @@ for some time.
[Quickstart]: /docs/quickstart/
[Change a Service]: /docs/guides/change-a-service/
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize
[ComponentFields]: /docs/api/author/v1alpha3/#ComponentFields
[Helm]: /docs/api/author/v1alpha4/#Helm
[Kubernetes]: /docs/api/author/v1alpha4/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha4/#Kustomize
[ComponentConfig]: /docs/api/author/v1alpha4/#ComponentConfig
[platform-files]: /docs/quickstart/#how-platform-rendering-works
[AppProject]: https://argo-cd.readthedocs.io/en/stable/user-guide/projects/
[unification operator]: https://cuelang.org/docs/reference/spec/#unification

File diff suppressed because it is too large Load Diff

View File

Before

Width:  |  Height:  |  Size: 690 KiB

After

Width:  |  Height:  |  Size: 690 KiB

View File

Before

Width:  |  Height:  |  Size: 997 KiB

After

Width:  |  Height:  |  Size: 997 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 287 KiB

After

Width:  |  Height:  |  Size: 287 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1009 KiB

After

Width:  |  Height:  |  Size: 1009 KiB

View File

Before

Width:  |  Height:  |  Size: 617 KiB

After

Width:  |  Height:  |  Size: 617 KiB

View File

Before

Width:  |  Height:  |  Size: 706 KiB

After

Width:  |  Height:  |  Size: 706 KiB

View File

Before

Width:  |  Height:  |  Size: 794 KiB

After

Width:  |  Height:  |  Size: 794 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 248 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 206 KiB

File diff suppressed because it is too large Load Diff

View File

Before

Width:  |  Height:  |  Size: 934 KiB

After

Width:  |  Height:  |  Size: 934 KiB

View File

Before

Width:  |  Height:  |  Size: 703 KiB

After

Width:  |  Height:  |  Size: 703 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 728 KiB

After

Width:  |  Height:  |  Size: 728 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 854 KiB

After

Width:  |  Height:  |  Size: 854 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 624 KiB

After

Width:  |  Height:  |  Size: 624 KiB

View File

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 116 KiB

View File

@@ -0,0 +1,52 @@
---
description: Introduction
---
# Introduction
Welcome to Holos. Holos is an open source tool to manage software development
platforms safely, easily, and consistently. We built Holos to help engineering
teams work more efficiently together by empowering them to build golden paths
and paved roads for other teams to leverage for quicker delivery.
## Documentation
- [Guides] are organized into example use-cases of how Holos helps engineering
teams at the fictional Bank of Holos deliver business value on the bank's
platform.
- The [API Reference] is a technical reference for when you're writing CUE code to define your platform.
## Backstory
At [Open Infrastructure Services], we've each helped dozens of companies build and operate their software development platforms. During the U.S. presidential election just before the pandemic, our second-largest client, Twitter, experienced a global outage that lasted nearly a full day. We were managing their production configuration system, allowing the core infrastructure team to focus on business-critical objectives. This gave us a front-row seat to the incident.
A close friend and engineer on the team made a trivial one-line change to the firewall configuration. Less than 30 minutes later, everything was down. That change, which passed code review, caused the host firewall to revert to its default state on hundreds of thousands of servers, blocking all connections globally—except for SSH, thankfully. Even a Presidential candidate complained loudly.
This incident forced us to reconsider key issues with Twitter's platform:
1. **Lack of Visibility** - Engineers couldn't foresee the impact of even a small change, making it difficult to assess risks.
2. **Large Blast Radius** - Small changes affected the entire global fleet in under 30 minutes. There was no way to limit the impact of a single change.
3. **Incomplete Tooling** - The right processes were in place, but the tooling didn't fully support them. The change was tested and reviewed, but critical information wasn't surfaced in time.
Over the next few years, we built features to address these issues. Meanwhile, I began exploring how these solutions could work in the Kubernetes and cloud-native space.
As Google Cloud partners, we worked with large customers to understand how they built their platforms on Kubernetes. During the pandemic, we built a platform using CNCF projects like ArgoCD, Prometheus Stack, Istio, Cert Manager, and External Secrets Operator, integrating them into a cohesive platform. We started with upstream recommendations—primarily Helm charts—and wrote scripts to integrate each piece into the platform. For example, we passed Helm outputs to Kustomize to add labels or fix bugs, and wrote umbrella charts to add Ingress, HTTPRoute, and ExternalSecret resources.
These scripts served as necessary glue to hold everything together but became difficult to manage across multiple environments, regions, and cloud providers. YAML templates and nested loops created friction, making them hard to troubleshoot. The scripts themselves made it difficult to see what was happening and to fix issues affecting the entire platform.
Still, the scripts had a key advantage: they produced fully rendered manifests in plain text, committed to version control, and applied via ArgoCD. This clarity made troubleshooting easier and reduced errors in production.
Despite the makeshift nature of the scripts, I kept thinking about the "[Why are we templating YAML]?" post on Hacker News. I wanted to replace our scripts and charts with something more robust and easier to maintain—something that addressed Twitter's issues head-on.
I rewrote our scripts and charts using CUE and Go, replacing the glue layer. The result is **Holos**—a tool designed to complement Helm, Kustomize, and Jsonnet, making it easier and safer to define golden paths and paved roads without bespoke scripts or templates.
Thanks for reading. Take Holos for a spin on your local machine with our [Quickstart] guide.
[Guides]: /docs/guides/
[API Reference]: /docs/api/
[Quickstart]: /docs/quickstart/
[CUE]: https://cuelang.org/
[Author API]: /docs/api/author/
[Core API]: /docs/api/core/
[Open Infrastructure Services]: https://openinfrastructure.co/
[Why are we templating YAML]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=https%3A%2F%2Fleebriggs.co.uk%2Fblog%2F2019%2F02%2F07%2Fwhy-are-we-templating-yaml&sort=byDate&type=story

View File

@@ -42,9 +42,9 @@ graph TB
Kustomize[<a href="#component">Kustomize</a>]
CUE[<a href="#component">CUE</a>]
Cluster --> Platform
Fleet --> Cluster
Component --> Fleet
Fleet --> Platform
Cluster --> Fleet
Component --> Cluster
Helm --> Component
Kustomize --> Component
CUE --> Component

View File

@@ -8,7 +8,7 @@ sidebar_position: 900
## Community Support
You can ask questions in our community forums in [GitHub Discussions](https://github.com/holos-run/holos/discussions) or [Google Groups](https://groups.google.com/g/holos-discuss).
You can ask questions in our community forums in [GitHub Discussions](https://github.com/holos-run/holos/discussions), [Discord](https://discord.gg/JgDVbNpye7), or [Google Groups](https://groups.google.com/g/holos-discuss).
## Commercial Support and Services

View File

@@ -1,8 +1,14 @@
---
slug: technical-overview
title: Technical Overview
description: Learn how Holos makes it easier for platform teams to integrate software into their platform.
---
<head>
<meta property="og:title" content="Technical Overview | Holos" />
<meta property="og:image" content="/img/cards/technical-overview.png" />
</head>
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
@@ -45,35 +51,34 @@ Go command line tool leveraging [CUE] to fill this gap.
```mermaid
---
title: Figure 1 - Render Pipeline
title: Figure 1 - v1alpha4 Rendered Manifest Pipeline
---
graph LR
PS[<a href="/docs/api/author/v1alpha3/#Platform">Platform</a>]
HC[<a href="/docs/api/author/v1alpha3/#ComponentFields">Components</a>]
BP[<a href="/docs/api/core/v1alpha3#BuildPlan">BuildPlan</a>]
Platform[<a href="/docs/api/author/v1alpha4/#Platform">Platform</a>]
Component[<a href="/docs/api/author/v1alpha4/#ComponentConfig">Components</a>]
H[<a href="/docs/api/author/v1alpha3/#Helm">Helm</a>]
K[<a href="/docs/api/author/v1alpha3/#Kustomize">Kustomize</a>]
O[<a href="/docs/api/author/v1alpha3/#Kubernetes">Kubernetes</a>]
Helm[<a href="/docs/api/author/v1alpha4/#Helm">Helm</a>]
Kustomize[<a href="/docs/api/author/v1alpha4/#Kustomize">Kustomize</a>]
Kubernetes[<a href="/docs/api/author/v1alpha4/#Kubernetes">Kubernetes</a>]
P[<a href="/docs/api/core/v1alpha3#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
BuildPlan[<a href="/docs/api/core/v1alpha4/#buildplan">BuildPlan</a>]
C[Kube API Server]
ResourcesArtifact[<a href="/docs/api/core/v1alpha4/#artifact">Resources<br/>Artifact</a>]
GitOpsArtifact[<a href="/docs/api/core/v1alpha4/#artifact">GitOps<br/>Artifact</a>]
PS --> HC --> BP
BP --> H --> P
BP --> K --> P
BP --> O --> P
Generators[<a href="/docs/api/core/v1alpha4/#generators">Generators</a>]
Transformers[<a href="/docs/api/core/v1alpha4/#transformer">Transformers</a>]
Files[Manifest<br/>Files]
P --> Y --> FS
P --> G --> FS
Platform --> Component
Component --> Helm --> BuildPlan
Component --> Kubernetes --> BuildPlan
Component --> Kustomize --> BuildPlan
FS --> ArgoCD --> C
FS --> Flux --> C
FS --> kubectl --> C
BuildPlan --> ResourcesArtifact --> Generators
BuildPlan --> GitOpsArtifact --> Generators
Generators --> Transformers --> Files
```
## Use Case
@@ -108,7 +113,7 @@ The development team registers their experimental project, creatively named
package holos
// The development team registers a project name.
#Projects: experiment: {
_Projects: experiment: {
// The project owner must be named.
Owner: Name: "dev-team"
// Expose Service podinfo at https://podinfo.example.com
@@ -139,7 +144,7 @@ HTTPRoute and AppProject go into two namespaces managed by the platform team.
Holos makes it easier for the platform team to organize these resources into
different components with different owners.
:::tip
:::important
Holos supports [CODEOWNERS] by clearly defining the teams responsible for each
platform component.
:::
@@ -152,12 +157,13 @@ holos render platform ./platform
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered namespaces for cluster overview in 93.024042ms
rendered projects for cluster overview in 96.080667ms
rendered httproutes for cluster overview in 96.047ms
rendered platform in 96.805292ms
rendered httproutes for cluster overview in 177.823625ms
rendered app-projects for cluster overview in 180.946834ms
rendered projects for cluster overview in 181.98725ms
rendered namespaces for cluster overview in 182.30725ms
rendered platform in 182.31075ms
```
:::note
:::tip
If you'd like to try this for yourself, `cd` into [examples/tech-overview] and
render the platform.
:::
@@ -169,112 +175,125 @@ The fully rendered manifests are written into the `deploy/` directory organized
by cluster and component for GitOps.
<Tabs groupId="07FBE14E-E9EA-437B-9FA1-C6D8806524AD">
<TabItem value="deploy/clusters/overview/components/namespaces/namespaces.gen.yaml" label="namespaces">
<TabItem value="deploy/clusters/local/components/namespaces/namespaces.gen.yaml" label="namespaces">
```
cat deploy/clusters/local/components/namespaces/namespaces.gen.yaml
```
```yaml showLineNumbers
# deploy/clusters/overview/components/namespaces/namespaces.gen.yaml
---
metadata:
name: experiment
labels:
kubernetes.io/metadata.name: experiment
example.com/project.name: experiment
example.com/owner.name: dev-team
example.com/owner.email: sg-dev-team@example.com
kind: Namespace
apiVersion: v1
kind: Namespace
metadata:
labels:
argocd.argoproj.io/instance: namespaces
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: namespaces
kubernetes.io/metadata.name: experiment
name: experiment
```
</TabItem>
<TabItem value="deploy/clusters/overview/components/projects/projects.gen.yaml" label="projects">
<TabItem value="deploy/clusters/local/components/projects/projects.gen.yaml" label="projects">
```
cat deploy/clusters/local/components/projects/projects.gen.yaml
```
```yaml showLineNumbers
# deploy/clusters/overview/components/projects/projects.gen.yaml
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: istio-ingress
namespace: experiment
labels:
example.com/project.name: experiment
example.com/owner.name: dev-team
argocd.argoproj.io/instance: projects
example.com/owner.email: sg-dev-team@example.com
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: istio-ingress
to:
- group: ""
kind: Service
---
metadata:
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: projects
name: admin
namespace: experiment
labels:
example.com/project.name: experiment
example.com/owner.name: dev-team
example.com/owner.email: sg-dev-team@example.com
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: oidc:sg-dev-team@example.com
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: oidc:sg-dev-team@example.com
---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
labels:
argocd.argoproj.io/instance: projects
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: projects
name: default
namespace: experiment
labels:
example.com/project.name: experiment
example.com/owner.name: dev-team
example.com/owner.email: sg-dev-team@example.com
spec:
provider:
kubernetes:
remoteNamespace: experiment
auth:
token:
bearerToken:
key: token
name: eso-reader
remoteNamespace: experiment
server:
caBundle: LS0tLS1CRUd...QVRFLS0tLS0K
url: https://management.example.com:6443
caBundle: LS0tLS1CRUd...S0tLS0K
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
labels:
argocd.argoproj.io/instance: projects
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: projects
name: istio-ingress
namespace: experiment
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: istio-ingress
to:
- group: ""
kind: Service
```
</TabItem>
<TabItem value="deploy/clusters/overview/components/httproutes/httproutes.gen.yaml" label="httproutes">
<TabItem value="deploy/clusters/local/components/httproutes/httproutes.gen.yaml" label="httproutes">
```
cat deploy/clusters/local/components/httproutes/httproutes.gen.yaml
```
```yaml showLineNumbers
# deploy/clusters/overview/components/httproutes/httproutes.gen.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: podinfo.holos.localhost
namespace: istio-ingress
labels:
example.com/project.name: experiment
example.com/owner.name: dev-team
argocd.argoproj.io/instance: httproutes
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: httproutes
name: podinfo.example.com
namespace: istio-ingress
spec:
hostnames:
- podinfo.holos.localhost
- podinfo.example.com
parentRefs:
- name: default
namespace: istio-ingress
- name: default
namespace: istio-ingress
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: podinfo
namespace: experiment
port: 9898
- backendRefs:
- name: podinfo
namespace: experiment
port: 9898
matches:
- path:
type: PathPrefix
value: /
```
</TabItem>
</Tabs>
@@ -282,7 +301,7 @@ spec:
The rendered manifests are derived from the project registration information by
definitions implemented by the platform team. The [Author API] provides a
[Project] schema, but does not define an implementation. The platform team
implements the [Project] schema by writing a `#Projects` definition to manage
implements the [Project] schema by adding a `_Projects` struct to manage
resources according to bank policies.
:::important
@@ -290,7 +309,7 @@ The Author API is intended as a convenient, ergonomic reference for component
authors. Definitions **are not** confined to the Author API.
:::
The following example shows how the platform team wrote the `#Projects`
The following example shows how the platform team wrote the `_Projects`
definition to derive the Namespace from the project registration provided by the
dev team.
@@ -302,97 +321,95 @@ projects/platform/components/namespaces/namespaces.cue
```cue showLineNumbers
package holos
let Objects = {
Name: "namespaces"
Resources: Namespace: #Namespaces
_Kubernetes: #Kubernetes & {
Name: "namespaces"
Resources: Namespace: _Namespaces
}
// Produce a kubernetes objects build plan.
(#Kubernetes & Objects).BuildPlan
_Kubernetes.BuildPlan
```
1. This is the namespaces component which simply manages all of the namespaces derived from the project registration data shown in the second tab.
1. This is the namespaces component which manages a collection of Namespace resources derived from the project registration data shown in the second tab.
2. Line 5 manages a Namespace for each value of the `#Namespaces` struct. See the second tab for how the platform team defines this structure.
</TabItem>
<TabItem value="projects/projects.cue" label="#Projects Definition">
<TabItem value="projects/projects.cue" label="Projects Definition">
```txt
projects/projects.cue
```
```cue showLineNumbers
package holos
import api "github.com/holos-run/holos/api/author/v1alpha3"
import api "github.com/holos-run/holos/api/author/v1alpha4"
// Projects defines the structure other teams register with to manage project
// resources. The platform team defines the schema, development teams provide
// the values.
#Projects: api.#Projects & {
[NAME=string]: {
Name: NAME
// The platform team requires the development teams to indicate an owner of
// the project.
Owner: Name: string
// The default value for the owner email address is derived from the owner
// name, but development teams can provide a different email address if
// needed.
Owner: Email: string | *"sg-\(Owner.Name)@\(#Organization.Domain)"
// The platform team constrains the project to a single namespace.
Namespaces: close({(NAME): Name: NAME})
// The platform team constrains the exposed services to the project
// namespace.
Hostnames: [HOST=string]: {
Name: HOST
Namespace: Namespaces[NAME].Name
Service: HOST
Port: number | *80
}
_Projects: api.#Projects & {
[NAME=string]: {
Name: NAME
// The platform team requires the development teams to indicate an owner of
// the project.
Owner: Name: string
// The default value for the owner email address is derived from the owner
// name, but development teams can provide a different email address if
// needed.
Owner: Email: string | *"sg-\(Owner.Name)@\(_Organization.Domain)"
// The platform team constrains the project to a single namespace.
Namespaces: close({(NAME): Name: NAME})
// The platform team constrains the exposed services to the project
// namespace.
Hostnames: [HOST=string]: {
Name: HOST
Namespace: Namespaces[NAME].Name
Service: HOST
Port: number | *80
}
// CommonLabels is not part of the Projects API, so we use a hidden field to
// provide common labels to components that render resources from CUE.
_CommonLabels: {
"\(#Organization.Domain)/project.name": Name
"\(#Organization.Domain)/owner.name": Owner.Name
"\(#Organization.Domain)/owner.email": Owner.Email
}
}
CommonLabels: {
"\(_Organization.Domain)/project.name": Name
"\(_Organization.Domain)/owner.name": Owner.Name
"\(_Organization.Domain)/owner.email": Owner.Email
}
}
}
for Project in #Projects {
// Register project namespaces with the namespaces component.
#Namespaces: {
for Namespace in Project.Namespaces {
(Namespace.Name): metadata: labels: Project._CommonLabels
}
}
for Project in _Projects {
// Register project namespaces with the namespaces component.
_Namespaces: {
for Namespace in Project.Namespaces {
(Namespace.Name): metadata: labels: Project.CommonLabels
}
}
}
```
1. On lines 8-37 the platform team derives most fields from the project name (line 9), and the owner name (line 13). The purpose is to fill in the remaining fields defined by the Author API.
1. On lines 8-35 the platform team derives most fields from the project name (line 9), and the owner name (line 13). The purpose is to fill in the remaining fields defined by the Author API.
2. Line 13 The dev team is expected to provide a concrete owner name, indicated by the `string` value.
3. Line 17 The platform team provides a default value for the email address. The project team may define a different value.
4. Line 19 The Author API allows a project to have many namespaces. The platform team constrains this down to one namespace per project by closing the struct. The namespace name must be the same as the project name.
5. Lines 22-27 The platform team derives values for a Gateway API [BackendObjectReference] from the hostname provided by the project team. These values are used later to build HTTPRoutes to expose their service.
6. Lines 31-35 Common labels aren't part of the Author API, so the platform team defines a hidden field to make them available throughout the configuration.
7. Lines 39-46 The platform team adds a namespace with common labels for each project to the struct we saw in the first tab.
6. Lines 30-32 Common labels are derived to mix into resources associated with this project.
7. Lines 37-44 The platform team adds a namespace with common labels for each project to the struct we saw in the first tab.
</TabItem>
</Tabs>
The RoleBinding, SecretScore, and ReferenceGrant are managed in the
[projects](https://github.com/holos-run/bank-of-holos/blob/v0.1.1/examples/tech-overview/projects/platform/components/projects/projects.cue)
[projects](https://github.com/holos-run/bank-of-holos/blob/v0.4.1/examples/tech-overview/projects/platform/components/projects/projects.cue)
component, similar to the previous namespaces example.
The HTTPRoute is managed separately in the
[httproutes](https://github.com/holos-run/bank-of-holos/blob/v0.1.1/examples/tech-overview/projects/platform/components/httproutes/httproutes.cue)
[httproutes](https://github.com/holos-run/bank-of-holos/blob/v0.4.1/examples/tech-overview/projects/platform/components/httproutes/httproutes.cue)
component.
All components are registered with the platform in the
[platform](https://github.com/holos-run/bank-of-holos/tree/v0.1.1/examples/tech-overview/platform)
[platform](https://github.com/holos-run/bank-of-holos/tree/v0.4.1/examples/tech-overview/platform)
directory.
:::important
Multiple components, potentially owned by different teams, derive fully rendered
resources from the same three project values. The dev team added these three
values to the `#Projects` definition. The platform team wrote the definition to
values to the `_Projects` struct. The platform team wrote the definition to
integrate software according to bank policies. CUE powers this _unified_
platform configuration model.
:::
@@ -416,13 +433,17 @@ projects/experiment/components/podinfo/podinfo.cue
package holos
// Produce a helm chart build plan.
(#Helm & Chart).BuildPlan
_HelmChart.BuildPlan
let Chart = {
Name: "podinfo"
Version: "6.6.2"
Repo: name: "podinfo"
Repo: url: "https://stefanprodan.github.io/podinfo"
_HelmChart: #Helm & {
Name: "podinfo"
Chart: {
version: "6.6.2"
repository: {
name: "podinfo"
url: "https://stefanprodan.github.io/podinfo"
}
}
}
```
This file represents a Helm chart component to add to the platform. The second
@@ -436,11 +457,13 @@ platform/podinfo.cue
package holos
// Manage the component on every workload Cluster, but not management clusters.
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name)/podinfo": {
path: "projects/experiment/components/podinfo"
cluster: Cluster.name
}
for Cluster in _Fleets.workload.clusters {
_Platform: Components: "\(Cluster.name):podinfo": {
name: "podinfo"
component: "projects/experiment/components/podinfo"
cluster: Cluster.name
tags: project: "experiment"
}
}
```
This file registers the component with the platform. When the platform is
@@ -449,6 +472,14 @@ across the platform.
</TabItem>
</Tabs>
The project tag links the component to the same field of the `_Projects` struct.
:::important
You can add your own key=value tags in your platform specification to inject
values into components. This feature is useful to reuse one component path for
several environments or customers.
:::
Once the dev team's component is registered, rendering the platform will render
their component.
@@ -460,13 +491,13 @@ holos render platform ./platform
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered app-projects for cluster overview in 92.087042ms
rendered projects for cluster overview in 95.6325ms
rendered httproutes for cluster overview in 96.968916ms
rendered namespaces for cluster overview in 97.610291ms
rendered namespaces for cluster overview in 185.64075ms
rendered app-projects for cluster overview in 186.729292ms
rendered httproutes for cluster overview in 195.222833ms
rendered projects for cluster overview in 195.217125ms
// highlight-next-line
rendered podinfo for cluster overview in 155.410417ms
rendered platform in 155.470542ms
rendered podinfo for cluster overview in 195.830042ms
rendered platform in 195.90275ms
```
</TabItem>
</Tabs>
@@ -474,7 +505,7 @@ rendered platform in 155.470542ms
<Tabs groupId="77BF500B-105A-4AB4-A615-DEC19F501AE1">
<TabItem value="command" label="Command">
```bash
cat deploy/clusters/overview/components/podinfo/podinfo.gen.yaml
cat deploy/clusters/local/components/podinfo/podinfo.gen.yaml
```
</TabItem>
<TabItem value="output" label="Output">
@@ -486,12 +517,13 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
helm.sh/chart: podinfo-6.6.2
holos.run/component.name: podinfo
name: podinfo
namespace: experiment
spec:
ports:
- name: http
@@ -504,9 +536,11 @@ spec:
targetPort: grpc
selector:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: podinfo
type: ClusterIP
---
apiVersion: apps/v1
@@ -516,20 +550,23 @@ metadata:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo
app.kubernetes.io/version: 6.6.2
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
helm.sh/chart: podinfo-6.6.2
holos.run/component.name: podinfo
name: podinfo
namespace: experiment
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: podinfo
strategy:
rollingUpdate:
maxUnavailable: 1
@@ -541,9 +578,11 @@ spec:
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: podinfo
argocd.argoproj.io/instance: podinfo
example.com/owner.email: sg-dev-team@example.com
example.com/owner.name: dev-team
example.com/project.name: experiment
holos.run/component.name: podinfo
spec:
containers:
- command:
@@ -617,7 +656,8 @@ platform team added a constraint to the project so all Helm charts are post
processed with Kustomize to add these common labels. The platform team
accomplishes this by adding a constraint in the project directory. This can be
seen in
[experiment/components.cue](https://github.com/holos-run/bank-of-holos/blob/v0.1.1/examples/tech-overview/projects/experiment/components.cue)
[schema.cue](https://github.com/holos-run/bank-of-holos/blob/v0.4.1/schema.cue#L35-L38)
where the platform team configures all component kinds for the platform.
We've covered how the platform team provides a golden path for development teams
to register their projects by defining a Projects structure. We've also covered
@@ -644,7 +684,7 @@ how the development team deploys their existing Helm chart onto the platform.
[Holos]: https://holos.run/
[Quickstart]: /docs/quickstart/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern/
[examples/tech-overview]: https://github.com/holos-run/bank-of-holos/tree/v0.1.1/examples/tech-overview
[examples/tech-overview]: https://github.com/holos-run/bank-of-holos/tree/v0.2.0/examples/tech-overview
[BackendObjectReference]: https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.BackendObjectReference
[CODEOWNERS]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
[Project]: /docs/api/author/v1alpha3/#Project

View File

@@ -1,5 +1,3 @@
import DocCardList from '@theme/DocCardList';
# Author Schema
# Author API
<DocCardList />
v1alpha5

View File

@@ -1,5 +1,3 @@
import DocCardList from '@theme/DocCardList';
# Core Schema
# Core API
<DocCardList />
v1alpha5

View File

@@ -1,15 +0,0 @@
---
description: Helm Component
slug: /guides/helm-component
sidebar_position: 400
---
# Helm Component
The [Deploy a Service](/docs/guides/deploy-a-service/) guide is the best guide
we have on wrapping a Helm chart in a Holos Component. The [Helm] section of
the Author API may also be useful.
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize

View File

@@ -1,20 +0,0 @@
---
description: Kubernetes Component
slug: /guides/kubernetes-component
sidebar_position: 500
---
# Kubernetes Component
:::warning
TODO
:::
This is a placeholder for a guide for managing Kubernetes resources directly
from a Holos Component with strong type checking.
In the meantime, please refer to the [Kubernetes] section of the Author API.
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize

View File

@@ -1,20 +0,0 @@
---
description: Wrap a Kustomize Kustomization in a Holos Component.
slug: /guides/kustomize-component
sidebar_position: 600
---
# Kustomize Component
:::warning
TODO
:::
This is a placeholder for a guide on wrapping a Kustomize Kustomization base
with a Holos component.
In the meantime, please refer to the [Kustomize] section of the Author API.
[Helm]: /docs/api/author/v1alpha3/#Helm
[Kubernetes]: /docs/api/author/v1alpha3/#Kubernetes
[Kustomize]: /docs/api/author/v1alpha3/#Kustomize

View File

@@ -1,830 +0,0 @@
---
description: Try Holos with this quick start guide.
slug: /quickstart
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Quickstart
Welcome to the Holos Quickstart guide. Holos is an open source tool to manage
software development platforms safely, easily, and consistently. We'll use
Holos to manage a fictional bank's platform, the Bank of Holos. In doing so
we'll take the time to explain the foundational concepts of Holos.
1. **Platform** - Holos breaks a Platform down into Components owned by teams.
2. **Component** - Components are CUE wrappers around unmodified upstream
vendor Helm Charts, Kustomize Bases, or plain Kubernetes manifests.
3. **CUE** - We write CUE to configure the platform. We'll cover the basics of
CUE syntax and why Holos uses CUE.
4. **Tree Unification** - CUE files are organized into a unified filesystem
tree. We'll cover how unification makes it easier and safer for multiple teams
to change the platform.
The Bank of Holos provides a good example of how Holos is designed to make it
easier for multiple teams to deliver services on a platform. These teams are:
- **Platform**
- **Software development**
- **Security**
- **Quality Assurance**
In this guide we'll show how Holos enables teams to work autonomously
while still allowing the platform team to enforce the standards and policies
they care about to provide a secure and consistent software development platform.
Here's a screenshot of the retail banking application we'll build and deploy on
our platform. We'll keep each of these teams in mind as we work through the
guides. Each of our guides focuses on different aspects of delivering the Bank
of Holos.
![Bank of Holos](./img/bank-home.png)
## What you'll need {#requirements}
This guide is intended to be informative without needing to run the commands.
If you'd like to render the platform and apply the manifests to a real Cluster,
complete the [Local Cluster Guide](/docs/guides/local-cluster) before this
guide.
You'll need the following tools installed to run the commands in this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap Helm charts.
3. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to render Holos
Components that render with Kustomize.
## Install Holos
Start by installing the `holos` command line tool with the following command.
If you don't have Go, refer to [Installation](/docs/install/) to download the
executable.
<Tabs groupId="go-install">
<TabItem value="command" label="Command">
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
</TabItem>
<TabItem value="output" label="Output">
```txt
go: downloading github.com/holos-run/holos v0.95.1
```
</TabItem>
</Tabs>
:::tip
Nearly all day-to-day platform management tasks use the `holos` command line
tool to render plain Kubernetes manifests.
:::
## Fork the Git Repository
Building a software development platform from scratch takes time so we've
published an example for our guides. [Fork the Bank of
Holos](https://github.com/holos-run/bank-of-holos/fork) to get started.
Clone the repository to your local machine.
<Tabs groupId="git-clone">
<TabItem value="command" label="Command">
```bash
# Change YourName
git clone https://github.com/YourName/bank-of-holos
cd bank-of-holos
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Cloning into 'bank-of-holos'...
remote: Enumerating objects: 1177, done.
remote: Counting objects: 100% (1177/1177), done.
remote: Compressing objects: 100% (558/558), done.
remote: Total 1177 (delta 394), reused 1084 (delta 303), pack-reused 0 (from 0)
Receiving objects: 100% (1177/1177), 2.89 MiB | 6.07 MiB/s, done.
Resolving deltas: 100% (394/394), done.
```
</TabItem>
</Tabs>
Run the rest of the commands in this guide from the root of the repository.
## Configuring GitOps {#configuring-gitops}
The Bank of Holos platform is organized as a collection of software components.
Each component represents a piece of software provided by an upstream vendor,
for example ArgoCD, or software developed in-house. Components are also used to
glue together, or integrate, other components into the platform.
The platform team provides ArgoCD as a means for teams to implement GitOps
within their software development workflow. Each team using the Bank of Holos
platform uses a Holos resource provided by the platform team to create their
ArgoCD Application definition. In doing so, the platform team has provided
a "golden path" for each team to independently make the changes they need
while still centrally enforcing the policies that provide a consistent and
safe experience.
Currently each team is using the upstream `bank-of-repo` repository as their
source of truth. We'll start by changing ArgoCD to point to our fork. This will
allow us to be able to see the results of our changes in ArgoCD using a GitOps
workflow.
<Tabs groupId="argocd-config">
<TabItem value="command" label="projects/argocd-config.cue">
```cue showLineNumbers
package holos
#ArgoConfig: {
Enabled: true
// highlight-next-line
RepoURL: "https://github.com/holos-run/bank-of-holos"
}
```
</TabItem>
</Tabs>
Change the RepoURL to the URL of your fork. For example:
<Tabs groupId="F3BF73E3-3A70-40AF-9D4D-7134AF0A1763">
<TabItem value="command" label="projects/argocd-config.cue">
```diff showLineNumbers
diff --git a/projects/argocd-config.cue b/projects/argocd-config.cue
index 5264f48..0214e99 100644
--- a/projects/argocd-config.cue
+++ b/projects/argocd-config.cue
@@ -2,5 +2,5 @@ package holos
#ArgoConfig: {
Enabled: true
- RepoURL: "https://github.com/holos-run/bank-of-holos"
+ RepoURL: "https://github.com/jeffmccune/bank-of-holos"
}
```
</TabItem>
</Tabs>
We need to render the platform manifests after we make changes.
## Render the Platform
Platform rendering is is the process of looping over all the components in the
platform and rendering each one into plain kubernetes manifest files. Holos is
designed to write plain manifest files which can be applied to Kubernetes, but
stops short of applying them so it's easier for team members to review and
understand changes before they're made.
<Tabs groupId="219C5B3D-1369-45F9-B010-64A87EF71190">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered bank-accounts-db for cluster workload in 142.661334ms
rendered bank-ledger-db for cluster workload in 144.041417ms
rendered bank-userservice for cluster workload in 157.828709ms
rendered bank-ledger-writer for cluster workload in 161.138292ms
rendered bank-backend-config for cluster workload in 168.923459ms
rendered bank-balance-reader for cluster workload in 171.877875ms
rendered bank-secrets for cluster workload in 207.958792ms
rendered gateway for cluster workload in 123.572583ms
rendered bank-contacts for cluster workload in 144.466291ms
rendered bank-transaction-history for cluster workload in 151.520041ms
rendered httproutes for cluster workload in 139.590834ms
rendered bank-frontend for cluster workload in 309.679834ms
rendered app-projects for cluster workload in 107.136083ms
rendered ztunnel for cluster workload in 160.679791ms
rendered cni for cluster workload in 238.937625ms
rendered cert-manager for cluster workload in 178.610834ms
rendered istiod for cluster workload in 340.953208ms
rendered argocd for cluster workload in 286.277ms
rendered local-ca for cluster workload in 98.720208ms
rendered external-secrets for cluster workload in 141.459708ms
rendered base for cluster workload in 454.356667ms
rendered namespaces for cluster workload in 115.401709ms
rendered gateway-api for cluster workload in 203.5625ms
rendered external-secrets-crds for cluster workload in 525.180209ms
rendered crds for cluster workload in 888.406167ms
rendered platform in 1.182857542s
```
</TabItem>
</Tabs>
Rendering the platform to plain manifest files allows us to see the changes
clearly. We can see this one line change affected dozens ArgoCD Application
resources across the platform.
<Tabs groupId="266D26D4-31FC-45D1-88EF-EAD23BBBDCDD">
<TabItem value="command" label="Command">
```bash
git status
```
</TabItem>
<TabItem value="output" label="Output">
```txt
On branch main
Your branch is up to date with 'origin/main'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: deploy/clusters/workload/gitops/app-projects.application.gen.yaml
modified: deploy/clusters/workload/gitops/argocd-crds.application.gen.yaml
modified: deploy/clusters/workload/gitops/argocd.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-accounts-db.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-backend-config.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-balance-reader.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-contacts.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-ledger-db.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-ledger-writer.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-secrets.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-transaction-history.application.gen.yaml
modified: deploy/clusters/workload/gitops/bank-userservice.application.gen.yaml
modified: deploy/clusters/workload/gitops/cert-manager.application.gen.yaml
modified: deploy/clusters/workload/gitops/external-secrets-crds.application.gen.yaml
modified: deploy/clusters/workload/gitops/external-secrets.application.gen.yaml
modified: deploy/clusters/workload/gitops/gateway-api.application.gen.yaml
modified: deploy/clusters/workload/gitops/httproutes.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-base.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-cni.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-gateway.application.gen.yaml
modified: deploy/clusters/workload/gitops/istio-ztunnel.application.gen.yaml
modified: deploy/clusters/workload/gitops/istiod.application.gen.yaml
modified: deploy/clusters/workload/gitops/local-ca.application.gen.yaml
modified: deploy/clusters/workload/gitops/namespaces.application.gen.yaml
modified: projects/argocd-config.cue
no changes added to commit (use "git add" and/or "git commit -a")
```
</TabItem>
</Tabs>
Take a look at the Application resource for the bank-frontend component to see
the changed `spec.source.repoURL` field.
<Tabs groupId="665E5402-FB42-4975-B654-3922EE73EE07">
<TabItem value="command" label="Command">
```bash
git diff deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
```
</TabItem>
<TabItem value="output" label="Output">
```diff showLineNumbers
diff --git a/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml b/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
index d8ede55..aed4338 100644
--- a/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
+++ b/deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml
@@ -9,5 +9,5 @@ spec:
project: bank-frontend
source:
path: ./deploy/clusters/workload/components/bank-frontend
- repoURL: https://github.com/holos-run/bank-of-holos
+ repoURL: https://github.com/jeffmccune/bank-of-holos
targetRevision: main
```
</TabItem>
</Tabs>
We'll add, commit, and push this change to our fork then take a little time to
explain what happened when we made the change and rendered the platform.
<Tabs groupId="BD6A968F-FFDF-486B-8EC0-BA8B39C19303">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m 'quickstart: change argocd repo url to our fork'
git push origin
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main f2f8bc2] quickstart: change argocd repo url to our fork
26 files changed, 26 insertions(+), 26 deletions(-)
Enumerating objects: 41, done.
Counting objects: 100% (41/41), done.
Delta compression using up to 14 threads
Compressing objects: 100% (31/31), done.
Writing objects: 100% (33/33), 2.95 KiB | 2.95 MiB/s, done.
Total 33 (delta 28), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (28/28), completed with 4 local objects.
To github.com:jeffmccune/bank-of-holos.git
c2951ec..f2f8bc2 main -> main
```
</TabItem>
</Tabs>
## Platform Rendering Explained
So what happens when we run `holos render platform`? We saw `holos` write plain
manifest files, let's dive into how and why we implemented platform rendering
like this.
```mermaid
---
title: Figure 1 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/author/v1alpha3/#Platform">Platform</a>]
HC[<a href="/docs/api/author/v1alpha3/#ComponentFields">Components</a>]
BP[<a href="/docs/api/core/v1alpha3#BuildPlan">BuildPlan</a>]
H[<a href="/docs/api/author/v1alpha3/#Helm">Helm</a>]
K[<a href="/docs/api/author/v1alpha3/#Kustomize">Kustomize</a>]
O[<a href="/docs/api/author/v1alpha3/#Kubernetes">Kubernetes</a>]
P[<a href="/docs/api/core/v1alpha3#Kustomize">Kustomize</a>]
Y[Kubernetes <br/>Resources]
G[GitOps <br/>Resource]
FS[Local Files]
C[Kube API Server]
PS --> HC --> BP
BP --> H --> P
BP --> K --> P
BP --> O --> P
P --> Y --> FS
P --> G --> FS
FS --> ArgoCD --> C
FS --> Flux --> C
FS --> kubectl --> C
```
### Why do we render the platform? {#why-render-the-platform}
We built Holos to make the process of managing a platform safer, easier, and
more consistent. Before Holos we used Helm, Kustomize, and scripts to glue
together all of the software that goes into a platform. Then we coaxed the
output of each tool into something that works with GitOps. This approach has a
number of shortcomings. We wanted to see the manifests before ArgoCD or Flux
applied them, so we wrote a lot of difficult to maintain scripts to get the
template output into something useful. We tried avoiding the scripts by having
ArgoCD handle the Helm charts directly, but we could no longer see the changes
clearly during code review.
The platform rendering process allows us to have it both ways. We avoid the
unsafe text templates and glue scripts by using CUE. We're able to review the
exact changes that _will be_ applied during code review because holos renders
the whole platform to plain manifest files.
Finally, because we usually make each change by rendering the whole platform,
we're able to see and consider how a single-line change, like the one we just
made, affects the whole platform. Before we made Holos we were frustrated with
how difficult it was to get this zoomed-out, broad perspective of each change we
made.
:::tip
Holos implements the [rendered manifests pattern] so you don't have to build it
yourself.
:::
### How does platform rendering work? {#how-platform-rendering-works}
Holos is declarative. CUE provides resources that declare what `holos` needs to
do. The output of `holos` is always the same for the same inputs, so `holos` is
also idempotent.
When we run `holos render platform`, CUE builds the Platform specification
(spec). This is a fancy way of saying a list of software to manage on each
cluster in the platform. The CUE files in the `platform` directory provide the
platform spec to `holos`.
Let's open up two of these CUE files to see how this works. Ignore the other
files for now, they behave the same as these two.
<Tabs groupId="6F01F2F7-C101-4212-A844-0E370B836B54">
<TabItem value="argocd" label="platform/argocd.cue">
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
// highlight-next-line
#Platform: Components: "\(Cluster.name)/argocd-crds": {
path: "projects/platform/components/argocd/crds"
cluster: Cluster.name
}
// highlight-next-line
#Platform: Components: "\(Cluster.name)/argocd": {
path: "projects/platform/components/argocd/argocd"
cluster: Cluster.name
}
}
}
```
</TabItem>
<TabItem value="external-secrets" label="platform/external-secrets.cue">
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
// highlight-next-line
#Platform: Components: "\(Cluster.name)/external-secrets-crds": {
path: "projects/platform/components/external-secrets-crds"
cluster: Cluster.name
}
// highlight-next-line
#Platform: Components: "\(Cluster.name)/external-secrets": {
path: "projects/platform/components/external-secrets"
cluster: Cluster.name
}
}
}
```
</TabItem>
</Tabs>
There's quite a few new concepts to unpack in these two CUE files.
1. A Fleet is just a collection of clusters that share a similar, but not
identical configuration. Most platforms have a management fleet with one
cluster to manage the platform, and a workload fleet for clusters that host the
services we deploy onto the platform.
2. A Cluster is a Kubernetes cluster. Each component is rendered to plain
manifests for a cluster.
:::important
On lines 6 and 10 we see a Component being assigned to the Platform. We also
start to dive into the syntax of CUE, which we need to understand a little
before going further.
:::
> In its simplest form, CUE looks a lot like JSON. This is because CUE is a
superset of JSON. Or, put differently: all valid JSON is CUE.[^1]
>
> 1. C-style comments are allowed
> 2. field names without special characters don't need to be quoted
> 3. commas after a field are optional (and are usually omitted)
> 4. commas after the final element of a list are allowed
> 5. **the outermost curly braces in a CUE file are optional**
>
> JSON objects are called structs in CUE. JSON arrays are called lists, Object
members are called fields, which link their name, or label, to a value.
There are two important things to know about CUE to understand these two files.
First, the curly braces have been omitted which is item 5 on the list above.
Second, CUE is all about _unification_. These files could have been written
like this:
<Tabs groupId="59FFCCB6-A584-42ED-AC37-1C2BDCF5A523">
<TabItem value="argocd" label="platform/argocd.cue">
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
#Platform: {
// highlight-next-line
// Define #Platform.Components
// highlight-next-line
Components: {
"\(Cluster.name)/argocd-crds": {
path: "projects/platform/components/argocd/crds"
cluster: Cluster.name
}
}
// highlight-next-line
// Define #Platform.Components again!? Error?
// highlight-next-line
Components: {
"\(Cluster.name)/argocd": {
path: "projects/platform/components/argocd/argocd"
cluster: Cluster.name
}
}
}
}
}
```
</TabItem>
<TabItem value="external-secrets" label="platform/external-secrets.cue">
```cue showLineNumbers
package holos
// Manage the component on every cluster in the platform
for Fleet in #Fleets {
for Cluster in Fleet.clusters {
#Platform: {
// highlight-next-line
// Define #Platform.Components
// highlight-next-line
Components: {
"\(Cluster.name)/external-secrets-crds": {
path: "projects/platform/components/external-secrets-crds"
cluster: Cluster.name
}
}
// highlight-next-line
// Define #Platform.Components again!? Error?
// highlight-next-line
Components: {
"\(Cluster.name)/external-secrets": {
path: "projects/platform/components/external-secrets"
cluster: Cluster.name
}
}
}
}
}
```
</TabItem>
</Tabs>
:::important
Unlike most other languages, it is common to declare the same field in multiple
places. CUE **unifies** the value of the field. We can think of CUE as a
Configuration Unification Engine.
:::
Now that we know curly braces can be omitted and values are unified, we can
understand how the rest of the CUE files in the platform directory behave.
:::tip
Each CUE file in the platform directory adds components to the
`#Platform.Components` struct.
:::
The final file in the directory is responsible for producing the Platform spec.
It looks like this.
<Tabs groupId="166F0925-9405-4571-A0AB-C7E2107876FD">
<TabItem value="command" label="platform/platform.gen.cue">
```cue showLineNumbers
package holos
#Platform.Output
```
</TabItem>
</Tabs>
This file provides the value of the `#Platform.Output` field, the platform spec,
to `holos`.
Let's take a look at that Output value:
<Tabs groupId="475C92AC-C6DA-4FB9-859C-722921277CFC">
<TabItem value="command" label="Command">
```bash
cue export --out yaml ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: Platform
apiVersion: v1alpha3
metadata:
name: guide
spec:
model: {}
components: # This is a trimmed list for readability.
- path: projects/bank-of-holos/security/components/bank-secrets
cluster: workload
- path: projects/bank-of-holos/frontend/components/bank-frontend
cluster: workload
- path: projects/platform/components/argocd/crds
cluster: workload
- path: projects/platform/components/argocd/argocd
cluster: workload
- path: projects/platform/components/external-secrets-crds
cluster: workload
- path: projects/platform/components/external-secrets
cluster: workload
```
</TabItem>
</Tabs>
:::tip
You don't normally need to execute `cue`, CUE is built into `holos`. We use it
here to gain insight.
:::
We see the platform spec is essentially a list of components, each assigned to a
cluster.
:::important
Notice CUE unifies `Components` from multiple files into one list.
We'll see this unification behavior again and again. Unification is the
defining characteristic of CUE that makes it a unique, powerful, and _safe_
configuration language.
:::
Holos takes this list of components and builds each one by executing:
```bash
holos render component --cluster-name="example" "path/to/the/component"
```
We can think of platform rendering as rendering a list of components, passing
the cluster name each time. Rendering each component writes the fully rendered
manifest for that component to the `deploy/` directory, organized by cluster for
GitOps.
## Render a Component
Rendering a component works much the same way as rendering a platform. `holos`
uses CUE to produce a specification, then processes it. The specification of a
component is called a BuildPlan. A BuildPlan is a list of zero or more
kubernetes resources, Helm charts, Kustomize bases, and additional files to
write into the `deploy/` directory.
Now let's look at the cert-manager component. Notice the
`platform/cert-manager.cue` file has the field `path:
"projects/platform/components/cert-manager"`. This path indicates where to
start working with the cert-manager component.
<Tabs groupId="129DD743-0FE3-44C0-ACA4-6569C98BA40E">
<TabItem value="cert-manager" label="projects/platform/components/cert-manager/cert-manager.cue">
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
// highlight-next-line
(#Helm & Chart).BuildPlan
// highlight-next-line
let Chart = {
Name: "cert-manager"
// #CertManager is defined in projects/cert-manager.cue
// highlight-next-line
Version: #CertManager.Version
// highlight-next-line
Namespace: #CertManager.Namespace
Repo: name: "jetstack"
Repo: url: "https://charts.jetstack.io"
// CUE offers type checking and validation of Helm values.
// highlight-next-line
Values: installCRDs: true
// highlight-next-line
Values: startupapicheck: enabled: false
}
```
</TabItem>
<TabItem value="root" label="projects/cert-manager.cue">
```cue showLineNumbers
package holos
// Platform wide configuration
#CertManager: {
// highlight-next-line
Version: "1.15.3"
// highlight-next-line
Namespace: "cert-manager"
}
// Register the namespace
// The underscore indicates the value is defined elsewhere in CUE.
#Namespaces: (#CertManager.Namespace): _
```
</TabItem>
</Tabs>
This file introduces a few new concepts.
1. Line 4 indicates this component produces a BuildPlan that wraps a Helm Chart.
2. On line 6 `let` binds a name to an expression for the current scope. The
current file in this case.
3. Notice Chart is referenced on line 4 before it's bound on line 6. **Order is
irrelevant in CUE**. Complex changes are simpler and easier when we don't have
to think about order.
4. The chart version and namespace are defined in a different file closer to the
root, `projects/cert-manager.cue`
5. We define Helm values in CUE to take advantage of strong type checking and
manage multiple Helm charts consistently with platform wide values.
Let's take a look at the BuildPlan that results from the CUE configuration
described above.
<Tabs groupId="B54D5791-4E5B-4148-A368-62D9BE80760C">
<TabItem value="command" label="Command">
```bash
cue export --out yaml ./projects/platform/components/cert-manager
```
</TabItem>
<TabItem value="output" label="Output">
```yaml showLineNumbers
kind: BuildPlan
apiVersion: v1alpha3
spec:
components:
resources:
gitops/cert-manager:
kind: KubernetesObjects
apiVersion: v1alpha3
metadata:
name: gitops/cert-manager
namespace: cert-manager
deployFiles:
clusters/no-name/gitops/cert-manager.application.gen.yaml: |
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cert-manager
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: platform
source:
path: ./deploy/clusters/no-name/components/cert-manager
repoURL: https://github.com/jeffmccune/bank-of-holos
targetRevision: main
skip: false
helmChartList:
- kind: HelmChart
apiVersion: v1alpha3
chart:
name: cert-manager
version: 1.15.3
release: cert-manager
repository:
name: jetstack
url: https://charts.jetstack.io
valuesContent: |
installCRDs: true
startupapicheck:
enabled: false
enableHooks: false
metadata:
name: cert-manager
namespace: cert-manager
apiObjectMap: {}
skip: false
```
</TabItem>
</Tabs>
:::important
Again, you don't normally need to execute `cue`, it's built into `holos`. We
use it here to show how Holos works with Helm.
:::
Looking at the BuildPlan, we see `holos` will render the Helm chart into the
deploy directory along with an ArgoCD Application resource in the `gitops/`
directory.
:::tip
The BuildPlan API is flexible enough to write any file into the `deploy/`
directory. Holos uses this flexibility to support both Flux and ArgoCD.
:::
When we run `cue export`, we get back a Core API BuildPlan. The BuildPlan is
produced by the `#Helm` definition on line 4 which is part of the Author API.
The Core API is the contract between CUE and `holos`. As such, it's not as
friendly as the Author API. The Author API is the contract component authors
and platform engineers use to configure and manage the platform. The Author API
is meant for people, the Core API is meant for machines. This explains why we
see quite a few fields in the exported BuildPlan we didn't cover in this guide.
Day to day we don't need to be concerned with those fields because the Author
API handles them for us.
:::tip
Our intent is to provide an ergonomic way to manage the platform with the Author
API.
:::
When the Author API doesn't offer a path forward, authors may use the Core API
directly from CUE. We can think of the Core API as an escape hatch for the
Author API. We'll see some examples of this in action in the more advanced
guides.
## Review
Let's review the concepts we've covered in this guide:
- A Holos platform is comprised of the CUE files that define the platform specification within the `platform` directory.
- The files in the `platform` directory each model an individual Holos component, and provide the path to the directory where the component's CUE configuration resides.
- A Holos platform must be rendered to generate Kubernetes manifest files.
- Holos resources enable teams to work autonomously while still allowing for centralized enforcement of company policies.
- Changes to one component can impact other components, and we can use `holos render platform` with `git diff` to assess the impact.
## Next Steps
Thank you for finishing the Quickstart guide. Dive deeper with the next guide
on how to [Deploy a Service] which explains how to take one of your existing
Helm charts or Deployments and manage it with Holos.
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[schema]: /docs/api/schema/v1alpha3/
[core]: /docs/api/core/v1alpha3/
[Deploy a Service]: /docs/guides/deploy-a-service/
[Manage a Project]: /docs/guides/manage-a-project/
[rendered manifests pattern]: https://akuity.io/blog/the-rendered-manifests-pattern/
[^1]: [The Basics of CUE](https://cuelang.org/docs/tour/basics/json-superset/)

15
doc/md/topics.mdx Normal file
View File

@@ -0,0 +1,15 @@
---
slug: /topics
title: Topics
description: Stand alone topics that often come up when using Holos.
---
import DocCardList from '@theme/DocCardList';
# Topics
This section has self-contained articles related to various topics that come up
when writing platform configuration code with Holos.
---
<DocCardList />

Some files were not shown because too many files have changed in this diff Show More