Compare commits

..

21 Commits

Author SHA1 Message Date
Jeff McCune
c9d892eee3 generate: consolidate holos generate component cue/helm
Previously helm and cue components were split into two different
subcommands off the holos generate component command.  This is
unnecessary, I'm not sure why it was there in the first place.  The code
seemed perfectly duplicated.

This patch combines them to focus on the concept of a Component.  It
doesn't matter what kind it is now that it's expected to be run from the
root of the platform repository and drop configuration at the root and
the leaf of the tree.
2024-09-11 11:12:53 -07:00
Jeff McCune
4c77eba72b website: automatically generate sidebars
Previously, each document needed to be manually included in the sidebars
to show up.  In addition, index paths like /docs/ and /docs/guides/ were
not found.

This patch addresses both problems by switching sidebars to
automatically generate from filesystem directories.  Important documents
like the getting started guide and introduction are expected to add a
`slug: /foo` front matter item to create a permalink.

The result is the sidebar reflects the filesystem while the URL bar is
more of a permalink.  Files should be able to be moved around the file
system and the sidebar tree without affecting their URL.

This patch also consolidates the API and Docs sidebars into one.
2024-09-11 10:24:01 -07:00
Jeff McCune
a8ae56b08b website: remove quickstart and localhost index
No need to have these pages in sub-folders.  If we need to add images or
resources we can simply create a quickstart folder and add them there.
2024-09-11 06:50:58 -07:00
Jeff McCune
b04837ede2 website: add a localhost guide to get a k3d cluster (#234)
Our guides should be useful reading them only from a mobile device.  For
those readers who also want to apply the manifests to a real cluster we
need a companion guide that describes how to get one.

This patch adds that guide, adapted from the old try holos locally page.
2024-09-10 15:28:46 -07:00
Jeff McCune
559c8bc79f quickstart: remove side by side comparisons
Accidentally left over from cleaning up typos and grammar.
2024-09-10 14:31:29 -07:00
Jeff McCune
a30335b171 concepts: add fleet and cluster 2024-09-10 14:12:23 -07:00
Jeff McCune
108831747a quickstart: fix broken link 2024-09-10 13:40:19 -07:00
Jeff McCune
c714a2b61e quickstart: top to bottom edit for grammar, typos, and voice 2024-09-10 12:51:43 -07:00
Jeff McCune
1cba383dc1 quickstart: incorporate feedback from review
This patch incorporates the main feedback from Gary and Nate from this
morning.  The note tab in argocd.cue was awkware to Gary and I.  The use
of _ in CUE needs an explicit comment which this patch adds.
2024-09-10 11:14:59 -07:00
Jeff McCune
265d5773b8 quickstart: add day 2 chart upgrade example
This patch focuses on the Day 2 benefits holos offers, specifically
making it easier to visiualize exactly what will change when upgrading
components.

In addition, it's easier to apply changes slowly and deliberately since
they're all just flat files in the local filesystem and Git repository.
2024-09-09 20:31:56 -07:00
Jeff McCune
44f8779136 quickstart: render a platform with workload clusters
Previously the quickstart didn't cover adding workload clusters and
rendering a platform with multiple clusters.  This patch demonstrates
how it's effectively a one line change to clone the configuration of a
workload cluster to another geographic region.
2024-09-09 19:43:32 -07:00
Jeff McCune
4127804092 quickstart: v0.93.2 with schema.#Platform
Make sure go install works from the quickstart documentation by doing a
release.  Otherwise, v0.93.1 is installed which doesn't include the
platform schema.
2024-09-09 17:04:32 -07:00
Jeff McCune
8f424cfabe quickstart: sync docs to this commit
Sync the documentation to the current output of the code at this commit.
2024-09-09 17:02:53 -07:00
Jeff McCune
699148abdd quickstart: define a convenince schema for the Platform
Previously, the quickstart step of generating the pod info component and
generating the platform as a whole left the task of integrating the
Component into the Platform as an exercise for the reader.  This is a
problem because it creates unnecessary friction.

This patch addresses the problem by lifting up the Platform concept
into the user-facing Schema API.  The generated platform includes a top
level #Platform definition which exposes the core Platform specification
on the Output field.

The Platform CUE instance then reduces to a simple `#Platform.Output`
which provides the Platform spec to holos for rendering each component
for each cluster.

The CUE code for the schema.#Platform iterates over each
Component to derive the list of components to manage for the Platform.

The CUE code for the generated quickstart platform links the definition
of StandardFleets, which is a Workload fleet and a Management cluster
fleet to the Platform conveninece wrapper.

Finally, the generated podinfo component drops a CUE file at the
repository root to automatically add the component to every workload
cluster.

The result is the only task left for the end user is to define at least
one workload cluster.  Once defined, the component is automatically
managed because it is managed on all workload clusters.

This approach futher opens the door to allow generated components to
define their namespaces and generated secrets on the management cluster
separate from their workloads on the workload clusters.

This patch includes a behavior change, from now on all generated
components should assume they are writing to the root of the user's Git
repository so that they can generate files through the whole tree.

In the future, we should template output paths for generated components.
A simple approach might be to embed a file with a .target suffix, with
the contents being a simple Go template of the file path to write to.
The holos generate subcommand can then check if any given embedded file
foo has a foo.target companion, then write the target to the rendered
template value.
2024-09-09 16:05:00 -07:00
Jeff McCune
73f777759e quickstart: mix-in argocd application resource
Users need to customize the default behavior of the core components,
like the Helm schema wrapper to mix-in an ArgoCD Application resource to
each component.  This patch wires up #Helm in the holos package to
schema.#Helm from the v1alpha3 api.

The result is illustrated in the Quickstart documentation, it is now
simple for users to modify the definition of a Helm component such that
Application resources are mixed in to every component in the platform.
2024-09-09 14:09:24 -07:00
Jeff McCune
8b9070f185 api: add schema to platform cue.mod for consistency
Previosly the end user needed to write, or at least copy and paste, a
large amount of boiler plate code to achieve the goal of declaring a
helm chart component.  There is a gap between the cue code:

    (#Helm & Chart).Output

And the full BuildPlan produced for the Holos cli to execute the
rendering process.  The boiler plate code in schema.cue at the root of
the platform infrastructure repository was largely responsible for
defining how a BuildPlan with one HelmChart component is derived from
this #Helm definition.

This patch moves the definitions into a new, documented API named
`schema`.  End users are expected to define their own #Helm definition
using the schema.#Helm, like so in the root level schema.cue:

    #Helm: schema.#Helm
2024-09-09 11:22:36 -07:00
Jeff McCune
1e8861c8b7 builder: relax api version requirement to fix deploy-dev
Without this patch deployments to the dev environment are failing with
the following error when commits are pushed to the main branch.

    GIT_DETAIL=v0.93.0-3-g4db3fb4 GIT_SUFFIX= bash ./hack/deploy-dev
    Cloning into 'holos-infra'...
    could not validate
    could not run: could not validate invalid BuildPlan: apiVersion invalid: want: v1alpha3 have: v1alpha2 at internal/builder/builder.go:308
    could not run: could not render component: exit status 1 at internal/render/platform.go:48
    make: *** [Makefile:147: dev-deploy] Error 1

This patch removes the api version check in the build plan validation
function.  In the future, we should pass an interface internally in the
holos executable.

The result is holos render platform ./platform succeeds with this patch
applied.
2024-09-06 20:58:56 -07:00
Jeff McCune
bdc182f4eb quickstart: generate podinfo helm chart 2024-09-06 20:57:35 -07:00
Jeff McCune
4db3fb4ead api: optional platform.spec.model
Previously the CUE code needed to specify the Platform.spec.model field,
which created friction.  This patch adds a cue struct tag to unify the
field with an open struct.

    ❯ holos render platform ./platform --log-level=debug
    could not run: could not marshal cue instance platform: cue: marshal error: spec.model: cannot convert incomplete value "_" to JSON at internal/builder/platform.go:45
    spec.model: cannot convert incomplete value "_" to JSON

The render command completes successfully with this patch without the
user having to provide a value for the spec.model field.
2024-09-06 13:38:48 -07:00
Jeff McCune
1911c7fe01 generate: add bare bones quickstart platform
This patch adds the minimal amount of CUE code necessary to successfully
run the following two commands from the quickstart.

    holos generate platform quickstart
    holos render platform ./platform

The result is no componets are rendered, so nothing is done, but it does
succeeed.

This patch surfaces some friction and inconsistency with how the Model
is passed in and the initial structure of the _PlatformConfig.  The tags
are required otherwise holos errors out.
2024-09-06 12:16:59 -07:00
Jeff McCune
5e582ec5c6 generate: do not require registration when generating a platform
Without this patch the `holos generate platform` command automatically
makes an rpc call to holos server.  This creates friction for the
quickstart guide because we don't need to require users to register and
have an organization and platform already created in the server just to
generate a simple platform to exercise a simple helm chart component.

A future patch should implement the behavior of linking a server side
platform to a local git repository by making the API call to get the
platform ID then updating the platform.metadata.json file.
2024-09-06 11:27:05 -07:00
70 changed files with 2075 additions and 252 deletions

View File

@@ -13,6 +13,7 @@
"authroutes",
"buildplan",
"cainjector",
"CAROOT",
"clsx",
"clusterissuer",
"cookiesecret",
@@ -30,6 +31,7 @@
"errgroup",
"fieldmaskpb",
"flushcache",
"gendoc",
"ghaction",
"gitops",
"godoc",
@@ -56,6 +58,7 @@
"libnss",
"loadbalancer",
"mattn",
"mindmap",
"mktemp",
"Multicluster",
"mxcl",
@@ -66,8 +69,10 @@
"otelconnect",
"Parentspanid",
"pflag",
"pipefail",
"PKCE",
"platformconnect",
"podinfo",
"promhttp",
"protobuf",
"protojson",
@@ -81,6 +86,7 @@
"spanid",
"spiffe",
"startupapicheck",
"stefanprodan",
"structpb",
"systemconnect",
"tablewriter",

View File

@@ -2,11 +2,6 @@ package v1alpha3
import "google.golang.org/protobuf/types/known/structpb"
type PlatformMetadata struct {
// Name represents the Platform name.
Name string `json:"name"`
}
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
@@ -24,13 +19,18 @@ type Platform struct {
Spec PlatformSpec `json:"spec"`
}
type PlatformMetadata struct {
// Name represents the Platform name.
Name string `json:"name"`
}
// PlatformSpec represents the specification of a Platform. Think of a platform
// specification as a list of platform components to apply to a list of
// kubernetes clusters combined with the user-specified Platform Model.
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `json:"model"`
Model structpb.Struct `json:"model" cue:"{...}"`
// Components represents a list of holos components to manage.
Components []PlatformSpecComponent `json:"components"`
}

View File

@@ -0,0 +1,150 @@
// Package v1alpha3 contains CUE definitions intended as convenience wrappers
// around the core data types defined in package core. The purpose of these
// wrappers is to make life easier for platform engineers by reducing boiler
// plate code and generating component build plans in a consistent manner.
package v1alpha3
import (
core "github.com/holos-run/holos/api/core/v1alpha3"
"google.golang.org/protobuf/types/known/structpb"
)
//go:generate ../../../hack/gendoc
// Helm provides a BuildPlan via the Output field which contains one HelmChart
// from package core. Useful as a convenience wrapper to render a HelmChart
// with optional mix-in resources and Kustomization post-processing.
type Helm struct {
// Name represents the chart name.
Name string
// Version represents the chart version.
Version string
// Namespace represents the helm namespace option when rendering the chart.
Namespace string
// Resources are kubernetes api objects to mix into the output.
Resources map[string]any `cue:"{...}"`
// Repo represents the chart repository
Repo struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Values represents data to marshal into a values.yaml for helm.
Values interface{} `cue:"{...}"`
// Chart represents the derived HelmChart for inclusion in the BuildPlan
// Output field value. The default HelmChart field values are derived from
// other Helm field values and should be sufficient for most use cases.
Chart core.HelmChart
// EnableKustomizePostProcessor processes helm output with kustomize if true.
EnableKustomizePostProcessor bool `cue:"true | *false"`
// KustomizeFiles represents additional files to include in a Kustomization
// resources list. Useful to patch helm output. The implementation is a
// struct with filename keys and structs as values. Holos encodes the struct
// value to yaml then writes the result to the filename key. Component
// authors may then reference the filename in the kustomization.yaml resources
// or patches lists.
// Requires EnableKustomizePostProcessor: true.
KustomizeFiles map[string]any `cue:"{[string]: {...}}"`
// KustomizePatches represents patches to apply to the helm output. Requires
// EnableKustomizePostProcessor: true.
KustomizePatches map[core.InternalLabel]any `cue:"{[string]: {...}}"`
// KustomizeResources represents additional resources files to include in the
// kustomize resources list.
KustomizeResources map[string]any `cue:"{[string]: {...}}"`
// ArgoConfig represents the ArgoCD GitOps configuration for this Component.
ArgoConfig ArgoConfig
// Output represents the derived BuildPlan for the Holos cli to render.
Output core.BuildPlan
}
// Resources represents the default schema for a Kubernetes API object resource.
// For example, a Service, Namespace or Deployment. The top level key is the
// kind of resource so default behavior and strict schema enforcement may be
// enforced for the kind. The second level keys are an arbitrary internal
// label, which serves as the default value for the resource metadata name
// field, but may differ for situations where the same resource kind and name
// are managed in different namespaces.
//
// Refer to [definitions.cue] for the CUE schema definition as an example to
// build on when defining your own Components.
//
// [definitions.cue]: https://github.com/holos-run/holos/blob/main/internal/generate/platforms/cue.mod/pkg/github.com/holos-run/holos/api/schema/v1alpha3/definitions.cue#L9
// type Resources map[string]map[string]any
// ArgoConfig represents the ArgoCD GitOps configuration for a Component.
// Useful to define once at the root of the Platform configuration and reuse
// across all Components.
type ArgoConfig struct {
// Enabled causes holos to render an ArgoCD Application resource for GitOps if true.
Enabled bool `cue:"true | *false"`
// ClusterName represents the cluster within the platform the Application
// resource is intended for.
ClusterName string
// DeployRoot represents the path from the git repository root to the `deploy`
// rendering output directory. Used as a prefix for the
// Application.spec.source.path field.
DeployRoot string `cue:"string | *\".\""`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
}
// Cluster represents a cluster managed by the Platform.
type Cluster struct {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
Name string `json:"name"`
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
Primary bool `json:"primary" cue:"true | *false"`
}
// Fleet represents a named collection of similarly configured Clusters. Useful
// to segregate workload clusters from their management cluster.
type Fleet struct {
Name string `json:"name"`
// Clusters represents a mapping of Clusters by their name.
Clusters map[string]Cluster `json:"clusters" cue:"{[Name=_]: name: Name}"`
}
// StandardFleets represents the standard set of Clusters in a Platform
// segmented into Fleets by their purpose. The management Fleet contains a
// single Cluster, for example a GKE autopilot cluster with no workloads
// deployed for reliability and cost efficiency. The workload Fleet contains
// all other Clusters which contain workloads and sync Secrets from the
// management cluster.
type StandardFleets struct {
// Workload represents a Fleet of zero or more workload Clusters.
Workload Fleet `json:"workload" cue:"{name: \"workload\"}"`
// Management represents a Fleet with one Cluster named management.
Management Fleet `json:"management" cue:"{name: \"management\", clusters: management: _}"`
}
// Platform is a convenience structure to produce a core Platform specification
// value in the Output field. Useful to collect components at the root of the
// Platform configuration tree as a struct, which are automatically converted
// into a list for the core Platform spec output.
type Platform struct {
// Name represents the Platform name.
Name string `cue:"string | *\"holos\""`
// Components is a structured map of components to manage by their name.
Components map[string]core.PlatformSpecComponent
// Model represents the Platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `cue:"{...}"`
// Output represents the core Platform spec for the holos cli to iterate over
// and render each listed Component, injecting the Model.
Output core.Platform
}

5
doc/md/api.md Normal file
View File

@@ -0,0 +1,5 @@
import DocCardList from '@theme/DocCardList';
# API Reference
<DocCardList />

5
doc/md/api/core.md Normal file
View File

@@ -0,0 +1,5 @@
import DocCardList from '@theme/DocCardList';
# Core API
<DocCardList />

View File

@@ -1,3 +0,0 @@
# Core API
- [v1alpha2](v1alpha2)

View File

@@ -368,7 +368,7 @@ PlatformSpec represents the specification of a Platform. Think of a platform spe
type PlatformSpec struct {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `json:"model"`
Model structpb.Struct `json:"model" cue:"{...}"`
// Components represents a list of holos components to manage.
Components []PlatformSpecComponent `json:"components"`
}

5
doc/md/api/schema.md Normal file
View File

@@ -0,0 +1,5 @@
import DocCardList from '@theme/DocCardList';
# Schema API
<DocCardList />

View File

@@ -0,0 +1,168 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# v1alpha3
```go
import "github.com/holos-run/holos/api/schema/v1alpha3"
```
Package v1alpha3 contains CUE definitions intended as convenience wrappers around the core data types defined in package core. The purpose of these wrappers is to make life easier for platform engineers by reducing boiler plate code and generating component build plans in a consistent manner.
## Index
- [type ArgoConfig](<#ArgoConfig>)
- [type Cluster](<#Cluster>)
- [type Fleet](<#Fleet>)
- [type Helm](<#Helm>)
- [type Platform](<#Platform>)
- [type StandardFleets](<#StandardFleets>)
<a name="ArgoConfig"></a>
## type ArgoConfig {#ArgoConfig}
ArgoConfig represents the ArgoCD GitOps configuration for a Component. Useful to define once at the root of the Platform configuration and reuse across all Components.
```go
type ArgoConfig struct {
// Enabled causes holos to render an ArgoCD Application resource for GitOps if true.
Enabled bool `cue:"true | *false"`
// ClusterName represents the cluster within the platform the Application
// resource is intended for.
ClusterName string
// DeployRoot represents the path from the git repository root to the `deploy`
// rendering output directory. Used as a prefix for the
// Application.spec.source.path field.
DeployRoot string `cue:"string | *\".\""`
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL string
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision string `cue:"string | *\"main\""`
}
```
<a name="Cluster"></a>
## type Cluster {#Cluster}
Cluster represents a cluster managed by the Platform.
```go
type Cluster struct {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
Name string `json:"name"`
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
Primary bool `json:"primary" cue:"true | *false"`
}
```
<a name="Fleet"></a>
## type Fleet {#Fleet}
Fleet represents a named collection of similarly configured Clusters. Useful to segregate workload clusters from their management cluster.
```go
type Fleet struct {
Name string `json:"name"`
// Clusters represents a mapping of Clusters by their name.
Clusters map[string]Cluster `json:"clusters" cue:"{[Name=_]: name: Name}"`
}
```
<a name="Helm"></a>
## type Helm {#Helm}
Helm provides a BuildPlan via the Output field which contains one HelmChart from package core. Useful as a convenience wrapper to render a HelmChart with optional mix\-in resources and Kustomization post\-processing.
```go
type Helm struct {
// Name represents the chart name.
Name string
// Version represents the chart version.
Version string
// Namespace represents the helm namespace option when rendering the chart.
Namespace string
// Resources are kubernetes api objects to mix into the output.
Resources map[string]any `cue:"{...}"`
// Repo represents the chart repository
Repo struct {
Name string `json:"name"`
URL string `json:"url"`
}
// Values represents data to marshal into a values.yaml for helm.
Values interface{} `cue:"{...}"`
// Chart represents the derived HelmChart for inclusion in the BuildPlan
// Output field value. The default HelmChart field values are derived from
// other Helm field values and should be sufficient for most use cases.
Chart core.HelmChart
// EnableKustomizePostProcessor processes helm output with kustomize if true.
EnableKustomizePostProcessor bool `cue:"true | *false"`
// KustomizeFiles represents additional files to include in a Kustomization
// resources list. Useful to patch helm output. The implementation is a
// struct with filename keys and structs as values. Holos encodes the struct
// value to yaml then writes the result to the filename key. Component
// authors may then reference the filename in the kustomization.yaml resources
// or patches lists.
// Requires EnableKustomizePostProcessor: true.
KustomizeFiles map[string]any `cue:"{[string]: {...}}"`
// KustomizePatches represents patches to apply to the helm output. Requires
// EnableKustomizePostProcessor: true.
KustomizePatches map[core.InternalLabel]any `cue:"{[string]: {...}}"`
// KustomizeResources represents additional resources files to include in the
// kustomize resources list.
KustomizeResources map[string]any `cue:"{[string]: {...}}"`
// ArgoConfig represents the ArgoCD GitOps configuration for this Component.
ArgoConfig ArgoConfig
// Output represents the derived BuildPlan for the Holos cli to render.
Output core.BuildPlan
}
```
<a name="Platform"></a>
## type Platform {#Platform}
Platform is a convenience structure to produce a core Platform specification value in the Output field. Useful to collect components at the root of the Platform configuration tree as a struct, which are automatically converted into a list for the core Platform spec output.
```go
type Platform struct {
// Name represents the Platform name.
Name string `cue:"string | *\"holos\""`
// Components is a structured map of components to manage by their name.
Components map[string]core.PlatformSpecComponent
// Model represents the Platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model structpb.Struct `cue:"{...}"`
// Output represents the core Platform spec for the holos cli to iterate over
// and render each listed Component, injecting the Model.
Output core.Platform
}
```
<a name="StandardFleets"></a>
## type StandardFleets {#StandardFleets}
StandardFleets represents the standard set of Clusters in a Platform segmented into Fleets by their purpose. The management Fleet contains a single Cluster, for example a GKE autopilot cluster with no workloads deployed for reliability and cost efficiency. The workload Fleet contains all other Clusters which contain workloads and sync Secrets from the management cluster.
```go
type StandardFleets struct {
// Workload represents a Fleet of zero or more workload Clusters.
Workload Fleet `json:"workload" cue:"{name: \"workload\"}"`
// Management represents a Fleet with one Cluster named management.
Management Fleet `json:"management" cue:"{name: \"management\", clusters: management: _}"`
}
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

Before

Width:  |  Height:  |  Size: 934 KiB

After

Width:  |  Height:  |  Size: 934 KiB

View File

Before

Width:  |  Height:  |  Size: 703 KiB

After

Width:  |  Height:  |  Size: 703 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 728 KiB

After

Width:  |  Height:  |  Size: 728 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 1014 KiB

After

Width:  |  Height:  |  Size: 1014 KiB

View File

Before

Width:  |  Height:  |  Size: 854 KiB

After

Width:  |  Height:  |  Size: 854 KiB

View File

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

Before

Width:  |  Height:  |  Size: 624 KiB

After

Width:  |  Height:  |  Size: 624 KiB

View File

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 116 KiB

View File

@@ -671,10 +671,10 @@ foundation of a software development platform that:
Dive deeper with the following resources that build on the foundation you have now.
1. Explore the [Rendering Process](/docs/design/rendering) in Holos.
1. Explore the [Rendering Process](/docs/concepts#rendering) in Holos.
2. Dive deeper into the [Platform Manifests](./platform-manifests) rendered in this guide.
3. Deploy [ArgoCD](/docs/guides/argocd) onto the foundation you built.
4. Deploy [Backstage](/docs/guides/backstage) as a portal to the integrated platform components.
3. Deploy [ArgoCD](../argocd) onto the foundation you built.
4. Deploy [Backstage](../backstage) as a portal to the integrated platform components.
## Clean-Up

View File

@@ -2,7 +2,7 @@
This document provides an example of how Holos uses CUE and Helm to unify and
render the platform configuration. It refers to the manifests rendered in the
[Try Holos Locally](/docs/guides/try-holos/) guide.
Try Holos Locally guide.
Take a moment to review the manifests `holos` rendered to build the platform.

View File

@@ -2,7 +2,7 @@
This document captures notes on locally developing Holos.
Follow the steps in [Try Holos Locally](/docs/guides/try-holos), but take care
Follow the steps in [Try Holos Locally](../guides/try-holos), but take care
to select `Develop` tabs when creating the k3d cluster so you have a local
registry to push to.

View File

@@ -1,47 +0,0 @@
# Rendering
:::tip
This document provides a brief overview of the rendering process, a core design
element in Holos.
:::
Holos uses the Kubernetes resource model to manage configuration. The `holos`
command line interface is the primary method you'll use to manage your platform.
Holos uses CUE to provide a unified configuration model of the platform. This
unified configuration is built up from components packaged with Helm, Kustomize,
CUE, or any other tool that can produce Kubernetes resource manifests as output.
This process can be thought of as a data **rendering pipeline**. The key
concept is that `holos` will always produce fully rendered output, but delegates
the _application_ of the configuration to other tools like `kubectl apply`,
ArgoCD, or Flux.
```mermaid
---
title: Figure 2 - Render Pipeline
---
graph LR
PS[<a href="/docs/api/core/v1alpha2#PlatformSpec">PlatformSpec</a>]
BP[<a href="/docs/api/core/v1alpha2#BuildPlan">BuildPlan</a>]
HC[<a href="/docs/api/core/v1alpha2#HolosComponent">HolosComponent</a>]
H[<a href="/docs/api/core/v1alpha2#HelmChart">HelmChart</a>]
K[<a href="/docs/api/core/v1alpha2#KustomizeBuild">KustomizeBuild</a>]
O[<a href="/docs/api/core/v1alpha2#KubernetesObjects">KubernetesObjects</a>]
P[<a href="/docs/api/core/v1alpha2#Kustomize">Kustomize</a>]
Y[Kubernetes <br>Resources]
G[GitOps <br>Resource]
C[Kube API Server]
PS --> BP --> HC
HC --> H --> P
HC --> K --> P
HC --> O --> P
P --> Y --> C
P --> G --> C
```

5
doc/md/guides.md Normal file
View File

@@ -0,0 +1,5 @@
import DocCardList from '@theme/DocCardList';
# Guides
<DocCardList />

View File

@@ -0,0 +1,95 @@
---
description: Use Holos to expose a Service with the Gateway API.
slug: /guides/expose-a-service
sidebar_position: 300
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Expose a Service
In this guide, you'll learn how to expose a service with Holos using the Gateway
API.
:::warning TODO
Complete this section once the steps are complete.
:::
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
## What you'll need {#requirements}
:::warning TODO
Complete this section once the steps are complete.
:::
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap upstream Helm charts.
Optionally, if you'd like to apply the rendered manifests to a real Cluster,
first complete the [localhost Guide](../local-cluster).
## Create a Git Repository
Start by initializing an empty Git repository. Holos operates on local files
stored in a Git repository.
<Tabs groupId="init">
<TabItem value="command" label="Command">
```bash
mkdir expose-a-service
cd expose-a-service
git init
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Initialized empty Git repository in /expose-a-service/.git/
```
</TabItem>
</Tabs>
This guide assumes you will run commands from the root directory of the Git
repository unless stated otherwise.
## Generate the Platform {#Generate-Platform}
Start by generating a platform used as the basis for our guides.
```bash
holos generate platform guide
```
Commit the generated platform config to the repository.
<Tabs groupId="commit-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate platform guide - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main (root-commit) 0b17b7f] holos generate platform guide - 0.93.3
213 files changed, 72349 insertions(+)
...
```
</TabItem>
</Tabs>
## Manage httpbin {#manage-httpbin}
The platform you generated is currently empty. Run the following command to
generate a Holos Component for the
[httpbin](https://github.com/mccutchen/go-httpbin) service.
httpbin is a simple backend service useful for end-to-end testing. In this
guide, we use httpbin as a example of a service your organization develops and
deploy onto your Platform.

View File

@@ -0,0 +1,196 @@
---
description: Build a local Cluster to use with these guides.
slug: /guides/local-cluster
sidebar_position: 200
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Local Cluster
In this guide you'll set up a Cluster on your local host to apply and explore
the configuration described in our other guides. After completing this guide
you'll have a standard Kubernetes API server with proper DNS and TLS
certificates. You'll be able to easily reset the cluster to a known good state
to iterate on your own Platform.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a k8s api server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
5. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to interact with the k8s api server.
6. [mkcert](https://github.com/FiloSottile/mkcert?tab=readme-ov-file#installation) - to make trusted TLS certificates.
7. [jq](https://jqlang.github.io/jq/download/) - to fiddle with JSON output.
## Configure DNS {#configure-dns}
Configure your machine to resolve `*.holos.localhost` to your loopback
interface. This is necessary for requests to reach the workload cluster. Save
this script to a file and execute it.
```bash showLineNumbers
#! /bin/bash
#
set -euo pipefail
tmpdir="$(mktemp -d)"
finish() {
[[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"
brew install dnsmasq
cat <<EOF >"$(brew --prefix)/etc/dnsmasq.d/holos.localhost.conf"
# Refer to https://holos.run/docs/tutorial/local/k3d/
address=/holos.localhost/127.0.0.1
EOF
if [[ -r /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist ]]; then
echo "dnsmasq already configured"
else
sudo cp "$(brew list dnsmasq | grep 'dnsmasq.plist$')" \
/Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo launchctl unload /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
dscacheutil -flushcache
echo "dnsmasq configured"
fi
sudo mkdir -p /etc/resolver
sudo tee /etc/resolver/holos.localhost <<EOF
domain holos.localhost
nameserver 127.0.0.1
EOF
sudo killall -HUP mDNSResponder
echo "all done."
```
## Create the Cluster {#create-the-cluster}
The Workload Cluster is where your applications and services will be deployed.
In production this is usually an EKS, GKE, or AKS cluster.
:::tip
Holos supports all compliant Kubernetes clusters. Holos was developed and tested
on GKE, EKS, Talos, k3s, and Kubeadm clusters.
:::
Create a local registry to speed up image builds and pulls.
```bash
k3d registry create registry.holos.localhost --port 5100
```
Create the workload cluster configured to use the local registry.
```bash
k3d cluster create workload \
--registry-use k3d-registry.holos.localhost:5100 \
--port "443:443@loadbalancer" \
--k3s-arg "--disable=traefik@server:0"
```
Traefik is disabled because Istio provides the same functionality.
## Setup Root CA {#setup-root-ca}
Platforms most often use cert-manager to issue tls certificates. The browser
and tools we're using need to trust these certificates to work together.
Generate a local, trusted root certificate authority with the following script.
Admin access is necessary for `mkcert` to manage the certificate into your trust
stores.
```bash
sudo -v
```
Manage the local CA and copy the CA key to the workload cluster so that cert
manager can manage trusted certificates.
Save this script to a file and execute it to configure a trusted certificate
authority.
```bash showLineNumbers
#! /bin/bash
#
set -euo pipefail
mkcert --install
tmpdir="$(mktemp -d)"
finish() {
[[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"
# Create the local CA Secret with ca.crt, tls.crt, tls.key
mkdir local-ca
cd local-ca
CAROOT="$(mkcert -CAROOT)"
cp -p "${CAROOT}/rootCA.pem" ca.crt
cp -p "${CAROOT}/rootCA.pem" tls.crt
cp -p "${CAROOT}/rootCA-key.pem" tls.key
kubectl create secret generic --from-file=. --dry-run=client -o yaml local-ca > ../local-ca.yaml
cd ..
echo 'type: kubernetes.io/tls' >> local-ca.yaml
kubectl apply --server-side=true -f- <<EOF
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: cert-manager
name: cert-manager
spec:
finalizers:
- kubernetes
EOF
kubectl apply -n cert-manager --server-side=true -f local-ca.yaml
```
:::warning
Take care to run the local-ca script each time you create the workload cluster
so that Certificates are issued correctly.
:::
## Clean Up {#clean-up}
If you'd like to clean up the resources you created in this guide, remove them
with:
```bash
k3d cluster delete workload
```
## Reset {#reset}
If you'd like to reset to a known good state, execute the [Clean Up](#clean-up)
section, then [Create the Cluster](#create-the-cluster) and the [Setup Root
CA](#setup-root-ca) tasks.
## Next Steps
Now that you have a real cluster, apply and explore the manifests Holos renders
in the [Quickstart](/docs/quickstart) guide.

View File

@@ -0,0 +1,724 @@
---
description: Try Holos with this quick start guide.
slug: /quickstart
sidebar_position: 100
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Quickstart
In this guide, you'll experience how Holos makes the process of operating a
Platform safer, easier, and more consistent. We'll use Holos to manage a
vendor-provided Helm chart as a Component. Next, we'll mix in our own custom
resources to manage the Component with GitOps. Finally, you'll see how Holos
makes it safer and easier to maintain software over time by surfacing the exact
changes that will be applied when upgrading the vendor's chart to a new version,
before they are actually made.
The [Concepts](/docs/concepts) page defines capitalized terms such as Platform
and Component.
## What you'll need {#requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the Platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos Components that
wrap upstream Helm charts.
Optionally, if you'd like to apply the rendered manifests to a real Cluster,
first complete the [Local Cluster Guide](/docs/guides/local-cluster).
## Install Holos
Install Holos with the following command or other methods listed on the
[Installation](/docs/install/) page.
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
## Create a Git Repository
Start by initializing an empty Git repository. Holos operates on local files
stored in a Git repository.
<Tabs groupId="init">
<TabItem value="command" label="Command">
```bash
mkdir holos-quickstart
cd holos-quickstart
git init
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Initialized empty Git repository in /holos-quickstart/.git/
```
</TabItem>
</Tabs>
This guide assumes you will run commands from the root directory of the Git
repository unless stated otherwise.
## Generate the Platform {#Generate-Platform}
Generate the Platform code in the repository root. A Platform refers to the
entire set of software holistically integrated to provide a software development
platform for your organization. In this guide, the Platform will include a
single Component to demonstrate how the concepts fit together.
```bash
holos generate platform quickstart
```
Commit the generated platform config to the repository.
<Tabs groupId="commit-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate platform quickstart - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main (root-commit) 0b17b7f] holos generate platform quickstart
213 files changed, 72349 insertions(+)
...
```
</TabItem>
</Tabs>
## Generate a Component {#generate-component}
The platform you generated is currently empty. Run the following command to
generate the CUE code that defines a Helm Component.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
holos generate component podinfo --component-version 6.6.1
```
</TabItem>
<TabItem value="output" label="Output">
```txt
generated component
```
</TabItem>
</Tabs>
The --component-version 6.6.1 flag intentionally installs an older release.
You'll see how Holos assists with software upgrades later in this guide.
The generate component command creates two files: a leaf file,
`components/podinfo/podinfo.gen.cue`, and a root file, `podinfo.gen.cue`. Holos
leverages the fact that [order is
irrelevant](https://cuelang.org/docs/tour/basics/order-irrelevance/) in CUE to
register the component with the Platform by adding a file to the root of the Git
repository. The second file defines the component in the leaf component
directory.
<Tabs groupId="podinfo-files">
<TabItem value="components/podinfo/podinfo.gen.cue" label="Leaf">
`components/podinfo/podinfo.gen.cue`
```cue showLineNumbers
package holos
// Produce a helm chart build plan.
(#Helm & Chart).Output
let Chart = {
Name: "podinfo"
Version: "6.6.1"
Namespace: "default"
Repo: name: "podinfo"
Repo: url: "https://stefanprodan.github.io/podinfo"
Values: {}
}
```
</TabItem>
<TabItem value="podinfo.gen.cue" label="Root">
`podinfo.gen.cue`
```cue showLineNumbers
package holos
// Manage podinfo on workload clusters only
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name)/podinfo": {
path: "components/podinfo"
cluster: Cluster.name
}
}
```
</TabItem>
</Tabs>
In this example, we provide the minimal information needed to manage the Helm
chart: the name, version, Kubernetes namespace for deployment, and the chart
repository location.
This chart deploys cleanly without any values provided, but we include an empty
Values struct to show how Holos improves consistency and safety in Helm by
leveraging the strong type-checking in CUE. You can safely pass shared values,
such as the organizations domain name, to all Components across all clusters in
the Platform by defining them at the root of the configuration.
Commit the generated component config to the repository.
<Tabs groupId="commit-component">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos generate component podinfo - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main cc0e90c] holos generate component podinfo
2 files changed, 24 insertions(+)
create mode 100644 components/podinfo/podinfo.gen.cue
create mode 100644 podinfo.gen.cue
```
</TabItem>
</Tabs>
## Render the Component
You can render individual components without adding them to a Platform, which is
helpful when developing a new component.
<Tabs groupId="render-podinfo">
<TabItem value="command" label="Command">
```bash
holos render component ./components/podinfo --cluster-name=default
```
</TabItem>
<TabItem value="output" label="Output">
```txt
cached
rendered podinfo
```
</TabItem>
</Tabs>
First, the command caches the Helm chart locally to speed up subsequent
renderings. Then, the command runs Helm to produce the output and writes it into
the deploy directory.
<Tabs groupId="tree-podinfo">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
└── default
└── components
└── podinfo
└── podinfo.gen.yaml
5 directories, 1 file
```
</TabItem>
</Tabs>
The component deploys to one cluster named `default`. In practice, the same
component is often deployed to multiple clusters, such as `east` and `west` to
provide redundancy and increase availability.
:::tip
This example is equivalent to running `helm template` on the chart and saving
the output to a file. Holos simplifies this task, making it safer and more
consistent when managing many charts.
:::
## Mix in an ArgoCD Application
We've seen how Holos works with Helm, but we haven't yet explored how Holos
makes it easier to consistently and safely manage all of the software in a
Platform.
Holos allows you to easily mix in resources that differentiate your Platform.
We'll use this feature to mix in an ArgoCD [Application][application] to manage
the podinfo Component with GitOps. We'll define this configuration in a way that
can be automatically and consistently reused across all future Components added
to the Platform.
Create a new file named `argocd.cue` in the root of your Git repository with the
following contents:
<Tabs groupId="argocd-config">
<TabItem value="command" label="argocd.cue">
```cue showLineNumbers
package holos
#ArgoConfig: {
Enabled: true
RepoURL: "https://example.com/holos-quickstart.git"
}
```
</TabItem>
</Tabs>
:::tip
If you plan to apply the rendered output to a real cluster, change the
`example.com` RepoURL to the URL of the Git repository you created in this
guide. You don't need to change the example if you're just exploring Holos by
inspecting the rendered output without applying it to a live cluster.
:::
With this file in place, render the component again.
<Tabs groupId="render-podinfo-argocd">
<TabItem value="command" label="Command">
```bash
holos render component ./components/podinfo --cluster-name=default
```
</TabItem>
<TabItem value="output" label="Output">
```txt
wrote deploy file
rendered gitops/podinfo
rendered podinfo
```
</TabItem>
</Tabs>
Holos uses the locally cached chart to improve performance and reliability. It
then renders the Helm template output along with an ArgoCD Application resource
for GitOps.
:::tip
By defining the ArgoCD configuration at the root, we again take advantage of the
fact that [order is
irrelevant](https://cuelang.org/docs/tour/basics/order-irrelevance/) in CUE.
:::
Defining the configuration at the root ensures all future leaf Components take
the ArgoCD configuration and render an Application manifest for GitOps
management.
<Tabs groupId="tree-podinfo-argocd">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
└── default
├── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── gitops
└── podinfo.application.gen.yaml
6 directories, 2 files
```
</TabItem>
</Tabs>
Notice the new `podinfo.application.gen.yaml` file created by enabling ArgoCD in
the Helm component. The Application resource in the file looks like this:
<Tabs groupId="podinfo-application">
<TabItem value="file" label="podinfo.application.gen.yaml">
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
path: ./deploy/clusters/default/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
</Tabs>
:::tip
Holos generates a similar Application resource for every additional Component
added to your Platform.
:::
Finally, add and commit the results to your Platform's Git repository.
<Tabs groupId="commit-argo">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos render component ./components/podinfo --cluster-name=default"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main f95cef1] holos render component ./components/podinfo --cluster-name=default
3 files changed, 134 insertions(+)
create mode 100644 argocd.cue
create mode 100644 deploy/clusters/default/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/default/gitops/podinfo.application.gen.yaml
```
</TabItem>
</Tabs>
In this section, we learned how Holos simplifies mixing resources into
Components, like an ArgoCD Application. Holos ensures consistency by managing an
Application resource for every Component added to the Platform through the
configuration you define in `argocd.cue` at the root of the repository.
## Define Workload Clusters {#workload-clusters}
We've generated a Component to manage podinfo and integrated it with our
Platform, but rendering the Platform doesn't render podinfo. Podinfo isn't
rendered because we haven't assigned any Clusters to the workload Fleet.
Define two new clusters, `east` and `west`, and assign them to the workload
Fleet. Create a new file named `clusters.cue` in the root of your Git repository
with the following contents:
<Tabs groupId="clusters">
<TabItem value="clusters.cue" label="clusters.cue">
```cue showLineNumbers
package holos
// Define two workload clusters for disaster recovery.
#Fleets: workload: clusters: {
// In CUE _ indicates values are defined elsewhere.
east: _
west: _
}
```
</TabItem>
</Tabs>
This example shows how Holos simplifies configuring multiple clusters with
similar configuration by grouping them into a Fleet.
:::tip
Fleets help segment a group of Clusters into one leader and multiple followers
by designating one cluster as the primary. Holos makes it safer, easier, and
more consistent to reconfigure which cluster is the primary. The primary can be
set to automatically restore persistent data from backups, while non-primary
clusters can be configured to automatically replicate from the primary.
Automatic database backup, restore, and streaming replication is an advanced
topic enabled by Cloud Native PG and CUE. Check back for a guide on this and
other Day 2 operations topics.
:::
## Render the Platform {#render-platform}
Render the Platform to render the podinfo Component for each of the workload
clusters.
<Tabs groupId="render-platform">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered components/podinfo for cluster west in 99.480792ms
rendered components/podinfo for cluster east in 99.882667ms
```
</TabItem>
</Tabs>
The render platform command iterates over every Cluster in the Fleet and renders
each Component assigned to the Fleet. Notice the two additional subdirectories
created under the deploy directory, one for each cluster: `east` and `west`.
<Tabs groupId="tree-platform">
<TabItem value="command" label="Command">
```bash
tree deploy
```
</TabItem>
<TabItem value="output" label="Output">
```txt
deploy
└── clusters
├── default
│   ├── components
│   │   └── podinfo
│   │   └── podinfo.gen.yaml
│   └── gitops
│   └── podinfo.application.gen.yaml
# highlight-next-line
├── east
│   ├── components
│   │   └── podinfo
│   │   └── podinfo.gen.yaml
│   └── gitops
│   └── podinfo.application.gen.yaml
# highlight-next-line
└── west
├── components
│   └── podinfo
│   └── podinfo.gen.yaml
└── gitops
└── podinfo.application.gen.yaml
14 directories, 6 files
```
</TabItem>
</Tabs>
Holos ensures consistency and safety by defining the ArgoCD Application once,
with strong type checking, at the configuration root.
New Application resources are automatically generated for the `east` and `west`
workload Clusters.
<Tabs groupId="applications">
<TabItem value="east" label="east">
`deploy/clusters/east/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/east/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
<TabItem value="west" label="west">
`deploy/clusters/west/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/west/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
<TabItem value="default" label="default">
`deploy/clusters/default/gitops/podinfo.application.gen.yaml`
```yaml showLineNumbers
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: podinfo
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
project: default
source:
# highlight-next-line
path: ./deploy/clusters/default/components/podinfo
repoURL: https://example.com/holos-quickstart.git
targetRevision: main
```
</TabItem>
</Tabs>
Add and commit the rendered Platform and workload Clusters.
<Tabs groupId="commit-render-platform">
<TabItem value="command" label="Command">
```bash
git add .
git commit -m "holos render platform ./platform - $(holos --version)"
```
</TabItem>
<TabItem value="output" label="Output">
```txt
[main 5aebcf5] holos render platform ./platform - 0.93.2
5 files changed, 263 insertions(+)
create mode 100644 clusters.cue
create mode 100644 deploy/clusters/east/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/east/gitops/podinfo.application.gen.yaml
create mode 100644 deploy/clusters/west/components/podinfo/podinfo.gen.yaml
create mode 100644 deploy/clusters/west/gitops/podinfo.application.gen.yaml
```
</TabItem>
</Tabs>
## Upgrade a Helm Chart
Holos is designed to ease the burden of Day 2 operations. With Holos, upgrading
software, integrating new software, and making safe platform-wide configuration
changes become easier.
Let's upgrade the podinfo Component to see how this works in practice. First,
update the Component version field to the latest upstream Helm chart version.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
holos generate component podinfo --component-version 6.6.2
```
</TabItem>
<TabItem value="output" label="Output">
```txt
generated component
```
</TabItem>
</Tabs>
Remove the cached chart version.
<Tabs groupId="gen-podinfo">
<TabItem value="command" label="Command">
```bash
rm -rf components/podinfo/vendor
```
</TabItem>
</Tabs>
Now re-render the Platform.
<Tabs groupId="render-platform2">
<TabItem value="command" label="Command">
```bash
holos render platform ./platform
```
</TabItem>
<TabItem value="output" label="Output">
```txt
rendered components/podinfo for cluster east in 327.10475ms
rendered components/podinfo for cluster west in 327.796541ms
```
</TabItem>
</Tabs>
Notice we're still using the upstream chart without modifying it. The Holos
component wraps around the chart to mix in additional resources and integrate
the component with the broader Platform.
## Visualize the Changes
Holos makes it easier to see exactly what changes are made and which resources
will be applied to the API server. By design, Holos operates on local files,
leaving the task of applying them to ecosystem tools like `kubectl` and ArgoCD.
This allows platform operators to inspect changes during code review, or before
committing the change at all.
For example, using `git diff`, we see that the only functional change when
upgrading this Helm chart is the deployment of a new container image tag to each
cluster. Additionally, we can roll out this change gradually by applying it to
the east cluster first, then to the west cluster, limiting the potential blast
radius of a problematic change.
<Tabs groupId="git-diff">
<TabItem value="command" label="Command">
```bash
git diff deploy/clusters/east
```
</TabItem>
<TabItem value="output" label="Output">
```diff showLineNumbers
diff --git a/deploy/clusters/east/components/podinfo/podinfo.gen.yaml b/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
index 7cc3332..8c1647d 100644
--- a/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
+++ b/deploy/clusters/east/components/podinfo/podinfo.gen.yaml
@@ -5,9 +5,9 @@ kind: Service
metadata:
name: podinfo
labels:
- helm.sh/chart: podinfo-6.6.1
+ helm.sh/chart: podinfo-6.6.2
app.kubernetes.io/name: podinfo
- app.kubernetes.io/version: "6.6.1"
+ app.kubernetes.io/version: "6.6.2"
app.kubernetes.io/managed-by: Helm
spec:
type: ClusterIP
@@ -29,9 +29,9 @@ kind: Deployment
metadata:
name: podinfo
labels:
- helm.sh/chart: podinfo-6.6.1
+ helm.sh/chart: podinfo-6.6.2
app.kubernetes.io/name: podinfo
- app.kubernetes.io/version: "6.6.1"
+ app.kubernetes.io/version: "6.6.2"
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
@@ -53,7 +53,7 @@ spec:
terminationGracePeriodSeconds: 30
containers:
- name: podinfo
# highlight-next-line
- image: "ghcr.io/stefanprodan/podinfo:6.6.1"
# highlight-next-line
+ image: "ghcr.io/stefanprodan/podinfo:6.6.2"
imagePullPolicy: IfNotPresent
command:
- ./podinfo
```
</TabItem>
</Tabs>
:::tip
Holos is designed to surface the _fully rendered_ manifests intended for the
Kubernetes API server, making it easier to see and reason about platform-wide
configuration changes.
:::
## Recap {#recap}
In this quickstart guide, we learned how Holos makes it easier, safer, and more
consistent to manage a Platform composed of multiple Clusters and upstream Helm
charts.
We covered how to:
1. Generate a Git repository for the Platform config.
2. Wrap the unmodified upstream podinfo Helm chart into a Component.
3. Render an individual Component.
4. Mix-in your Platform's unique resources to all Components. For example, ArgoCD Application resources.
5. Define multiple similar, but not identical, workload clusters.
6. Render the manifests for the entire Platform with the `holos render platform` command.
7. Upgrade a Helm chart to the latest version as an important Day 2 task.
8. Visualize and surface the details of planned changes Platform wide.
## Dive Deeper
If you'd like to dive deeper, check out the [Schema API][schema] and [Core
API][core] reference docs. The main difference between the schema and core
packages is that the schema is used by users to write refined CUE, while the
core package is what the schema produces for `holos` to execute. Users rarely
need to interact with the Core API when on the happy path, but can use the core
package as an escape hatch when the happy path doesn't go where you want.
[application]: https://argo-cd.readthedocs.io/en/stable/user-guide/application-specification/
[schema]: /docs/api/schema/v1alpha3/
[core]: /docs/api/core/v1alpha3/

10
doc/md/introduction.md Normal file
View File

@@ -0,0 +1,10 @@
---
description: Holos Documentation
slug: /
---
# Introduction
:::warning TODO
See [introduction](https://github.com/facebook/docusaurus/blob/main/website/docs/introduction.mdx?plain=1)
:::

View File

@@ -1,69 +0,0 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Admonition from '@theme/Admonition';
# Quickstart Guide
This guide shows you the basics of how Holos. You'll deploy a Helm chart to
Kubernetes using a Component to see how Holos makes the process safer, easier,
and more consistent.
## What you'll need {#Requirements}
You'll need the following tools installed to complete this guide.
1. [holos](/docs/install) - to build the platform.
2. [helm](https://helm.sh/docs/intro/install/) - to render Holos components that wrap upstream Helm charts.
3. [k3d](https://k3d.io/#installation) - to provide a Kubernetes API server.
4. [OrbStack](https://docs.orbstack.dev/install) or [Docker](https://docs.docker.com/get-docker/) - to use k3d.
5. [kubectl](https://kubernetes.io/docs/tasks/tools/) - to interact with the k8s api server.
## Install Holos
Install Holos with the following command or other methods listed on the
[Installation](/docs/install/) page.
```bash
go install github.com/holos-run/holos/cmd/holos@latest
```
## Git repository
Start by initializing an empty Git repository. Holos is designed to operate
against local files in a Git repository.
<Tabs groupId="init">
<TabItem value="command" label="Command">
```bash
mkdir holos-quickstart
cd holos-quickstart
git init
```
</TabItem>
<TabItem value="output" label="Output">
```txt
Initialized empty Git repository in /holos-quickstart/.git/
```
</TabItem>
</Tabs>
This guide assumes commands are run from the root directory of this Git
repository unless otherwise stated.
## Generate the Platform {#Generate-Platform}
Generate the Platform code in the repository root. A Platform refers to all of
the software and services integrated together to provide your organization's
software development platform. In this guide the platform will contain a single
Component to demonstrate how the concepts fit together.
```bash
holos generate platform quickstart
```
Commit the generated platform config to the repository.
```bash
git add .
git commit -m "holos generate platform quickstart - $(holos --version)"
```

5
doc/md/start.md Normal file
View File

@@ -0,0 +1,5 @@
import DocCardList from '@theme/DocCardList';
# Get Started
<DocCardList />

View File

@@ -1,9 +1,14 @@
# Comparison with other tools
---
description: Compare Holos with other tools in the ecosystem.
slug: /comparison
sidebar_position: 300
---
# Comparison
:::tip
Holos is designed to complement and improve, not replace, existing tools in the cloud native ecosystem.
Holos is designed to complement and improve, not replace, existing tools in the
cloud native ecosystem.
:::
## Helm
@@ -30,13 +35,13 @@ TODO
## Timoni
| Aspect | Timoni | Holos | Comment |
| -- | -- | -- | -- |
| Language | CUE | CUE | Like Holos, Timoni is also built on CUE. |
| Artifact | OCI Image | Plain YAML Files | The Holos Authors find plain files easier to work with and reason about than OCI images. |
| Outputs to | OCI Image Repository | Local Git repository | Holos is designed for use with existing GitOps tools. |
| Concept | Module | Component | A Timoni Module is analogous to a Holos Component. |
| Concept | Bundle | Platform | A Timoni Bundle is somewhat similar, but smaller in scope to a Holos Platform. |
| Aspect | Timoni | Holos | Comment |
| ---------- | -------------------- | -------------------- | ---------------------------------------------------------------------------------------- |
| Language | CUE | CUE | Like Holos, Timoni is also built on CUE. |
| Artifact | OCI Image | Plain YAML Files | The Holos Authors find plain files easier to work with and reason about than OCI images. |
| Outputs to | OCI Image Repository | Local Git repository | Holos is designed for use with existing GitOps tools. |
| Concept | Module | Component | A Timoni Module is analogous to a Holos Component. |
| Concept | Bundle | Platform | A Timoni Bundle is somewhat similar, but smaller in scope to a Holos Platform. |
:::important

View File

@@ -1,3 +1,9 @@
---
description: Learn the concepts and domain language Holos uses.
slug: /concepts
sidebar_position: 200
---
# Concepts
## Introduction
@@ -23,21 +29,34 @@ platform engineers.
- [Platform](<#platform>) - A collection of Components integrated into a software development platform.
- [Model](<#model>) - Structured data included in the Platform specification, available to all Components. For example, your organization's domain name.
- [Rendering](<#rendering>) - Holos is a tool that makes the process of rendering Kubernetes manifests safer, easier, and consistent.
- [Cluster](<#cluster>) - A Kubernetes cluster. Components are rendered for and applied to a Cluster.
- [Fleet](<#fleet>) - A collection of Clusters with a similar purpose. A Platform is typically composed of two Fleets, one for management the second for workloads.
```mermaid
graph BT
graph TB
Platform[<a href="#platform">Platform</a>]
Component[<a href="#component">Components</a>]
Cluster[<a href="#cluster">Cluster</a>]
Fleet[<a href="#fleet">Fleet</a>]
Component[<a href="#component">Component</a>]
Helm[<a href="#component">Helm</a>]
Kustomize[<a href="#component">Kustomize</a>]
CUE[<a href="#component">CUE</a>]
Component --> Platform
Cluster --> Platform
Fleet --> Cluster
Component --> Fleet
Helm --> Component
Kustomize --> Component
CUE --> Component
```
:::tip
This graph is organized as a tree. We often say configuration at the root
defines the broad Platform. Configuration at a leaf defines a Component of the
Platform. The concept of a tree also reflects the filesystem organization of
the configuration.
:::
<!--
```mermaid
@@ -336,6 +355,16 @@ graph LR
FS --> kubectl --> C
```
## Cluster
A Cluster represents a Kubernetes cluster. One component may be reused across
multiple different Clusters.
## Fleet
A Fleet represents a group of Clusters that share a similar purpose. A Platform
typically has two Fleets, one for management and one for workloads.
[krm]: https://docs.google.com/document/d/1RmHXdLhNbyOWPW_AtnnowaRfGejw-qlKQIuLKQWlwzs/view#heading=h.sa6p0aye4ide
[Platform]: /docs/api/core/v1alpha2/#Platform
[BuildPlan]: /docs/api/core/v1alpha2/#BuildPlan

View File

@@ -1,3 +1,9 @@
---
description: Install the Holos executable.
slug: /install
sidebar_position: 100
---
# Installation
Holos is distributed as a single file executable.

View File

@@ -41,12 +41,7 @@ const config: Config = {
[
'@docusaurus/plugin-client-redirects',
{
redirects: [
{
from: "/docs/tutorial/local/k3d/",
to: "/docs/guides/try-holos/",
},
],
redirects: [],
},
],
],
@@ -98,19 +93,14 @@ const config: Config = {
items: [
{
type: 'doc',
docId: 'quickstart/index',
docId: 'guides/quickstart',
position: 'left',
label: 'Try Holos',
},
{ to: '/docs', label: 'Docs', position: 'left' },
{
type: 'doc',
docId: 'concepts',
position: 'left',
label: 'Docs',
},
{
type: 'docSidebar',
sidebarId: 'api',
docId: 'api',
position: 'left',
label: 'API',
},
@@ -135,7 +125,7 @@ const config: Config = {
title: 'Docs',
items: [
{
label: 'Get Started',
label: 'Quickstart',
to: '/docs/quickstart',
},
{
@@ -144,11 +134,11 @@ const config: Config = {
},
{
label: 'Documentation',
to: '/docs/intro',
to: '/docs',
},
{
label: 'API Reference',
to: '/docs/api/core/v1alpha2',
to: '/docs/api',
},
],
},

View File

@@ -12,22 +12,63 @@ import type { SidebarsConfig } from '@docusaurus/plugin-content-docs';
*/
const sidebars: SidebarsConfig = {
doc: [
'quickstart/index',
'concepts',
'install',
'comparison',
],
api: [
'introduction',
{
label: 'Core API',
label: 'Getting Started',
type: 'category',
collapsed: true,
link: { type: 'doc', id: 'start' },
items: [
'api/core/v1alpha3',
'api/core/v1alpha2',
{
type: 'autogenerated',
dirName: 'start',
},
],
},
'cli',
{
label: 'Guides',
type: 'category',
collapsed: false,
link: { type: 'doc', id: 'guides' },
items: [
{
type: 'autogenerated',
dirName: 'guides',
},
],
},
{
label: 'API Reference',
type: 'category',
collapsed: true,
link: { type: 'doc', id: 'api' },
items: [
{
label: 'Schema API',
type: 'category',
link: { type: 'doc', id: 'api/schema' },
collapsed: true,
items: [
{
type: 'autogenerated',
dirName: 'api/schema',
},
]
},
{
label: 'Core API',
type: 'category',
link: { type: 'doc', id: 'api/core' },
collapsed: true,
items: [
{
type: 'autogenerated',
dirName: 'api/core',
},
]
},
]
},
],
};

View File

@@ -4,6 +4,12 @@
* work well for content-centric websites.
*/
/* Enable wrapping by default for mobile */
pre code {
white-space: pre-wrap;
overflow-wrap: anywhere;
}
/* You can override the default Infima variables here. */
:root {
--ifm-link-color: #268bd2;

View File

@@ -0,0 +1,38 @@
#! /bin/bash
#
set -euo pipefail
mkcert --install
tmpdir="$(mktemp -d)"
finish() {
[[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"
# Create the local CA Secret with ca.crt, tls.crt, tls.key
mkdir local-ca
cd local-ca
CAROOT="$(mkcert -CAROOT)"
cp -p "${CAROOT}/rootCA.pem" ca.crt
cp -p "${CAROOT}/rootCA.pem" tls.crt
cp -p "${CAROOT}/rootCA-key.pem" tls.key
kubectl create secret generic --from-file=. --dry-run=client -o yaml local-ca > ../local-ca.yaml
cd ..
echo 'type: kubernetes.io/tls' >> local-ca.yaml
kubectl apply --server-side=true -f- <<EOF
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: cert-manager
name: cert-manager
spec:
finalizers:
- kubernetes
EOF
kubectl apply -n cert-manager --server-side=true -f local-ca.yaml

View File

@@ -0,0 +1,38 @@
#! /bin/bash
#
set -euo pipefail
tmpdir="$(mktemp -d)"
finish() {
[[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"
brew install dnsmasq
cat <<EOF >"$(brew --prefix)/etc/dnsmasq.d/holos.localhost.conf"
# Refer to https://holos.run/docs/tutorial/local/k3d/
address=/holos.localhost/127.0.0.1
EOF
if [[ -r /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist ]]; then
echo "dnsmasq already configured"
else
sudo cp "$(brew list dnsmasq | grep 'dnsmasq.plist$')" \
/Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo launchctl unload /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
dscacheutil -flushcache
echo "dnsmasq configured"
fi
sudo mkdir -p /etc/resolver
sudo tee /etc/resolver/holos.localhost <<EOF
domain holos.localhost
nameserver 127.0.0.1
EOF
sudo killall -HUP mDNSResponder
echo "all done."

View File

@@ -75,9 +75,6 @@ func (b *buildPlanWrapper) validate() error {
if bp.Kind != v1.BuildPlanKind {
errs = append(errs, fmt.Sprintf("kind invalid: want: %s have: %s", v1alpha1.BuildPlanKind, bp.Kind))
}
if bp.APIVersion != v1.APIVersion {
errs = append(errs, fmt.Sprintf("apiVersion invalid: want: %s have: %s", v1.APIVersion, bp.APIVersion))
}
if len(errs) > 0 {
return errors.New("invalid BuildPlan: " + strings.Join(errs, ", "))
}

View File

@@ -7,7 +7,6 @@ import (
"strings"
"github.com/holos-run/holos/internal/cli/command"
"github.com/holos-run/holos/internal/client"
"github.com/holos-run/holos/internal/errors"
"github.com/holos-run/holos/internal/generate"
"github.com/holos-run/holos/internal/holos"
@@ -35,11 +34,9 @@ func NewPlatform(cfg *holos.Config) *cobra.Command {
cmd.Args = cobra.ExactArgs(1)
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := cmd.Root().Context()
clientContext := holos.NewClientContext(ctx)
client := client.New(client.NewConfig(cfg))
for _, name := range args {
if err := generate.GeneratePlatform(ctx, client, clientContext.OrgID, name); err != nil {
if err := generate.GeneratePlatform(ctx, name); err != nil {
return errors.Wrap(err)
}
}
@@ -55,33 +52,13 @@ func NewComponent() *cobra.Command {
cmd := command.New("component")
cmd.Short = "generate a component from an embedded schematic"
cmd.AddCommand(NewCueComponent())
cmd.AddCommand(NewHelmComponent())
return cmd
}
func NewHelmComponent() *cobra.Command {
cmd := command.New("helm")
cmd.Short = "generate a helm component from a schematic"
for _, name := range generate.HelmComponents() {
cmd.AddCommand(makeSchematicCommand("helm", name))
for _, name := range generate.Components("v1alpha3") {
cmd.AddCommand(makeSchematicCommand("v1alpha3", name))
}
return cmd
}
func NewCueComponent() *cobra.Command {
cmd := command.New("cue")
cmd.Short = "generate a cue component from a schematic"
for _, name := range generate.CueComponents() {
cmd.AddCommand(makeSchematicCommand("cue", name))
}
return cmd
}
func makeSchematicCommand(kind, name string) *cobra.Command {
cmd := command.New(name)
cfg, err := generate.NewSchematic(filepath.Join("components", kind), name)

View File

@@ -81,22 +81,10 @@ func (s *Schematic) FlagSet() *flag.FlagSet {
return fs
}
// CueComponents returns a slice of embedded component schematics or nil if there are none.
func CueComponents() []string {
entries, err := fs.ReadDir(components, filepath.Join(componentsRoot, "cue"))
if err != nil {
return nil
}
dirs := make([]string, 0, len(entries))
for _, entry := range entries {
dirs = append(dirs, entry.Name())
}
return dirs
}
// HelmComponents returns a slice of embedded component schematics or nil if there are none.
func HelmComponents() []string {
entries, err := fs.ReadDir(components, filepath.Join(componentsRoot, "helm"))
// Components returns a slice of embedded component schematics or nil if there
// are none.
func Components(name string) []string {
entries, err := fs.ReadDir(components, filepath.Join(componentsRoot, name))
if err != nil {
return nil
}
@@ -131,8 +119,8 @@ func makeRenderFunc[T any](log *slog.Logger, path string, cfg T) func([]byte) *b
func GenerateComponent(ctx context.Context, kind string, name string, cfg *Schematic) error {
// use name from args to build the source path
path := filepath.Join(componentsRoot, kind, name)
// use cfg.Name from flags to build the destination path
dstPath := filepath.Join(getCwd(ctx), cfg.Name)
// write to the current directory.
dstPath := filepath.Join(getCwd(ctx))
log := logger.FromContext(ctx).With("name", cfg.Name, "path", dstPath)
log.DebugContext(ctx, "mkdir")
if err := os.MkdirAll(dstPath, os.ModePerm); err != nil {

View File

@@ -1,5 +1,8 @@
package holos
// Produce a helm chart build plan.
(#Helm & Chart).Output
let Chart = {
Name: "{{ .Name }}"
Version: "{{ .Version }}"
@@ -10,6 +13,3 @@ let Chart = {
Values: {}
}
// Produce a helm chart build plan.
(#Helm & Chart).Output

View File

@@ -0,0 +1,9 @@
package holos
// Manage podinfo on workload clusters only
for Cluster in #Fleets.workload.clusters {
#Platform: Components: "\(Cluster.name)/podinfo": {
path: "components/podinfo"
cluster: Cluster.name
}
}

View File

@@ -37,27 +37,9 @@ func Platforms() []string {
return dirs
}
func writePlatformMetadata(ctx context.Context, rpc *client.Client, orgID string, name string) error {
func initPlatformMetadata(ctx context.Context, name string) error {
log := logger.FromContext(ctx)
// Link the local platform the SaaS platform ID.
rpcPlatforms, err := rpc.Platforms(ctx, orgID)
if err != nil {
return errors.Wrap(err)
}
var rpcPlatform *platform.Platform
for _, p := range rpcPlatforms {
if p.GetName() == name {
rpcPlatform = p
break
}
log.DebugContext(ctx, "checking platform", "want", name, "have", p.GetName())
}
if rpcPlatform == nil {
return errors.Wrap(errors.New("cannot generate: platform not found in the holos server"))
}
rpcPlatform := &platform.Platform{Name: name}
// Write the platform data.
encoder := protojson.MarshalOptions{Indent: " "}
data, err := encoder.Marshal(rpcPlatform)
@@ -67,7 +49,7 @@ func writePlatformMetadata(ctx context.Context, rpc *client.Client, orgID string
if len(data) > 0 {
data = append(data, '\n')
}
log = log.With("platform_id", rpcPlatform.GetId())
if err := os.WriteFile(client.PlatformMetadataFile, data, 0644); err != nil {
return errors.Wrap(fmt.Errorf("could not write platform metadata: %w", err))
}
@@ -78,7 +60,7 @@ func writePlatformMetadata(ctx context.Context, rpc *client.Client, orgID string
// GeneratePlatform writes the cue code for a platform to the local working
// directory.
func GeneratePlatform(ctx context.Context, rpc *client.Client, orgID string, name string) error {
func GeneratePlatform(ctx context.Context, name string) error {
log := logger.FromContext(ctx)
// Check for a valid platform
platformPath := filepath.Join(platformsRoot, name)
@@ -90,7 +72,7 @@ func GeneratePlatform(ctx context.Context, rpc *client.Client, orgID string, nam
log.DebugContext(ctx, fmt.Sprintf("skipped write %s: already exists", client.PlatformConfigFile))
} else {
if os.IsNotExist(err) {
if err := writePlatformMetadata(ctx, rpc, orgID, name); err != nil {
if err := initPlatformMetadata(ctx, name); err != nil {
return errors.Wrap(err)
}
} else {

View File

@@ -6,11 +6,6 @@ package v1alpha3
import "google.golang.org/protobuf/types/known/structpb"
#PlatformMetadata: {
// Name represents the Platform name.
name: string @go(Name)
}
// Platform represents a platform to manage. A Platform resource informs holos
// which components to build. The platform resource also acts as a container
// for the platform model form values provided by the PlatformService. The
@@ -30,13 +25,18 @@ import "google.golang.org/protobuf/types/known/structpb"
spec: #PlatformSpec @go(Spec)
}
#PlatformMetadata: {
// Name represents the Platform name.
name: string @go(Name)
}
// PlatformSpec represents the specification of a Platform. Think of a platform
// specification as a list of platform components to apply to a list of
// kubernetes clusters combined with the user-specified Platform Model.
#PlatformSpec: {
// Model represents the platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
model: structpb.#Struct @go(Model)
model: structpb.#Struct & {...} @go(Model)
// Components represents a list of holos components to manage.
components: [...#PlatformSpecComponent] @go(Components,[]PlatformSpecComponent)

View File

@@ -0,0 +1,151 @@
// Code generated by cue get go. DO NOT EDIT.
//cue:generate cue get go github.com/holos-run/holos/api/schema/v1alpha3
// Package v1alpha3 contains CUE definitions intended as convenience wrappers
// around the core data types defined in package core. The purpose of these
// wrappers is to make life easier for platform engineers by reducing boiler
// plate code and generating component build plans in a consistent manner.
package v1alpha3
import (
core "github.com/holos-run/holos/api/core/v1alpha3"
"google.golang.org/protobuf/types/known/structpb"
)
// Helm provides a BuildPlan via the Output field which contains one HelmChart
// from package core. Useful as a convenience wrapper to render a HelmChart
// with optional mix-in resources and Kustomization post-processing.
#Helm: {
// Name represents the chart name.
Name: string
// Version represents the chart version.
Version: string
// Namespace represents the helm namespace option when rendering the chart.
Namespace: string
// Resources are kubernetes api objects to mix into the output.
Resources: {...} & {...} @go(,map[string]any)
// Repo represents the chart repository
Repo: {
name: string @go(Name)
url: string @go(URL)
} @go(,"struct{Name string \"json:\\\"name\\\"\"; URL string \"json:\\\"url\\\"\"}")
// Values represents data to marshal into a values.yaml for helm.
Values: _ & {...} @go(,interface{})
// Chart represents the derived HelmChart for inclusion in the BuildPlan
// Output field value. The default HelmChart field values are derived from
// other Helm field values and should be sufficient for most use cases.
Chart: core.#HelmChart
// EnableKustomizePostProcessor processes helm output with kustomize if true.
EnableKustomizePostProcessor: bool & (true | *false)
// KustomizeFiles represents additional files to include in a Kustomization
// resources list. Useful to patch helm output. The implementation is a
// struct with filename keys and structs as values. Holos encodes the struct
// value to yaml then writes the result to the filename key. Component
// authors may then reference the filename in the kustomization.yaml resources
// or patches lists.
// Requires EnableKustomizePostProcessor: true.
KustomizeFiles: {...} & {[string]: {...}} @go(,map[string]any)
// KustomizePatches represents patches to apply to the helm output. Requires
// EnableKustomizePostProcessor: true.
KustomizePatches: {...} & {[string]: {...}} @go(,map[core.InternalLabel]any)
// KustomizeResources represents additional resources files to include in the
// kustomize resources list.
KustomizeResources: {...} & {[string]: {...}} @go(,map[string]any)
// ArgoConfig represents the ArgoCD GitOps configuration for this Component.
ArgoConfig: #ArgoConfig
// Output represents the derived BuildPlan for the Holos cli to render.
Output: core.#BuildPlan
}
// ArgoConfig represents the ArgoCD GitOps configuration for a Component.
// Useful to define once at the root of the Platform configuration and reuse
// across all Components.
#ArgoConfig: {
// Enabled causes holos to render an ArgoCD Application resource for GitOps if true.
Enabled: bool & (true | *false)
// ClusterName represents the cluster within the platform the Application
// resource is intended for.
ClusterName: string
// DeployRoot represents the path from the git repository root to the `deploy`
// rendering output directory. Used as a prefix for the
// Application.spec.source.path field.
DeployRoot: string & (string | *".")
// RepoURL represents the value passed to the Application.spec.source.repoURL
// field.
RepoURL: string
// TargetRevision represents the value passed to the
// Application.spec.source.targetRevision field. Defaults to the branch named
// main.
TargetRevision: string & (string | *"main")
}
// Cluster represents a cluster managed by the Platform.
#Cluster: {
// Name represents the cluster name, for example "east1", "west1", or
// "management".
name: string @go(Name)
// Primary represents if the cluster is marked as the primary among a set of
// candidate clusters. Useful for promotion of database leaders.
primary: bool & (true | *false) @go(Primary)
}
// Fleet represents a named collection of similarly configured Clusters. Useful
// to segregate workload clusters from their management cluster.
#Fleet: {
name: string @go(Name)
// Clusters represents a mapping of Clusters by their name.
clusters: {[string]: #Cluster} & {[Name=_]: name: Name} @go(Clusters,map[string]Cluster)
}
// StandardFleets represents the standard set of Clusters in a Platform
// segmented into Fleets by their purpose. The management Fleet contains a
// single Cluster, for example a GKE autopilot cluster with no workloads
// deployed for reliability and cost efficiency. The workload Fleet contains
// all other Clusters which contain workloads and sync Secrets from the
// management cluster.
#StandardFleets: {
// Workload represents a Fleet of zero or more workload Clusters.
workload: #Fleet & {name: "workload"} @go(Workload)
// Management represents a Fleet with one Cluster named management.
management: #Fleet & {name: "management", clusters: management: _} @go(Management)
}
// Platform is a convenience structure to produce a core Platform specification
// value in the Output field. Useful to collect components at the root of the
// Platform configuration tree as a struct, which are automatically converted
// into a list for the core Platform spec output.
#Platform: {
// Name represents the Platform name.
Name: string & (string | *"holos")
// Components is a structured map of components to manage by their name.
Components: {[string]: core.#PlatformSpecComponent} @go(,map[string]core.PlatformSpecComponent)
// Model represents the Platform model holos gets from from the
// PlatformService.GetPlatform rpc method and provides to CUE using a tag.
Model: structpb.#Struct & {...}
// Output represents the core Platform spec for the holos cli to iterate over
// and render each listed Component, injecting the Model.
Output: core.#Platform
}

View File

@@ -0,0 +1,210 @@
package v1alpha3
import (
"encoding/yaml"
core "github.com/holos-run/holos/api/core/v1alpha3"
kc "sigs.k8s.io/kustomize/api/types"
corev1 "k8s.io/api/core/v1"
appsv1 "k8s.io/api/apps/v1"
rbacv1 "k8s.io/api/rbac/v1"
batchv1 "k8s.io/api/batch/v1"
app "argoproj.io/application/v1alpha1"
)
#Resources: {
[Kind=string]: [InternalLabel=string]: {
kind: Kind
metadata: name: string | *InternalLabel
}
ClusterRole: [_]: rbacv1.#ClusterRole
ClusterRoleBinding: [_]: rbacv1.#ClusterRoleBinding
ConfigMap: [_]: corev1.#ConfigMap
CronJob: [_]: batchv1.#CronJob
Deployment: [_]: appsv1.#Deployment
Job: [_]: batchv1.#Job
Namespace: [_]: corev1.#Namespace
Role: [_]: rbacv1.#Role
RoleBinding: [_]: rbacv1.#RoleBinding
Service: [_]: corev1.#Service
ServiceAccount: [_]: corev1.#ServiceAccount
StatefulSet: [_]: appsv1.#StatefulSet
...
}
#Helm: {
Name: string
Version: string
Namespace: string
Resources: #Resources
Repo: {
name: string | *""
url: string | *""
}
Values: {...}
Chart: core.#HelmChart & {
metadata: name: string | *Name
metadata: namespace: string | *Namespace
chart: name: string | *Name
chart: release: chart.name
chart: version: string | *Version
chart: repository: Repo
// Render the values to yaml for holos to provide to helm.
valuesContent: yaml.Marshal(Values)
// Kustomize post-processor
if EnableKustomizePostProcessor == true {
// resourcesFile represents the file helm output is written two and
// kustomize reads from. Typically "resources.yaml" but referenced as a
// constant to ensure the holos cli uses the same file.
kustomize: resourcesFile: core.#ResourcesFile
// kustomizeFiles represents the files in a kustomize directory tree.
kustomize: kustomizeFiles: core.#FileContentMap
for FileName, Object in KustomizeFiles {
kustomize: kustomizeFiles: "\(FileName)": yaml.Marshal(Object)
}
}
apiObjectMap: (#APIObjects & {apiObjects: Resources}).apiObjectMap
}
// EnableKustomizePostProcessor processes helm output with kustomize if true.
EnableKustomizePostProcessor: true | *false
// KustomizeFiles represents additional files to include in a Kustomization
// resources list. Useful to patch helm output. The implementation is a
// struct with filename keys and structs as values. Holos encodes the struct
// value to yaml then writes the result to the filename key. Component
// authors may then reference the filename in the kustomization.yaml resources
// or patches lists.
// Requires EnableKustomizePostProcessor: true.
KustomizeFiles: {
// Embed KustomizeResources
KustomizeResources
// The kustomization.yaml file must be included for kustomize to work.
"kustomization.yaml": kc.#Kustomization & {
apiVersion: "kustomize.config.k8s.io/v1beta1"
kind: "Kustomization"
resources: [core.#ResourcesFile, for FileName, _ in KustomizeResources {FileName}]
patches: [for x in KustomizePatches {x}]
}
}
// KustomizePatches represents patches to apply to the helm output. Requires
// EnableKustomizePostProcessor: true.
KustomizePatches: [ArbitraryLabel=string]: kc.#Patch
// KustomizeResources represents additional resources files to include in the
// kustomize resources list.
KustomizeResources: [FileName=string]: {...}
// ArgoConfig represents the ArgoCD GitOps integration for this Component.
ArgoConfig: _
// output represents the build plan provided to the holos cli.
Output: #BuildPlan & {
_Name: Name
_Namespace: Namespace
_ArgoConfig: ArgoConfig
spec: components: helmChartList: [Chart]
}
}
#BuildPlan: core.#BuildPlan & {
_Name: string
_Namespace?: string
_ArgoConfig: #ArgoConfig
if _ArgoConfig.Enabled {
let NAME = "gitops/\(_Name)"
// Render the ArgoCD Application for GitOps as an additional Component of
// the BuildPlan.
spec: components: resources: (NAME): {
metadata: name: NAME
if _Namespace != _|_ {
metadata: namespace: _Namespace
}
deployFiles: (#Argo & {ComponentName: _Name, ArgoConfig: _ArgoConfig}).deployFiles
}
}
}
// #Argo represents an argocd Application resource for each component, written
// using the #HolosComponent.deployFiles field.
#Argo: {
ComponentName: string
ArgoConfig: #ArgoConfig
Application: app.#Application & {
metadata: name: ComponentName
metadata: namespace: "argocd"
spec: {
destination: server: "https://kubernetes.default.svc"
project: "default"
source: {
path: "\(ArgoConfig.DeployRoot)/deploy/clusters/\(ArgoConfig.ClusterName)/components/\(ComponentName)"
repoURL: ArgoConfig.RepoURL
targetRevision: ArgoConfig.TargetRevision
}
}
}
// deployFiles represents the output files to write along side the component.
deployFiles: "clusters/\(ArgoConfig.ClusterName)/gitops/\(ComponentName).application.gen.yaml": yaml.Marshal(Application)
}
// #ArgoDefaultSyncPolicy represents the default argo sync policy.
#ArgoDefaultSyncPolicy: {
automated: {
prune: bool | *true
selfHeal: bool | *true
}
syncOptions: [
"RespectIgnoreDifferences=true",
"ServerSideApply=true",
]
retry: limit: number | *2
retry: backoff: {
duration: string | *"5s"
factor: number | *2
maxDuration: string | *"3m0s"
}
}
// #APIObjects defines the output format for kubernetes api objects. The holos
// cli expects the yaml representation of each api object in the apiObjectMap
// field.
#APIObjects: core.#APIObjects & {
// apiObjects represents the un-marshalled form of each kubernetes api object
// managed by a holos component.
apiObjects: {
[Kind=string]: {
[string]: {
kind: Kind
...
}
}
}
// apiObjectMap holds the marshalled representation of apiObjects
for kind, v in apiObjects {
for name, obj in v {
apiObjectMap: (kind): (name): yaml.Marshal(obj)
}
}
}
#Platform: {
Name: _
Model: _
Components: [string]: _
Output: metadata: name: Name
Output: spec: model: Model
Output: spec: components: [for c in Components {c}]
}

View File

@@ -9,6 +9,9 @@ package platforms
//go generate rm -rf cue.mod/gen/github.com/holos-run/holos/api/meta
//go:generate cue get go github.com/holos-run/holos/api/meta/...
//go generate rm -rf cue.mod/gen/github.com/holos-run/holos/api/schema
//go:generate cue get go github.com/holos-run/holos/api/schema/...
//go generate rm -rf cue.mod/gen/github.com/holos-run/holos/service/gen/holos/object
//go:generate cue import ../../../service/holos/object/v1alpha1/object.proto -o cue.mod/gen/github.com/holos-run/holos/service/gen/holos/object/v1alpha1/object.proto_gen.cue -I ../../../proto -f
//go:generate rm -f cue.mod/gen/github.com/holos-run/holos/service/gen/holos/object/v1alpha1/object.pb_go_gen.cue

View File

@@ -0,0 +1,8 @@
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
vendor/

View File

@@ -0,0 +1 @@
{}

View File

@@ -0,0 +1,3 @@
package holos
#Platform: Name: "guide"

View File

@@ -0,0 +1,3 @@
package holos
#Platform.Output

View File

@@ -0,0 +1,5 @@
# Holos
Generated for use with [Holos Guides][guides].
[guides]: https://holos.run/docs/guides/

View File

@@ -0,0 +1,15 @@
package holos
import schema "github.com/holos-run/holos/api/schema/v1alpha3"
#Helm: schema.#Helm & {
ArgoConfig: #ArgoConfig
}
#ArgoConfig: schema.#ArgoConfig & {
ClusterName: _ClusterName
}
#Fleets: schema.#StandardFleets
#Platform: schema.#Platform

View File

@@ -0,0 +1,14 @@
package holos
import (
"encoding/json"
dto "github.com/holos-run/holos/service/gen/holos/object/v1alpha1:object"
)
// _ClusterName is the --cluster-name flag value provided by the holos cli.
_ClusterName: string @tag(cluster, type=string)
// _PlatformConfig represents all of the data passed from holos to cue, used to
// carry the platform and project models.
_PlatformConfig: dto.#PlatformConfig & json.Unmarshal(_PlatformConfigJSON)
_PlatformConfigJSON: string | *"{}" @tag(platform_config, type=string)

View File

@@ -0,0 +1,8 @@
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
vendor/

View File

@@ -0,0 +1 @@
{}

View File

@@ -0,0 +1,3 @@
package holos
#Platform.Output

View File

@@ -0,0 +1,3 @@
package holos
#Platform: Name: "quickstart"

View File

@@ -0,0 +1,5 @@
# Holos Quickstart
Generated from the [Holos Quickstart][quickstart] guide.
[quickstart]: https://holos.run/docs/quickstart/

View File

@@ -0,0 +1,15 @@
package holos
import schema "github.com/holos-run/holos/api/schema/v1alpha3"
#Helm: schema.#Helm & {
ArgoConfig: #ArgoConfig
}
#ArgoConfig: schema.#ArgoConfig & {
ClusterName: _ClusterName
}
#Fleets: schema.#StandardFleets
#Platform: schema.#Platform

View File

@@ -0,0 +1,14 @@
package holos
import (
"encoding/json"
dto "github.com/holos-run/holos/service/gen/holos/object/v1alpha1:object"
)
// _ClusterName is the --cluster-name flag value provided by the holos cli.
_ClusterName: string @tag(cluster, type=string)
// _PlatformConfig represents all of the data passed from holos to cue, used to
// carry the platform and project models.
_PlatformConfig: dto.#PlatformConfig & json.Unmarshal(_PlatformConfigJSON)
_PlatformConfigJSON: string | *"{}" @tag(platform_config, type=string)

View File

@@ -1 +1 @@
0
3