Install github.com/rogpeppe/go-internal/cmd/testscript when running
make tools for script-based testing support.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Jeff McCune <jeff@openinfrastructure.co>
Previously the command and kustomize transformers were implemented as a
stand alone function. The current method is to implement tasks as
methods on the generator, transformer, or validator structs. This patch
refactors the stand alone functions to methods for consistency.
It also adds test coverage for the command and kustomize transformers.
Previously, holos loaded the full CUE Instance to discriminate against
the APIVersion and Kind. This is a problem because we need to know the
api version to know what tags to inject. For example, v1alpha6 needs to
have the build context injected but v1alpha5 does not.
This patch fixes the problem by changing holos render component to look
for a `typemeta.yaml` file in the component directory. The command line
tool discriminates on the kind and apiVersion specified in this file
prior to building the CUE instance.
Result:
We're able to inject version specific cue tags into the CUE instance.
Note previously the build context was passed to CUE by writing a file to
the filesystem and using the embed functionality. We cannot proceed
with this embed approach because the filesystem writes are not safe for
concurrent use. Therefore, we inject the build context as a tag which
is safe for concurrent use.
Previously using make bump to bump a version did not also update all of
the test cases and documentation to reflect the new version. This patch
updates the make bump tasks call HOLOS_UPDATE_SCRIPTS=1 scripts/test to
keep the test cases and documentation in sync with the new version.
Previously the image is build on merge to main, but not deployed
anywhere. This patch adds steps to the publish workflow to deploy the
image that was published using gitops and argocd.
On a release, make tools is run which pulls in the latest connect tools
for angular. This is a problem because it makes the git tree dirty.
The packages should be in the package.json file and the lock file so
these additional steps should not be necessary.
Remove them.
Desired result is make tools is idempotent and installs the correct
pinned versions necessary to build and release the container image.
Now that we have multi-platform images, we need a way to easily deploy
them. This involves changing the image tag. kustomize edit is often
used to bump image tags, but we can do better providing it directly in
the unified CUE configuration.
This patch modifies the builder to unify user data *.json files
recursively under userdata/ into the #UserData definition of the holos
entrypoint.
This is to support automation that writes simple json files to version
control, executes holos render platform, then commits and pushes the
results for git ops to take over deployment.
The make deploy target is the reason this change exists, to demonstrate
how to automatically deploy a new container image.
Previously, the Tiltfile was hard-wired to Jeff's development
environment on the k2 cluster on-prem. This doesn't work for other
contributors.
This patch fixes the problem by re-using the [Try Holos Locally][1]
documentation to create a local development enironment. This has a
number of benefits. The evaluation documentation will be kept up to
date because it doubles as our development environment. Developing
locally is preferrable to developing in a remote cluster. Hostnames and
URL's can be constant, e.g. https://app.holos.localhost/ for local dev
and https://app.holos.run/ for production. We don't need to push to a
remote container registry, k3d has a local registry built in that works
with Tilt.
The only difference presently between evaluation and development when
following the local/k3d doc is the addition of a local registry.
With this patch holos starts up and is accessible at
https://app.holos.localhost/
[1]: https://holos.run/docs/tutorial/local/k3d/
Generating the docusaurus site is not idempotent like generating the
Angular web app. This is a problem for building and releasing the
executable because it creates a dirty git state.
Embedding the doc website into the executable is no longer necessary
since we're deploying the site with Cloudflare pages. Remove it from
the compiled executable as a result.
The API docs are not published yet becuase the module is private. Our
own docs site does not have any API reference docs.
This patch adds auto-generated markdown docs for the core v1alpha2 types
by generating them directly from the go source code.
Some light editing of the output of `gomarkdoc` is necessary to get the
heading anchor tags to align correctly for Docusaurus.
The github workflows fail because yarn is not available. The Angular
frontend app uses npm so we should also use npm for the website to
minimize dependencies.
Previously `go install` fails to install holos.
```
❯ go install github.com/holos-run/holos/cmd/holos@latest
../../go/pkg/mod/github.com/holos-run/holos@v0.86.0/internal/frontend/frontend.go:25:12: pattern holos/dist/holos/ui/index.html: no matching files found
../../go/pkg/mod/github.com/holos-run/holos@v0.86.0/doc/website/website.go:14:12: pattern all:build: no matching files found
```
This is because we do not commit required files. This patch fixes the
problem by following Rob Pike's guidance to commit generated files.
This patch also replaces the previous use of Makefile tasks to generate
code with //go:generate directives.
This means the process of keeping the source code clean is straight
forward:
```
git clone
make tools
make generate
make build
```
Refer to https://go.dev/blog/generate
> Also, if the containing package is intended for import by go get, once
> the file is generated (and tested!) it must be checked into the source
> code repository to be available to clients. - Rob Pike
Previously docs are not published. This patch adds Docusaurus into the
doc/website directory which is also a Go package to embed the static
site into the executable.
Serve the site using http.Server with a h2c handler with the command:
holos website --log-format=json --log-drop=source
The website subcommand is intended to be run from a container as a
Deployment. For expedience, the website subcommand doesn't use the
signals package like the server subcommand does. Consider using it for
graceful Deployment restarts.
Refer to https://github.com/ent/ent/tree/master/doc/website
Previously in v1alpha1, all Holos structs are located in the same
package. This makes it difficult to focus on only the structs necessary
to transfer configuration data from CUE to the `holos` cli.
This patch splits the structs into `meta` and `core` where the core
package holds the structs end users should refer to and focus on. Only
the Platform resource is in core now, but other BuildPlan types will be
added shortly.
This sub-command renders the web app form from CUE code and updates the
form using the `holos.platform.v1alpha1.PlatformService/UpdatePlatform`
rpc method.
Example use case, starting fresh:
```
rm -rf ~/holos
mkdir ~/holos
cd ~/holos
```
Step 1: Login
```sh
holos login
```
```txt
9:53AM INF login.go:40 logged in as jeff@ois.run version=0.79.0 name="Jeff McCune" exp="2024-05-17 21:16:07 -0700 PDT" email=jeff@ois.run
```
Step 2: Register to create server side resources.
```sh
holos register user
```
```
9:52AM INF register.go:68 user version=0.79.0 email=jeff@ois.run user_id=018f826d-85a8-751d-81ee-64d0f2775b3f org_id=018f826d-85a8-751e-98dd-a6cddd9dd8f0
```
Step 3: Generate the bare platform in the local filesystem.
```sh
holos generate platform bare
```
```txt
9:52AM INF generate.go:79 wrote platform.metadata.json version=0.79.0 platform_id=018f826d-85a8-751f-96d0-0d2bf70df909 path=/home/jeff/holos/platform.metadata.json
9:52AM INF generate.go:91 generated platform bare version=0.79.0 platform_id=018f826d-85a8-751f-96d0-0d2bf70df909 path=/home/jeff/holos
```
Step 4: Push the platform form to the `holos server` web app.
```sh
holos push platform form .
```
```txt
9:52AM INF client.go:67 updated platform version=0.79.0 platform_id=018f826d-85a8-751f-96d0-0d2bf70df909 duration=73.62995ms
```
At this point the platform form is published and functions as expected
when visiting the platform web interface.
When the user generates a platform, we need to know the platform ID it's
linked to in the holos server. If there is no platform with the same
name, the `holos generate platform` command should error out.
This is necessary because the first thing we want to show is pushing an
updated form to `holos server`. To update the web ui the CLI needs to
know the platform ID to update.
This patch modifies the generate command to obtain a list of platforms
for the org and verify the generated name matches one of the platforms
that already exists.
A future patch could have the `generate platform` command call the
`holos.platform.v1alpha1.PlatformService.CreatePlatform` method if the
platform isn't found.
Results:
```sh
holos generate platform bare
```
```txt
4:15PM INF generate.go:77 wrote platform.metadata.json version=0.77.1 platform_id=018f826d-85a8-751f-96d0-0d2bf70df909 path=/home/jeff/holos/platform.metadata.json
4:15PM INF generate.go:89 generated platform bare version=0.77.1 platform_id=018f826d-85a8-751f-96d0-0d2bf70df909 path=/home/jeff/holos
```
```sh
cat platform.metadata.json
```
```json
{
"id": "018f826d-85a8-751f-96d0-0d2bf70df909",
"name": "bare",
"display_name": "Bare Platform"
}
```
This patch refactors the API to be resource-oriented around one service
per resource type. PlatformService, OrganizationService, UserService,
etc...
Validation is improved to use CEL rules provided by [protovalidate][1].
Place holders for FieldMask and other best practices are added, but are
unimplemented as per [API Best Practices][2].
The intent is to set us up well for copying and pasting solid existing
examples as we add features.
With this patch the server and web app client are both updated to use
the refactored API, however the following are not working:
1. Update the model.
2. Field Masks.
[1]: https://buf.build/bufbuild/protovalidate
[2]: https://protobuf.dev/programming-guides/api/
Add Tilt back from holos server
Note with this patch the ec-creds.yaml file needs to be applied to the
provisioner and an external secret used to sync the image pull creds.
With this patch the dev instance is accessible behind the auth proxy.
pgAdmin also works from the Tilt UI.
https://jeff.holos.dev.k2.ois.run/app/start
Prior to this, when running the 'install' or 'build' Makefile target,
the version of holos being built was not shown even though the 'build'
target attempted to show the version.
```
.PHONY: build
build: generate ## Build holos executable.
@echo "building ${BIN_NAME} ${VERSION}"
```
For example:
```
> make install
go generate ./...
building holos
...
```
Holo's version is stored in pkg/version/embedded/{major,minor,patch},
not the `Version` const. So the fix is to change the value of `VERSION`
so that it comes from those embedded files.
Now the version of holos is shown:
```
> make install
go generate ./...
building holos 0.61.1
...
```
This also adds a new Makefile target called `show-version` which shows
the full version string (i.e. the value of `$VERSION`).
