54 Commits

Author SHA1 Message Date
Jeff McCune
6fb6afe8d5 v0.97.3 2024-10-31 21:04:22 -07:00
Jeff McCune
490f91f580 cli: hide unsupported commands (#289)
Use a simple feature flag system that checks env vars if a feature is
enabled.
2024-10-31 10:04:01 -07:00
Jeff McCune
c6a13059f3 v0.97.1 2024-10-28 11:12:55 -07:00
Jeff McCune
bd56f3118c v0.96.0 - v1alpha4 Core API
Release v0.96.0 with the new Core API design.
2024-10-10 13:55:06 -07:00
Jeff McCune
6a47edbc3d guides: Add Change Service Guide (#253)
This patch also adds Organization to the Author API as an example of a
regular expression constraint.
2024-09-23 19:54:46 -07:00
Jeff McCune
6ad24a6eec package: rename schema api to author api (#248)
Schema API is unclear, Author API is more clear the API is intended for
component authors.
2024-09-19 08:51:01 -07:00
Jeff McCune
31d1086345 render: shorten the component rendered in the output
It's too long for documentation.  Shorten it for clarity.

Result:

```
❯ holos render platform ./platform
rendered bank-accounts-db for cluster workload in 160.7245ms
rendered bank-ledger-db for cluster workload in 162.465625ms
rendered bank-userservice for cluster workload in 166.150417ms
rendered bank-ledger-writer for cluster workload in 168.075459ms
rendered bank-balance-reader for cluster workload in 172.492292ms
rendered bank-backend-config for cluster workload in 198.117916ms
rendered bank-secrets for cluster workload in 223.200042ms
rendered gateway for cluster workload in 124.841917ms
rendered httproutes for cluster workload in 131.86625ms
rendered bank-contacts for cluster workload in 154.463792ms
rendered bank-transaction-history for cluster workload in 159.968208ms
rendered bank-frontend for cluster workload in 325.24425ms
rendered app-projects for cluster workload in 110.577916ms
rendered ztunnel for cluster workload in 137.502792ms
rendered cni for cluster workload in 209.993375ms
rendered cert-manager for cluster workload in 172.933834ms
rendered external-secrets for cluster workload in 135.759792ms
rendered local-ca for cluster workload in 98.026708ms
rendered istiod for cluster workload in 403.050833ms
rendered argocd for cluster workload in 294.663167ms
rendered gateway-api for cluster workload in 228.47875ms
rendered namespaces for cluster workload in 113.586916ms
rendered base for cluster workload in 533.76675ms
rendered external-secrets-crds for cluster workload in 529.053375ms
rendered crds for cluster workload in 931.180458ms
rendered platform in 1.248310167s
```

Previously:

```
❯ holos render platform ./platform
rendered projects/bank-of-holos/backend/components/bank-ledger-db for cluster workload in 158.534875ms
rendered projects/bank-of-holos/backend/components/bank-accounts-db for cluster workload in 159.836166ms
rendered projects/bank-of-holos/backend/components/bank-userservice for cluster workload in 160.360667ms
rendered projects/bank-of-holos/backend/components/bank-balance-reader for cluster workload in 169.478584ms
rendered projects/bank-of-holos/backend/components/bank-ledger-writer for cluster workload in 169.437833ms
rendered projects/bank-of-holos/backend/components/bank-backend-config for cluster workload in 182.089333ms
rendered projects/bank-of-holos/security/components/bank-secrets for cluster workload in 196.502792ms
rendered projects/platform/components/istio/gateway for cluster workload in 122.273083ms
rendered projects/bank-of-holos/frontend/components/bank-frontend for cluster workload in 307.573584ms
rendered projects/platform/components/httproutes for cluster workload in 149.631583ms
rendered projects/bank-of-holos/backend/components/bank-contacts for cluster workload in 153.529708ms
rendered projects/bank-of-holos/backend/components/bank-transaction-history for cluster workload in 165.375667ms
rendered projects/platform/components/app-projects for cluster workload in 107.253958ms
rendered projects/platform/components/istio/ztunnel for cluster workload in 137.22275ms
rendered projects/platform/components/istio/cni for cluster workload in 233.980958ms
rendered projects/platform/components/cert-manager for cluster workload in 171.966958ms
rendered projects/platform/components/external-secrets for cluster workload in 134.207792ms
rendered projects/platform/components/istio/istiod for cluster workload in 403.19ms
rendered projects/platform/components/local-ca for cluster workload in 97.544708ms
rendered projects/platform/components/argocd/argocd for cluster workload in 289.577208ms
rendered projects/platform/components/gateway-api for cluster workload in 218.290458ms
rendered projects/platform/components/namespaces for cluster workload in 109.534125ms
rendered projects/platform/components/istio/base for cluster workload in 526.32525ms
rendered projects/platform/components/external-secrets-crds for cluster workload in 523.7495ms
rendered projects/platform/components/argocd/crds for cluster workload in 1.002546375s
rendered platform in 1.312824333s
```
2024-09-17 11:40:14 -07:00
Jeff McCune
8a7a010b94 version 0.94.0 2024-09-15 15:41:17 -07:00
Jeff McCune
6f39cc6fdc docs: add istio section to expose-a-service
This patch adds Istio to the Expose a Service documentation and
introduces new concepts.  The Kubernetes build plan schema, the
namespaces component, and an example of how to safely re-use Helm values
from the root to multiple leaf components.

fix: istio cni not ready on k3d
---

The istio-k3d component embedded into holos fixes the cni pod not
becoming ready with our k3d local cluster guide.  The pod log error this
fixes is:

    configuration requires updates, (re)writing CNI config file at "": no networks found in /host/etc/cni/net.d
    Istio CNI is configured as chained plugin, but cannot find existing CNI network config: no networks found in /host/etc/cni/net.d
    Waiting for CNI network config file to be written in /host/etc/cni/net.d...

[Platform k3d]: https://istio.io/latest/docs/ambient/install/platform-prerequisites/#k3d

docs: clarify how to reset the local cluster
---

This is something we do all the time while developing and documenting,
so make it easy and fast to reset the cluster to a known good state.
2024-09-12 10:36:56 -07:00
Jeff McCune
c9d892eee3 generate: consolidate holos generate component cue/helm
Previously helm and cue components were split into two different
subcommands off the holos generate component command.  This is
unnecessary, I'm not sure why it was there in the first place.  The code
seemed perfectly duplicated.

This patch combines them to focus on the concept of a Component.  It
doesn't matter what kind it is now that it's expected to be run from the
root of the platform repository and drop configuration at the root and
the leaf of the tree.
2024-09-11 11:12:53 -07:00
Jeff McCune
4127804092 quickstart: v0.93.2 with schema.#Platform
Make sure go install works from the quickstart documentation by doing a
release.  Otherwise, v0.93.1 is installed which doesn't include the
platform schema.
2024-09-09 17:04:32 -07:00
Jeff McCune
5e582ec5c6 generate: do not require registration when generating a platform
Without this patch the `holos generate platform` command automatically
makes an rpc call to holos server.  This creates friction for the
quickstart guide because we don't need to require users to register and
have an organization and platform already created in the server just to
generate a simple platform to exercise a simple helm chart component.

A future patch should implement the behavior of linking a server side
platform to a local git repository by making the API call to get the
platform ID then updating the platform.metadata.json file.
2024-09-06 11:27:05 -07:00
Jeff McCune
3c1fcd9d6e cli: remove unused subcommands (#223)
This patch cleans up the cli commands, improves the short, long, and use
help strings, and makes some other minor changes for publishing the
code.
2024-07-29 14:41:59 -07:00
Jeff McCune
a0fd53deaa builder: fix cue panic (#212)
Previously CUE paniced when holos tried to unify values originating from
two different cue runtimes.  This patch fixes the problem by
initializaing cue.Value structs from the same cue context.

Log messages are also improved after making one complete pass through
the Try Holos Locally guide.
2024-07-22 10:14:32 -07:00
Jeff McCune
b2ce455aa2 generate/platform: use platform metadata as source of truth (#200)
This patch addresses Nate's feedback that it's difficult to know what
platform is being operated on.

Previously it wasn't clear where the platform id used for push and pull
comes from.  The source of truth is the platform.metadata.json file
created when the platform is first generated using `holos generate
platform k3d`.

This patch removes the platformId field from the platform.config.json
file, renames the platform.config.json file to platform.model.json and
renames the internal symbols to match the domain language of "Platform
Model" instead of the less clear "config"

This patch also changes the API between holos and CUE to use the proto
json imported from the proto file instead of generated from the go code
generated from the proto file.  The purpose is to ensure protojson
encoding is used end to end.

Default log handler:

The patch also changes the default log output to print only the message
to stderr.  This addresses similar feedback from both Gary and Nate that
the output is skipped over because it feels like internal debug logs.

We still want 100% of output to go through the logger so we can ensure
each line can be made into valid json.  Info messages however are meant
for the user and all other attributes can be stripped off by default.
If additional source location is necessary, enable the text or json
output format.

Protobuf JSON:

This patch modifies the API contract between holos and CUE to ensure
data is exchanged exclusively using protojson.  This is necessary
because protobuf has a canonical json format which is not compatible
with the go json package struct tags.  When Holos handles a protobuf
message, it must marshal and unmarshal it using the protojson package.

Similarly, when importing protobuf messages into CUE, we must use `cue
import` instead of `cue go get` so that the canonical format is used
instead of the invalid go json struct tags.

Finally, when a Go struct like v1alpha1.Form is used to represent data
defined in cue which contains a nested protobuf message, Holos should
use a cue.Value to lookup the nested path, marshal it into json bytes,
then unmarshal it again using protojson.
2024-07-21 09:31:09 -07:00
Jeff McCune
327193215b version: 0.89.1 2024-07-18 14:42:24 -07:00
Jeff McCune
ddba69517f version 0.89.0 2024-07-18 12:00:49 -07:00
Jeff McCune
1b539b8874 cmd/holos: version 0.87.2 2024-07-06 11:15:28 -07:00
Jeff McCune
5e2c0e7d64 doc/website: use npm instead of yarn (#192)
The github workflows fail because yarn is not available.  The Angular
frontend app uses npm so we should also use npm for the website to
minimize dependencies.
2024-07-02 14:11:52 -07:00
Jeff McCune
af1c009dad doc/website: add holos website command to serve docusaurus (#84)
Previously docs are not published.  This patch adds Docusaurus into the
doc/website directory which is also a Go package to embed the static
site into the executable.

Serve the site using http.Server with a h2c handler with the command:

    holos website --log-format=json --log-drop=source

The website subcommand is intended to be run from a container as a
Deployment.  For expedience, the website subcommand doesn't use the
signals package like the server subcommand does. Consider using it for
graceful Deployment restarts.

Refer to https://github.com/ent/ent/tree/master/doc/website
2024-07-01 22:10:28 -07:00
Jeff McCune
53cb9ba7fb (#189) Make the v1alpha2 API data only
Previously a couple of methods were defined on the Result struct.

This patch moves the methods to an internal wrapper struct to remove
them from the API documentation.

With this patch the API between holos and CUE is entirely a data API.
2024-06-30 17:19:35 -07:00
Jeff McCune
33970dafe8 (#189) Version 0.85.0 v1alpha2 2024-06-30 10:27:48 -07:00
Jeff McCune
30ddde7b49 (maint) Add make image to make help
Previously it wasn't clear how to build the image, wasn't showing up in
make help.
2024-06-24 20:48:47 -07:00
Jeff McCune
5cced6fb51 Version 0.84.0 2024-06-24 20:40:00 -07:00
Jeff McCune
7845ce62e0 (#178) Update buf with make tools
Previously go releaser was failing because buf has been updated again.
2024-06-03 11:10:42 -07:00
Jeff McCune
7956475363 (#178) Add BuildPlan deployFiles field
Previously, each BuildPlan has no clear way to produce an ArgoCD
Application resource.  This patch provides a general solution where each
BuildPlan can provide arbitrary files as a map[string]string where the
key is the file path relative to the gitops repository `deploy/` folder.
2024-06-03 10:00:35 -07:00
Jeff McCune
d81729857b (#178) v0.81.2 for holos
Use v0.81.2 to build out the holos platform.  Once we have the
components structured fairly well we can circle back around and copy the
components to schematics.  There's a bit of friction regenerating the
platform from schematic each time.
2024-05-23 09:14:27 -07:00
Jeff McCune
9411a65dd8 (#178) Add namespaces component schematic
The first thing most platforms need to do is come up with a strategy for
managing namespaces across multiple clusters.

This patch defines #Namespaces in the holos platform and adds a
namespaces component which loops over all values in the #Namespaces
struct and manages a kubernetes Namespace object.

The platform resource itself loops over all clusters in the platform to
manage all namespaces across all clusters.

From a blank slate:

```
❯ holos generate platform holos
4:26PM INF platform.go:79 wrote platform.metadata.json version=0.82.0 platform_id=018fa1cf-a609-7463-aa6e-fa53bfded1dc path=/home/jeff/workspace/holos-run/holos-infra/saas/platform.metadata.json
4:26PM INF platform.go:91 generated platform holos version=0.82.0 platform_id=018fa1cf-a609-7463-aa6e-fa53bfded1dc path=/home/jeff/workspace/holos-run/holos-infra/saas

❯ holos pull platform config .
4:26PM INF pull.go:64 pulled platform model version=0.82.0 server=https://jeff.app.dev.k2.holos.run:443 platform_id=018fa1cf-a609-7463-aa6e-fa53bfded1dc
4:26PM INF pull.go:75 saved platform config version=0.82.0 server=https://jeff.app.dev.k2.holos.run:443 platform_id=018fa1cf-a609-7463-aa6e-fa53bfded1dc path=platform.config.json

❯ (cd components && holos generate component cue namespaces)
4:26PM INF component.go:147 generated component version=0.82.0 name=namespaces path=/home/jeff/workspace/holos-run/holos-infra/saas/components/namespaces

❯ holos render platform ./platform/
4:26PM INF platform.go:29 ok render component version=0.82.0 path=components/namespaces cluster=management num=1 total=2 duration=464.055541ms
4:26PM INF platform.go:29 ok render component version=0.82.0 path=components/namespaces cluster=aws1 num=2 total=2 duration=467.978499ms
```

The result:

```sh
cat deploy/clusters/management/components/namespaces/namespaces.gen.yaml
```

```yaml
---
metadata:
  name: holos
  labels:
    kubernetes.io/metadata.name: holos
kind: Namespace
apiVersion: v1
```
2024-05-22 16:32:59 -07:00
Jeff McCune
a02c7a4015 (#178) Fix the PlatformService CreatePlatform rpc
Without this patch the
holos.platform.v1alpha1.PlatformService.CreatePlatform doesn't work as
expected.  The Platform message is used which incorrectly requires a
client supplied id which is ignored by the server.

This patch allows the creation of a new platform by reusing the update
operation as a mutation that applies to both create and update.  Only
modifiable fields are part of the PlatformMutation message.
2024-05-22 12:39:24 -07:00
Jeff McCune
bba3895f35 (#175) Add holos generate component helm command
This patch adds a schematic to generate a holos component that wraps a
helm chart.  The cert-manager chart is the current example.

Usage:

```bash
set -euo pipefail

rm -rf ~/holos/dev/bare
mkdir ~/holos/dev/bare
cd ~/holos/dev/bare

holos generate platform bare
holos pull platform config .
holos render platform ./platform/
(cd components && holos generate component helm cert-manager)
```

The chart builds:

```bash
holos build ./components/cert-manager | yq .
```

And renders:

```bash
holos render component ./components/cert-manager --cluster-name k2
find deploy -type f
```

```txt
9:41PM INF render.go:83 rendered cert-manager version=0.81.1 cluster=k2 status=ok action=rendered name=cert-manager
deploy/clusters/k2/holos/components/cert-manager-kustomization.gen.yaml
deploy/clusters/k2/components/cert-manager/cert-manager.gen.yaml
```
2024-05-21 11:05:53 -07:00
Jeff McCune
9e60ddbe85 (#175) Add holos generate component cue command
This patch adds a command to generate CUE based holos components from
examples embedded in the executable.  The examples are passed through
the go template rendering engine with values pulled from flags.

Each directory in the embedded filesystem becomes a unique command for
nice tab completion.  The `--name` flag defaults to "example" and is the
resulting component name.

A follow up patch with more flags will set the stage for a Helm
component schematic.

```
holos generate component cue minimal
```

```txt
3:07PM INF component.go:91 generated component version=0.80.2 name=example path=/home/jeff/holos/dev/bare/components/example
```
2024-05-20 15:10:54 -07:00
Jeff McCune
34f2a52cb7 (#175) Add holos render platform command
Split holos render into component and platform.

This patch splits the previous `holos render` command into subcommands.
`holos render component ./path/to/component/` behaves as the previous
`holos render` command and renders an individual component.

The new `holos render platform ./path/to/platform/` subcommand makes
space to render the entire platform using the platform model pulled from
the PlatformService.

Starting with an empty directory:

```sh
holos register user
holos generate platform bare
holos pull platform config .
holos render platform ./platform/
```

```txt
10:01AM INF platform.go:29 ok render component version=0.80.2 path=components/configmap cluster=k1 num=1 total=1 duration=448.133038ms
```

The bare platform has a single component which refers to the platform
model pulled from the PlatformService:

```sh
cat deploy/clusters/mycluster/components/platform-configmap/platform-configmap.gen.yaml
```

```yaml
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: platform
  namespace: default
data:
  platform: |
    spec:
      model:
        cloud:
          providers:
            - cloudflare
        cloudflare:
          email: platform@openinfrastructure.co
        org:
          displayName: Open Infrastructure Services
          name: ois
```
2024-05-20 10:41:24 -07:00
Jeff McCune
3845871368 (#175) holos pull platform config
This patch adds a subcommand to pull the data necessary to construct a
PlatformConfig DTO.  The PlatformConfig message contains all of the
fields and values necessary to build a platform and the platform
components.  This is an alternative to holos passing multiple tags to
CUE.  The PlatformConfig is marshalled and passed once.

The platform config is also stored in the local filesystem in the root
directory of the platform.  This enables repeated local building and
rendering without making an rpc call.

The build / render pipeline is expected to cache the PlatformConfig once
at the start of the pipeline using the pull subcommand.
2024-05-19 08:27:21 -07:00
Jeff McCune
3049694a0a (#175) holos register user
This patch adds a `holos register user` command.  Given an authenticated
id token and no other record of the user in the database, the cli tool
use the API to:

 1. User is registered in `holos server`
 2. User is linked to one Holos Organization.
 3. Holos Organization has the `bare` platform.
 4. Holos Organization has the `reference` platform.
 5. Ensure `~/.holos/client-context.json` contains the user id and an
    org id.

The `holos.ClientContext` struct is intended as a light weight way to
save and load the current organization id to the file system for further
API calls.

The assumption is most users will have only one single org.  We can add
a more complicated config context system like kubectl uses if and when
we need it.
2024-05-16 10:51:40 -07:00
Jeff McCune
5860c5747b (#87) generate sub-command with embedded platform
This patch adds a generate subcommand that copies a platform embedded
into the executable to the local filesystem.  The purpose is to
accelerate initial setup with canned example platforms.

Two platforms are intended to start, one bare and one reference
platform.  The number of platforms embedded into holos should be kept
small (2-3) to limit our support burden.
2024-05-14 15:03:21 -07:00
Jeff McCune
ac2ff47a9c (#172) Wire Version Info in the UI
This patch adds the GetVersion rpc method to
holos.system.v1alpha1.SystemService and wires the version information up
to the Web UI.

This is a good example to crib from later regarding fetching and
refreshing data from the web ui using grpc and field masks.
2024-05-14 11:50:06 -07:00
Jeff McCune
9a2773c618 (#171) Refactor API to use FieldMasks
This patch refactors the API following the [API Best Practices][api]
documentation.  The UpdatePlatform method is modeled after a mutating
operation described [by Netflix][nflx] instead of using a REST resource
representation.  This makes it much easier to iterate over the fields
that need to be updated as the PlatformUpdateOperation is a flat data
structure while a Platform resource may have nested fields.  Nested
fields are more complicated and less clear to handle with a FieldMask.

This patch also adds a snapckbar message on save.  Previously, the save
button didn't give any indication of success or failure.  This patch
fixes the problem by adding a snackbar message that pop up at the bottom
of the screen nicely.

When the snackbar message is dismissed or times out the save button is
re-enabled.

[api]: https://protobuf.dev/programming-guides/api/
[nflx]: https://netflixtechblog.com/practical-api-design-at-netflix-part-2-protobuf-fieldmask-for-mutation-operations-2e75e1d230e4

Examples:

FieldMask for ListPlatforms

```
grpcurl -H "x-oidc-id-token: $(holos token)" -d @ ${HOLOS_SERVER##*/} holos.platform.v1alpha1.PlatformService.ListPlatforms <<EOF
{
  "org_id": "018f36fb-e3f7-7f7f-a1c5-c85fb735d215",
  "field_mask": { "paths": ["id","name"] }
}
EOF
```

```json
{
 "platforms": [
   {
     "id": "018f36fb-e3ff-7f7f-a5d1-7ca2bf499e94",
     "name": "bare"
   },
   {
     "id": "018f6b06-9e57-7223-91a9-784e145d998c",
     "name": "gary"
   },
   {
     "id": "018f6b06-9e53-7223-8ae1-1ad53d46b158",
     "name": "jeff"
   },
   {
     "id": "018f6b06-9e5b-7223-8b8b-ea62618e8200",
     "name": "nate"
   }
 ]
}
```

Closes: #171
2024-05-13 16:20:20 -07:00
Jeff McCune
bac7aec0ba (#167) Restructure the bare platform
This patch restructures the bare platform in preparation for a
`Platform` kind of output from CUE in addition to the existing
`BuildPlan` kind.

This patch establishes a pattern where our own CUE defined code goes
into the two CUE module paths:

1. `internal/platforms/cue.mod/gen/github.com/holos-run/holos/api/v1alpha1`
2. `internal/platforms/cue.mod/pkg/github.com/holos-run/holos/api/v1alpha1`
3. `internal/platforms/cue.mod/usr/github.com/holos-run/holos/api/v1alpha1`

The first path is automatically generated from Go structs.  The second
path is where we override and provide additional cue level integration.

The third path is reserved for the end user to further refine and
constrain our definitions.
2024-05-07 16:53:00 -07:00
Jeff McCune
47a5e237e0 (#162) Lint go, typescript, and proto3 files
This patch adds lint coverage for proto3 and typescript to keep our code
reasonably clean.  The go linter was already enabled.
2024-05-06 14:17:08 -07:00
Jeff McCune
1279e2351a (#162) Move Platform back to holos.v1alpha1
No need to have a separate package for the PlatformService and related
protobuf messages.
2024-05-06 13:47:37 -07:00
Jeff McCune
6894f45b6c (#165) Deploy Holos to Dev
This patch deploys holos to the dev environment on the k2 cluster.  It's
accessible at https://app.dev.k2.holos.run/ behind the auth proxy by
default.
2024-05-06 11:10:29 -07:00
Jeff McCune
79e8ab639a (#161) Fix the FormGroup & Refactor API
The way we were organizing fields into section broke Formly validation.
This patch fixes the problem by using the recommended approach of
[Nested Forms][1].

This patch also refactors the PlatformService API to clean it up.
GetForm / PutForm are separated from the Platform methods.  Similarly
GetModel / PutModel are separated out and are specific to get and put
the model data.

NOTE: I'm not sure we should have separated out the platform service
into it's own protobuf package.  Seems maybe unnecessary.

❯ grpcurl -H "x-oidc-id-token: $(holos token)" -d '{"platform_id":"018f36fb-e3ff-7f7f-a5d1-7ca2bf499e94"}' jeff.app.dev.k2.holos.run:443 holos.platform.v1alpha1.PlatformService.GetModel
{
  "model": {
    "org": {
      "contactEmail": "platform@openinfrastructure.co",
      "displayName": "Open Infrastructure Services LLC",
      "domain": "ois.run",
      "name": "ois"
    },
    "privacy": {
      "country": "earth",
      "regions": [
        "us-east-2",
        "us-west-2"
      ]
    },
    "terms": {
      "didAgree": true
    }
  }
}

[1]: https://formly.dev/docs/examples/other/nested-formly-forms
2024-05-04 10:14:37 -07:00
Jeff McCune
64a117b0c3 (#150) Add PlatformService.GetConfig and refactor ConfigValues proto
Problem:
The use of google.protobuf.Any was making it awkward to work with the
data provided by the user.  The structure of the form data is defined by
the platform engineer, so the intent of Any was to wrap the data in a
way we can pass over the network and persist in the database.

The escaped JSON encoding was problematic and error prone to decode on
the other end.

Solution:
Define the Platform values as a two level map with string keys, but with
protobuf message fields "sections" and "fields" respectively.  Use
google.protobuf.Value from the struct package to encode the actual
value.

Result:
In TypeScript, google.protobuf.Value encodes and decodes easily to a
JSON value.  On the go side, connect correctly handles the value as
well.

No more ugly error prone escaping:

```
❯ grpcurl -H "x-oidc-id-token: $(holos token)" -d '{"platform_id":"'${platformId}'"}' $host holos.v1alpha1.PlatformService.GetConfig
{
  "sections": {
    "org": {
      "fields": {
        "contactEmail": "jeff@openinfrastructure.co",
        "displayName": "Open Infrastructure Services LLC",
        "domain": "ois.run",
        "name": "ois"
      }
    }
  }
}
```

This return value is intended to be directly usable in the CUE code, so
we may further nest the values into a platform.spec key.
2024-05-01 21:30:30 -07:00
Jeff McCune
99f2763fdf (#150) Store Platform Config Form and Values as JSON
This patch changes the backend to store the platform config form
definition and the config values supplied by the form as JSON in the
database.

The gRPC API does not change with this patch, but may need to depending
on how this works and how easy it is to evolve the data model and add
features.
2024-05-01 09:11:53 -07:00
Jeff McCune
615f147bcb (#150) Add PutPlatformConfig to store the config values
This patch is a work in progress wiring up the form to put the values to
the holos server using grpc.

In an effort to simplify the platform configuration, the structure is a
two level map with the top level being configuration sections and the
second level being the fields associated with the config section.

To support multiple kinds of values and field controls, the values are
serialized to JSON for rpc over the network and for storage in the
database.  When they values are used, either by the UI or by the `holos
render` command, they're to be unmarshalled and in-lined into the
Platform Config data structure.

Pick back up ensuring the Platform rpc handler correctly encodes and
decodes the structure to the database.

Consider changing the config_form and config_values fields to JSON field
types in the database.  It will likely make working with this a lot
easier.

With this patch we're ready to wire up the holos render command to fetch
the platform configuration and create the end to end demo.

Here's essentially what the render command will fetch and lay down as a
json file for CUE:

```
❯ grpcurl -H "x-oidc-id-token: $(holos token)" -d '{"platform_id":"018f2c4e-ecde-7bcb-8b89-27a99e6cc7a1"}' jeff.app.dev.k2.holos.run:443 holos.v1alpha1.PlatformService.GetPlatform | jq .platform.config.values
{
  "sections": {
    "org": {
      "values": {
        "contactEmail": "\"platform@openinfrastructure.co\"",
        "displayName": "\"Open Infrastructure Services  LLC\"",
        "domain": "\"ois.run\"",
        "name": "\"ois\""
      }
    }
  }
}
```
2024-04-30 20:21:15 -07:00
Jeff McCune
d0ad3bfc69 (#150) Add Platform Detail to edit platform config
This patch adds a /platform/:id route path to a PlatformDetail
component.  The platform detail component calls the GetPlatform method
given the platform ID and renders the platform config form on the detail
tab.

The submit button is not yet wired up.

The API for adding platforms changes, allowing raw json bytes using the
RawConfig.  The raw bytes are not presented on the read path though,
calling GetPlatforms provides the platform and the config form inline in
the response.

Use the `raw_config` field instead of `config` when creating the form
data.

```
❯ grpcurl -H "x-oidc-id-token: $(holos token)" -d @ jeff.app.dev.k2.holos.run:443 holos.v1alpha1.PlatformService.AddPlatform <<EOF
{
  "platform": {
    "org_id": "018f27cd-e5ac-7f98-bfe1-2dbab208a48c",
    "name": "bare2",
    "raw_config": {
      "form": "$(cue export ./forms/platform/ --out json | jq -cM | base64 -w0)"
    }
  }
}
EOF
```
2024-04-30 14:02:49 -07:00
Jeff McCune
26e537e768 (#150) Add platform config form, values, cue
This patch adds 4 fields to the Platform table:

 1. Config Form represents the JSON FormlyFieldConfig for the UI.
 2. Config CUE represents the CUE file containing a definition the
    Config Values must unify with.
 3. Config Definition is the CUE definition variable name used to unify
    the values with the cue code.  Should be #PlatformSpec in most
    cases.
 4. Config Values represents the JSON values provided by the UI.

The use case is the platform engineer defines the #PlatformSpec in cue,
and provides the form field config.  The platform engineer then provides
1-3 above when adding or updating a Platform.

The UI then presents the form to the end user and provides values for 4
when the user submits the form.

This patch also refactors the AddPlatform method to accept a Platform
message.  To do so we make the id field optional since it is server
assigned.

The patch also adds a database constraint to ensure platform names are
unique within the scope of an organization.

Results:

Note how the CUE representation of the Platform Form is exported to JSON
then converted to a base64 encoded string, which is the protobuf JSON
representation of a bytes[] value.

```
grpcurl -H "x-oidc-id-token: $(holos token)" -d @ jeff.app.dev.k2.holos.run:443 holos.v1alpha1.PlatformService.AddPlatform <<EOF
{
  "platform": {
    "id": "0d3dc0c0-bbc8-41f8-8c6e-75f0476509d6",
    "org_id": "018f27cd-e5ac-7f98-bfe1-2dbab208a48c",
    "name": "bare",
    "config": {
      "form": "$(cd internal/platforms/bare && cue export ./forms/platform/ --out json | jq -cM | base64 -w0)"
    }
  }
}
EOF
```

Note the requested platform ID is ignored.

```
{
  "platforms": [
    {
      "id": "018f2af9-f7ba-772a-9db6-f985ece8fed1",
      "timestamps": {
        "createdAt": "2024-04-29T17:49:36.058379Z",
        "updatedAt": "2024-04-29T17:49:36.058379Z"
      },
      "name": "bare",
      "creator": {
        "id": "018f27cd-e591-7f98-a9d2-416167282d37"
      },
      "config": {
        "form": "eyJhcGlWZXJzaW9uIjoiZm9ybXMuaG9sb3MucnVuL3YxYWxwaGExIiwia2luZCI6IlBsYXRmb3JtRm9ybSIsIm1ldGFkYXRhIjp7Im5hbWUiOiJiYXJlIn0sInNwZWMiOnsic2VjdGlvbnMiOlt7Im5hbWUiOiJvcmciLCJkaXNwbGF5TmFtZSI6Ik9yZ2FuaXphdGlvbiIsImRlc2NyaXB0aW9uIjoiT3JnYW5pemF0aW9uIGNvbmZpZyB2YWx1ZXMgYXJlIHVzZWQgdG8gZGVyaXZlIG1vcmUgc3BlY2lmaWMgY29uZmlndXJhdGlvbiB2YWx1ZXMgdGhyb3VnaG91dCB0aGUgcGxhdGZvcm0uIiwiZmllbGRDb25maWdzIjpbeyJrZXkiOiJuYW1lIiwidHlwZSI6ImlucHV0IiwicHJvcHMiOnsibGFiZWwiOiJOYW1lIiwicGxhY2Vob2xkZXIiOiJleGFtcGxlIiwiZGVzY3JpcHRpb24iOiJETlMgbGFiZWwsIGUuZy4gJ2V4YW1wbGUnIiwicmVxdWlyZWQiOnRydWV9fSx7ImtleSI6ImRvbWFpbiIsInR5cGUiOiJpbnB1dCIsInByb3BzIjp7ImxhYmVsIjoiRG9tYWluIiwicGxhY2Vob2xkZXIiOiJleGFtcGxlLmNvbSIsImRlc2NyaXB0aW9uIjoiRE5TIGRvbWFpbiwgZS5nLiAnZXhhbXBsZS5jb20nIiwicmVxdWlyZWQiOnRydWV9fSx7ImtleSI6ImRpc3BsYXlOYW1lIiwidHlwZSI6ImlucHV0IiwicHJvcHMiOnsibGFiZWwiOiJEaXNwbGF5IE5hbWUiLCJwbGFjZWhvbGRlciI6IkV4YW1wbGUgT3JnYW5pemF0aW9uIiwiZGVzY3JpcHRpb24iOiJEaXNwbGF5IG5hbWUsIGUuZy4gJ0V4YW1wbGUgT3JnYW5pemF0aW9uJyIsInJlcXVpcmVkIjp0cnVlfX0seyJrZXkiOiJjb250YWN0RW1haWwiLCJ0eXBlIjoiaW5wdXQiLCJwcm9wcyI6eyJsYWJlbCI6IkNvbnRhY3QgRW1haWwiLCJwbGFjZWhvbGRlciI6InBsYXRmb3JtLXRlYW1AZXhhbXBsZS5jb20iLCJkZXNjcmlwdGlvbiI6IlRlY2huaWNhbCBjb250YWN0IGVtYWlsIGFkZHJlc3MiLCJyZXF1aXJlZCI6dHJ1ZX19XX1dfX0K"
      }
    }
  ]
}
```
2024-04-29 10:53:23 -07:00
Jeff McCune
ad70a6c4fe (#150) Add holos.v1alpha1.PlatformService.AddPlatform
This patch adds a basic AddPlatform method that adds a platform with a
name and a display name.

Next steps are to add fields for the Platform Config Form definition and
the Platform Config values submitted from the form.
2024-04-29 09:35:49 -07:00
Jeff McCune
22a04da6bb (#150) Add holos.v1alpha1.PlatformService.GetPlatforms
Next step: AddPlatform

Also consider extracting the queries to get the requested org_id to a
helper function.  This will likely eventually move to an interceptor
because every request is org scoped and needs authorization checks
against the org.

```
grpcurl -H "x-oidc-id-token: $(holos token)" -d '{"org_id":"018f27cd-e5ac-7f98-bfe1-2dbab208a48c"}' jeff.app.dev.k2.holos.run:443 holos.v1alpha1.PlatformService.GetPlatforms
```
2024-04-28 20:21:32 -07:00
Jeff McCune
28813eba5b (#126) v0.69.0 2024-04-23 10:24:58 -07:00