Compare commits

...

17 Commits

Author SHA1 Message Date
Jeff McCune
40ac705f0d (#16) Add create secret --append-hash=false
So we can easily create secrets for use with ExternalSecret resources.
2024-02-27 12:04:00 -08:00
Jeff McCune
b4ad6425e5 (#14) Validate SecretStore works
This patch validates a SecretStore in the holos-system namespace works
after provisioner credentials are refreshed.
2024-02-27 11:25:00 -08:00
Jeff McCune
3343d226e5 (#14) Fix namespaces "external-secrets" not found
Needed for the `prod-secrets-eso` component to reconcile with flux.

NAME                                    REVISION                SUSPENDED       READY   MESSAGE
flux-system                             main@sha1:28b9ab6b      False           True    Applied revision: main@sha1:28b9ab6b
prod-secrets-eso                        main@sha1:28b9ab6b      False           True    Applied revision: main@sha1:28b9ab6b
prod-secrets-eso-creds-refresher        main@sha1:28b9ab6b      False           True    Applied revision: main@sha1:28b9ab6b
prod-secrets-namespaces                 main@sha1:28b9ab6b      False           True    Applied revision: main@sha1:28b9ab6b
2024-02-26 20:53:43 -08:00
Jeff McCune
f3a9b7cfbc (#10) Additional test coverage for secrets
Also fix a bug, secrets were created with keys that have a sub-directory
which is not a valid kubernetes secret.
2024-02-26 16:58:38 -08:00
Jeff McCune
53b7246d5e (#10) Add tests for holos get secrets command
This patch adds basic test data to run integration level tests on the
holos cli command.  Tests are structured similar to how the go and cue
maintainers test their own cli tools using the testscripts package.

Fixture data is loaded into a fake kubernetes.Clientset.

The holos root command is executed without using a full sub-process so
the fake kubernetes interface persists across multiple holos commands in
the same test case.

The fake kubernetes interface is reset after the testcase script
concludes and a new one starts.

Take care to read and write absolute paths from the test scripts, the
current working directory of the test runner is not set to $WORK when
executing the custom holos command.
2024-02-26 16:16:27 -08:00
Jeff McCune
c20872c92f v0.45.1 2024-02-24 11:37:03 -08:00
Jeff McCune
ecce1f797e (#8) Get secret subcommand
This patch adds a get secret subcommand.  With no args, lists holos
secrets.  With args, gets each argument.

The use cases are:

 1. Extract specified keys to files with --to-file
 2. Extract all keys to files with --extract-all
 3. Print one key to stdout with --print-key

If no key is specified, the key is implicitly set to the holos secret
name.  This behavior should be preserved as part of the api.
2024-02-24 11:32:48 -08:00
Jeff McCune
0d7033d063 (#8) Create secret subcommand
This patch adds a holos create secret command that behaves like kubectl
create secret, but for the specific use case of provisioning holos
clusters.

```
❯ holos create secret k2-talos --cluster-name=k2 --from-file=secrets.yaml
4:48PM INF secret.go:104 created: k2-talos-49546d9fd7 version=0.45.0 secret=k2-talos-49546d9fd7 name=k2-talos namespace=secrets
```

Once the corresponding `holos get secret` subcommands are implemented
the kv subcommand may be removed.
2024-02-23 16:49:13 -08:00
Jeff McCune
84bf0c8945 (#6) Holos kv put command to create secrets
A "holos secret" is a Secret in the secrets namespace of the provisioner
cluster.  The put command creates a unique secret from files and
directories listed as arguments, or from a txtar archive provided on
standard input.

Secret data may come from any or all of the following sources:

1. Create a secret from raw data on standard input.  --name and --file
   must be specified.
2. Create a secret from txtar data on standard input.  The secret name
   is taken from the --name flag if provided, otherwise is taken from
   the first line of the txtar comment.
3. Create a secret from files and directories specified as arguments.
   The secret name is the base name of the first argument unless it is
   overridden by the --name flag.

This is likely doing too much, really all we care about is this use
case:

holos kv put talosconfig

holos kv get talosconfig | holos txtar

Additionally, I want to get get one command without writing a file:

DATA="$(holos kv get talosconfig --file talosconfig)
2024-02-23 12:03:47 -08:00
Jeff McCune
466b48966a (#3) holos kv list command
Simple list command that finds the unique holos.run/secret.name label
values and prints them out.

    holos kv list
    k2-flux-system
    k2-talos
    test
2024-02-22 22:06:23 -08:00
Jeff McCune
84bcf4b2d0 Handle write errors when creating an archive 2024-02-22 21:46:41 -08:00
Jeff McCune
bdd76c78a7 Refactor txtar package for readability 2024-02-22 21:42:07 -08:00
Jeff McCune
95e0dfa44a Refactor render cli to a package
Tidy up the structure of the cli package, keep subcommand related
functions grouped together in a package.
2024-02-22 21:20:51 -08:00
Jeff McCune
90d70a6afa Refactor build cli to a package
Tidy up the structure of the cli package, keep subcommand related
functions grouped together in a package.
2024-02-22 21:20:45 -08:00
Jeff McCune
d0c2d85246 (#3) Refactor txtar cli to a package
Tidy up the structure of the cli package, keep txtar related functions
grouped together in a package.
2024-02-22 21:13:40 -08:00
Jeff McCune
7e637b4647 (#3) Refactor kv command to kv package
The structure of the cli package was getting to be a bit of a mess, time
to clean it up.  The structure is much easier to read with each command
in a separate package of related functionality.
2024-02-22 21:09:45 -08:00
Jeff McCune
9bc96d0783 (#3) holos txtar command for provisioner secrets
This patch makes it easy to fetch one or multiple files from a Secret in
the provisioner cluster to address two primary use cases:

 1. Extract files into a temporary directory to provide to other tools.
 2. Print one file to stdout.

For example, the secrets.yaml file necessary to reset a talos cluster is
printed to stdout in txtar format with one command:

    holos kv get k2-talos

The output has the secret name as the comment, then the value of each key of the data
field is printed as the txtar name and data.

    k2-talos-49546d9fd7
    -- secrets.yaml --
    ...

Extracting all of the files in the secret is also simple:

    holos kv get k2-talos | holos txtar
    8:34PM INF txtar.go:94 writing: secrets.yaml version=0.43.0 header=k2-talos-49546d9fd7 path=secrets.yaml bytes=4841

Extracting one file to stdout is also simple:

    holos kv get k2-talos | holos txtar --index=1
2024-02-22 20:38:44 -08:00
47 changed files with 1424 additions and 238 deletions

View File

@@ -1,28 +1,10 @@
package main
import (
"context"
"errors"
"github.com/holos-run/holos/pkg/cli"
"github.com/holos-run/holos/pkg/config"
"github.com/holos-run/holos/pkg/wrapper"
"log/slog"
"os"
)
func main() {
cfg := config.New()
slog.SetDefault(cfg.Logger())
ctx := context.Background()
if err := cli.New(cfg).ExecuteContext(ctx); err != nil {
log := cfg.NewTopLevelLogger()
var errAt *wrapper.ErrorAt
const msg = "could not execute"
if ok := errors.As(err, &errAt); ok {
log.ErrorContext(ctx, msg, "err", errAt.Unwrap(), "loc", errAt.Source.Loc())
} else {
log.ErrorContext(ctx, msg, "err", err)
}
os.Exit(1)
}
os.Exit(cli.MakeMain()())
}

20
cmd/holos/main_test.go Normal file
View File

@@ -0,0 +1,20 @@
package main
import (
"github.com/holos-run/holos/pkg/cli"
"github.com/rogpeppe/go-internal/testscript"
"os"
"testing"
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"holos": cli.MakeMain(),
}))
}
func TestGetSecrets(t *testing.T) {
testscript.Run(t, testscript.Params{
Dir: "testdata",
})
}

5
cmd/holos/testdata/version.txt vendored Normal file
View File

@@ -0,0 +1,5 @@
exec holos --version
# want version with no v on stdout
stdout -count=1 '^\d+\.\d+\.\d+$'
# want nothing on stderr
! stderr .

View File

@@ -15,6 +15,10 @@ objects: #CredsRefresherService.objects
#TargetNamespace: #CredsRefresher.namespace
#Kustomization: spec: {
dependsOn: [{name: #InstancePrefix + "-namespaces"}]
}
let NAME = #CredsRefresher.name
let AUD = "//iam.googleapis.com/projects/\(#InputKeys.gcpProjectNumber)/locations/global/workloadIdentityPools/holos/providers/k8s-\(#InputKeys.cluster)"
let MOUNT = "/var/run/service-account"

View File

@@ -1,3 +1,30 @@
package holos
// Manages the External Secrets Operator from the official upstream Helm chart.
#TargetNamespace: "external-secrets"
#InputKeys: component: "eso"
#InputKeys: {
project: "secrets"
service: "eso"
}
#Kustomization: spec: {
dependsOn: [{name: #InstancePrefix + "-namespaces"}]
targetNamespace: #TargetNamespace
}
#HelmChart & {
values: installCrds: true
namespace: #TargetNamespace
chart: {
name: "external-secrets"
version: "0.9.12"
repository: {
name: "external-secrets"
url: "https://charts.external-secrets.io"
}
}
}

View File

@@ -1,8 +0,0 @@
package holos
#TargetNamespace: "external-secrets"
#InputKeys: {
project: "secrets"
service: "eso"
}

View File

@@ -1,16 +0,0 @@
package holos
#Kustomization: spec: dependsOn: [{name: #InstancePrefix + "-namespaces"}]
#HelmChart & {
values: installCrds: true
namespace: #TargetNamespace
chart: {
name: "external-secrets"
version: "0.9.12"
repository: {
name: "external-secrets"
url: "https://charts.external-secrets.io"
}
}
}

View File

@@ -1,11 +1,21 @@
package holos
// Validate ESO by syncing a secret with a SecretStore.
#TargetNamespace: "holos-system"
#InputKeys: {
project: "secrets"
component: "validate"
}
#Kustomization: spec: dependsOn: [{name: #InstancePrefix + "-eso"}]
objects: [
#SecretStore,
#ExternalSecret & {
_name: "validate"
metadata: namespace: #TargetNamespace
spec: dataFrom: [{extract: key: "ns/" + #TargetNamespace + "/test"}]
},
]

View File

@@ -1,6 +1,7 @@
package holos
#PlatformNamespaces: [
{name: "external-secrets"},
{name: "holos-system"},
{name: "flux-system"},
{name: "ceph-system"},

View File

@@ -1,8 +0,0 @@
package holos
#TargetNamespace: "default"
#InputKeys: {
project: "secrets"
component: "validate"
}

View File

@@ -79,6 +79,8 @@ _apiVersion: "holos.run/v1alpha1"
kind: string | *"GitRepository"
name: string | *"flux-system"
}
suspend?: bool
targetNamespace?: string
timeout: string | *"3m0s"
wait: bool | *true
}
@@ -88,7 +90,7 @@ _apiVersion: "holos.run/v1alpha1"
#ExternalSecret: #NamespaceObject & es.#ExternalSecret & {
_name: string
metadata: {
namespace: string | *"default"
namespace: string
name: _name
}
spec: {

19
go.mod
View File

@@ -6,6 +6,9 @@ require (
cuelang.org/go v0.7.0
github.com/mattn/go-isatty v0.0.20
github.com/spf13/cobra v1.7.0
golang.org/x/tools v0.18.0
k8s.io/apimachinery v0.29.2
k8s.io/client-go v0.29.2
)
require (
@@ -14,6 +17,7 @@ require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/emicklei/proto v1.10.0 // indirect
github.com/evanphx/json-patch v4.12.0+incompatible // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
@@ -21,7 +25,6 @@ require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/imdario/mergo v0.3.6 // indirect
@@ -36,27 +39,25 @@ require (
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc4 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/protocolbuffers/txtpbfmt v0.0.0-20230328191034-3462fbc510c0 // indirect
github.com/rogpeppe/go-internal v1.12.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/net v0.21.0 // indirect
golang.org/x/oauth2 v0.10.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/term v0.15.0 // indirect
golang.org/x/sys v0.17.0 // indirect
golang.org/x/term v0.17.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.16.1 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/api v0.29.2 // indirect
k8s.io/apimachinery v0.29.2 // indirect
k8s.io/client-go v0.29.2 // indirect
k8s.io/klog/v2 v2.110.1 // indirect
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 // indirect
k8s.io/kubectl v0.29.2 // indirect
k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect

39
go.sum
View File

@@ -13,6 +13,8 @@ github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxER
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/emicklei/proto v1.10.0 h1:pDGyFRVV5RvV+nkBK9iy3q67FBy9Xa7vwrOTE+g5aGw=
github.com/emicklei/proto v1.10.0/go.mod h1:rn1FgRS/FANiZdD2djyH7TMA9jdRDcYQ9IEN9yvjX0A=
github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84=
github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY=
github.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE=
@@ -23,6 +25,8 @@ github.com/go-openapi/swag v0.22.3 h1:yMBqmnQ0gyZvEb/+KzuWZOXgllrXT4SADYbvDaXHv/
github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/go-quicktest/qt v1.101.0 h1:O1K29Txy5P2OK0dGo59b7b0LR6wKfIhttaAhHUyn7eI=
github.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -38,6 +42,8 @@ github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeN
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
@@ -76,15 +82,24 @@ github.com/mpvl/unique v0.0.0-20150818121801-cbe035fff7de h1:D5x39vF5KCwKQaw+OC9
github.com/mpvl/unique v0.0.0-20150818121801-cbe035fff7de/go.mod h1:kJun4WP5gFuHZgRjZUWWuH1DTxCtxbHDOIJsudS8jzY=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.13.0 h1:0jY9lJquiL8fcf3M4LAXN5aMlS/b2BV86HFFPCPMgE4=
github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o=
github.com/onsi/gomega v1.29.0 h1:KIA/t2t5UBzoirT4H9tsML45GEbo3ouUnBHsCfD2tVg=
github.com/onsi/gomega v1.29.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0-rc4 h1:oOxKUJWnFC4YGHCCMNql1x4YaDfYBTS5Y4x/Cgeo1E0=
github.com/opencontainers/image-spec v1.1.0-rc4/go.mod h1:X4pATf0uXsnn3g5aiGIsVnJBR4mxhKzfwmvK/B2NTm8=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/protocolbuffers/txtpbfmt v0.0.0-20230328191034-3462fbc510c0 h1:sadMIsgmHpEOGbUs6VtHBXRR1OHevnj7hLx9ZcdNGW4=
github.com/protocolbuffers/txtpbfmt v0.0.0-20230328191034-3462fbc510c0/go.mod h1:jgxiZysxFPM+iWKwQwPR+y+Jvo54ARd4EisXxKYpB5c=
github.com/rogpeppe/go-internal v1.11.1-0.20231026093722-fa6a31e0812c h1:fPpdjePK1atuOg28PXfNSqgwf9I/qD1Hlo39JFwKBXk=
github.com/rogpeppe/go-internal v1.11.1-0.20231026093722-fa6a31e0812c/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
@@ -97,6 +112,8 @@ github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UV
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
@@ -106,15 +123,15 @@ golang.org/x/exp v0.0.0-20231006140011-7918f672742d h1:jtJma62tbqLibJ5sFQz8bKtEM
golang.org/x/exp v0.0.0-20231006140011-7918f672742d/go.mod h1:ldy0pHrwJyGW56pPQzzkH36rKxoZW1tw7ZJpeKx+hdo=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0=
golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.15.0 h1:SernR4v+D55NyBH2QiEQrlBAnj1ECL6AGrA5+dPaMY8=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
golang.org/x/net v0.21.0 h1:AQyQV4dYCvJ7vGmJyKki9+PBdyvhkSd8EIx/qb0AYv4=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/oauth2 v0.10.0 h1:zHCpF2Khkwy4mMB4bv0U37YtJdTGW8jI0glAApi0Kh8=
golang.org/x/oauth2 v0.10.0/go.mod h1:kTpgurOux7LqtuxjuyZa4Gj2gdezIt/jQtGnNFfypQI=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -124,10 +141,10 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.15.0 h1:y/Oo/a/q3IXu26lQgl04j/gjuBDOBlx7X6Om1j2CPW4=
golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.17.0 h1:mkTF7LCd6WGJNL3K1Ad7kwxNfYAW6a8a8QqtMblp/4U=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
@@ -139,8 +156,8 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.16.1 h1:TLyB3WofjdOEepBHAU20JdNC1Zbg87elYofWYAY5oZA=
golang.org/x/tools v0.16.1/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0=
golang.org/x/tools v0.18.0 h1:k8NLag8AGHnn+PHbl7g43CtqZAwG60vZkLqgyZgIHgQ=
golang.org/x/tools v0.18.0/go.mod h1:GL7B4CwcLLeo59yx/9UWWuNOW1n3VZ4f5axWfML7Lcg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -172,6 +189,8 @@ k8s.io/klog/v2 v2.110.1 h1:U/Af64HJf7FcwMcXyKm2RPM22WZzyR7OSpYj5tg3cL0=
k8s.io/klog/v2 v2.110.1/go.mod h1:YGtd1984u+GgbuZ7e08/yBuAfKLSO0+uR1Fhi6ExXjo=
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 h1:aVUu9fTY98ivBPKR9Y5w/AuzbMm96cd3YHRTU83I780=
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00/go.mod h1:AsvuZPBlUDVuCdzJ87iajxtXuR9oktsTctW/R9wwouA=
k8s.io/kubectl v0.29.2 h1:uaDYaBhumvkwz0S2XHt36fK0v5IdNgL7HyUniwb2IUo=
k8s.io/kubectl v0.29.2/go.mod h1:BhizuYBGcKaHWyq+G7txGw2fXg576QbPrrnQdQDZgqI=
k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI=
k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=

View File

@@ -1,8 +1,9 @@
package cli
package build
import (
"fmt"
"github.com/holos-run/holos/pkg/config"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/internal/builder"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
@@ -10,7 +11,7 @@ import (
)
// makeBuildRunFunc returns the internal implementation of the build cli command
func makeBuildRunFunc(cfg *config.Config) runFunc {
func makeBuildRunFunc(cfg *holos.Config) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
build := builder.New(builder.Entrypoints(args), builder.Cluster(cfg.ClusterName()))
results, err := build.Run(cmd.Context())
@@ -29,9 +30,9 @@ func makeBuildRunFunc(cfg *config.Config) runFunc {
}
}
// newBuildCmd returns the build subcommand for the root command
func newBuildCmd(cfg *config.Config) *cobra.Command {
cmd := newCmd("build [directory...]")
// New returns the build subcommand for the root command
func New(cfg *holos.Config) *cobra.Command {
cmd := command.New("build [directory...]")
cmd.Args = cobra.MinimumNArgs(1)
cmd.Short = "build kubernetes api objects from a directory"
cmd.RunE = makeBuildRunFunc(cfg)

37
pkg/cli/command/cmd.go Normal file
View File

@@ -0,0 +1,37 @@
package command
import (
"fmt"
"github.com/holos-run/holos/pkg/version"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
)
// RunFunc is a cobra.Command RunE function.
type RunFunc func(c *cobra.Command, args []string) error
// New returns a new subcommand
func New(name string) *cobra.Command {
cmd := &cobra.Command{
Use: name,
Version: version.Version,
Args: cobra.NoArgs,
CompletionOptions: cobra.CompletionOptions{
HiddenDefaultCmd: true,
},
RunE: func(c *cobra.Command, args []string) error {
return wrapper.Wrap(fmt.Errorf("could not run %v: not implemented", c.Name()))
},
SilenceUsage: true,
SilenceErrors: true,
}
return cmd
}
// EnsureNewline adds a trailing newline if not already there.
func EnsureNewline(b []byte) []byte {
if len(b) > 0 && b[len(b)-1] != '\n' {
b = append(b, '\n')
}
return b
}

23
pkg/cli/create/create.go Normal file
View File

@@ -0,0 +1,23 @@
package create
import (
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/cli/secret"
"github.com/holos-run/holos/pkg/holos"
"github.com/spf13/cobra"
)
// New returns the create command for the cli
func New(hc *holos.Config) *cobra.Command {
cmd := command.New("create")
cmd.Short = "create resources"
cmd.Flags().SortFlags = false
cmd.RunE = func(c *cobra.Command, args []string) error {
return c.Usage()
}
// flags
cmd.PersistentFlags().SortFlags = false
// commands
cmd.AddCommand(secret.NewCreateCmd(hc))
return cmd
}

23
pkg/cli/get/get.go Normal file
View File

@@ -0,0 +1,23 @@
package get
import (
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/cli/secret"
"github.com/holos-run/holos/pkg/holos"
"github.com/spf13/cobra"
)
// New returns the get command for the cli.
func New(hc *holos.Config) *cobra.Command {
cmd := command.New("get")
cmd.Short = "get resources"
cmd.Flags().SortFlags = false
cmd.RunE = func(c *cobra.Command, args []string) error {
return c.Usage()
}
// flags
cmd.PersistentFlags().SortFlags = false
// commands
cmd.AddCommand(secret.NewGetCmd(hc))
return cmd
}

View File

@@ -1,97 +0,0 @@
package cli
import (
"github.com/holos-run/holos/pkg/config"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"sort"
)
const NameLabel = "holos.run/secret.name"
// newKVRootCmd returns the kv root command for the cli
func newKVRootCmd(cfg *config.Config) *cobra.Command {
cmd := newCmd("kv")
cmd.Short = "work with secrets in the provisioner cluster"
cmd.Flags().SortFlags = false
cmd.RunE = func(c *cobra.Command, args []string) error {
return c.Usage()
}
// flags
cmd.PersistentFlags().SortFlags = false
cmd.PersistentFlags().AddGoFlagSet(cfg.KVFlagSet())
// subcommands
cmd.AddCommand(newKVGetCmd(cfg))
return cmd
}
func newKVGetCmd(cfg *config.Config) *cobra.Command {
cmd := newCmd("get")
cmd.Args = cobra.MinimumNArgs(1)
cmd.Short = "print secret data in txtar format"
cmd.Flags().SortFlags = false
cmd.RunE = makeKVGetRunFunc(cfg)
return cmd
}
func makeKVGetRunFunc(cfg *config.Config) runFunc {
return func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
log := logger.FromContext(ctx)
kcfg, err := clientcmd.BuildConfigFromFlags("", cfg.KVKubeconfig())
if err != nil {
return wrapper.Wrap(err)
}
clientset, err := kubernetes.NewForConfig(kcfg)
if err != nil {
return wrapper.Wrap(err)
}
for _, name := range args {
nlog := log.With(NameLabel, name)
opts := metav1.ListOptions{
LabelSelector: NameLabel + "=" + name,
}
list, err := clientset.CoreV1().Secrets("secrets").List(ctx, opts)
if err != nil {
return wrapper.Wrap(err)
}
nlog.DebugContext(ctx, "results", "len", len(list.Items))
if len(list.Items) < 1 {
continue
}
sort.Slice(list.Items, func(i, j int) bool {
return list.Items[i].CreationTimestamp.Before(&list.Items[j].CreationTimestamp)
})
// most recent secret is the one we want.
secret := list.Items[len(list.Items)-1]
for k, v := range secret.Data {
nlog.DebugContext(ctx, "data", "name", secret.Name, "key", k, "len", len(v))
}
if len(secret.Data) > 0 {
cfg.Println(secret.Name)
}
for k, v := range secret.Data {
cfg.Printf("-- %s --\n", k)
cfg.Write(ensureNewline(v))
}
}
return nil
}
}
func ensureNewline(b []byte) []byte {
if len(b) > 0 && b[len(b)-1] != '\n' {
b = append(b, '\n')
}
return b
}

97
pkg/cli/kv/get.go Normal file
View File

@@ -0,0 +1,97 @@
package kv
import (
"flag"
"fmt"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/cli/secret"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sort"
)
type getConfig struct {
file *string
}
func newGetCmd(cfg *holos.Config) *cobra.Command {
cmd := command.New("get")
cmd.Args = cobra.MinimumNArgs(1)
cmd.Short = "print secret data in txtar format"
cf := getConfig{}
flagSet := flag.NewFlagSet("", flag.ContinueOnError)
cf.file = flagSet.String("file", "", "file to print to stdout")
cmd.Flags().SortFlags = false
cmd.Flags().AddGoFlagSet(cfg.ClusterFlagSet())
cmd.Flags().AddGoFlagSet(flagSet)
cmd.RunE = makeGetRunFunc(cfg, cf)
return cmd
}
func makeGetRunFunc(cfg *holos.Config, cf getConfig) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
log := logger.FromContext(ctx)
cs, err := newClientSet(cfg)
if err != nil {
return err
}
for _, name := range args {
nlog := log.With(secret.NameLabel, name)
opts := metav1.ListOptions{
LabelSelector: secret.NameLabel + "=" + name,
}
if name := cfg.ClusterName(); name != "" {
opts.LabelSelector += fmt.Sprintf(",%s=%s", secret.ClusterLabel, name)
}
list, err := cs.CoreV1().Secrets(cfg.KVNamespace()).List(ctx, opts)
if err != nil {
return wrapper.Wrap(err)
}
nlog.DebugContext(ctx, "results", "len", len(list.Items))
if len(list.Items) < 1 {
continue
}
sort.Slice(list.Items, func(i, j int) bool {
return list.Items[i].CreationTimestamp.Before(&list.Items[j].CreationTimestamp)
})
// most recent secret is the one we want.
secret := list.Items[len(list.Items)-1]
keys := make([]string, 0, len(secret.Data))
for k, v := range secret.Data {
keys = append(keys, k)
nlog.DebugContext(ctx, "data", "name", secret.Name, "key", k, "len", len(v))
}
// Print one file to stdout
if key := *cf.file; key != "" {
if data, found := secret.Data[key]; found {
cfg.Write(command.EnsureNewline(data))
return nil
}
return wrapper.Wrap(fmt.Errorf("not found: %s have %#v", key, keys))
}
if len(secret.Data) > 0 {
cfg.Println(secret.Name)
}
for k, v := range secret.Data {
cfg.Printf("-- %s --\n", k)
cfg.Write(command.EnsureNewline(v))
}
}
return nil
}
}

40
pkg/cli/kv/kv.go Normal file
View File

@@ -0,0 +1,40 @@
package kv
import (
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
)
// New returns the kv root command for the cli
func New(cfg *holos.Config) *cobra.Command {
cmd := command.New("kv")
cmd.Short = "work with secrets in the provisioner cluster"
cmd.Flags().SortFlags = false
cmd.RunE = func(c *cobra.Command, args []string) error {
return c.Usage()
}
// flags
cmd.PersistentFlags().SortFlags = false
cmd.PersistentFlags().AddGoFlagSet(cfg.KVFlagSet())
// subcommands
cmd.AddCommand(newGetCmd(cfg))
cmd.AddCommand(newListCmd(cfg))
cmd.AddCommand(newPutCmd(cfg))
return cmd
}
func newClientSet(cfg *holos.Config) (*kubernetes.Clientset, error) {
kcfg, err := clientcmd.BuildConfigFromFlags("", cfg.KVKubeconfig())
if err != nil {
return nil, wrapper.Wrap(err)
}
clientset, err := kubernetes.NewForConfig(kcfg)
if err != nil {
return nil, wrapper.Wrap(err)
}
return clientset, nil
}

46
pkg/cli/kv/list.go Normal file
View File

@@ -0,0 +1,46 @@
package kv
import (
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/cli/secret"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func newListCmd(cfg *holos.Config) *cobra.Command {
cmd := command.New("list")
cmd.Args = cobra.NoArgs
cmd.Short = "list secrets"
cmd.Flags().SortFlags = false
cmd.Flags().AddGoFlagSet(cfg.ClusterFlagSet())
cmd.RunE = makeListRunFunc(cfg)
return cmd
}
func makeListRunFunc(cfg *holos.Config) command.RunFunc {
return func(cmd *cobra.Command, _ []string) error {
ctx := cmd.Context()
cs, err := newClientSet(cfg)
if err != nil {
return err
}
selector := metav1.ListOptions{LabelSelector: secret.NameLabel}
secrets, err := cs.CoreV1().Secrets(cfg.KVNamespace()).List(ctx, selector)
if err != nil {
return wrapper.Wrap(err)
}
labels := make(map[string]bool)
for _, s := range secrets.Items {
if value, ok := s.Labels[secret.NameLabel]; ok {
labels[value] = true
}
}
for label := range labels {
cfg.Println(label)
}
return nil
}
}

200
pkg/cli/kv/put.go Normal file
View File

@@ -0,0 +1,200 @@
package kv
import (
"bytes"
"context"
"flag"
"fmt"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/cli/secret"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
"golang.org/x/tools/txtar"
"io"
"io/fs"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/kubectl/pkg/util/hash"
"os"
"path/filepath"
"sigs.k8s.io/yaml"
"strings"
)
type putConfig struct {
secretName *string
file *string
dryRun *bool
}
func newPutCmd(cfg *holos.Config) *cobra.Command {
cmd := command.New("put")
cmd.Args = cobra.MinimumNArgs(0)
cmd.Short = "put a secret from stdin or file args"
cmd.Flags().SortFlags = false
pcfg := putConfig{}
flagSet := flag.NewFlagSet("", flag.ContinueOnError)
pcfg.secretName = flagSet.String("name", "", "secret name to use instead of txtar comment")
pcfg.file = flagSet.String("file", "", "file name to use instead of txtar path")
pcfg.dryRun = flagSet.Bool("dry-run", false, "print to standard output instead of creating")
cmd.Flags().AddGoFlagSet(flagSet)
cmd.Flags().AddGoFlagSet(cfg.ClusterFlagSet())
cmd.RunE = makePutRunFunc(cfg, pcfg)
return cmd
}
func makePutRunFunc(cfg *holos.Config, pcfg putConfig) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
a := &txtar.Archive{}
// Add stdin to the archive.
if len(args) == 0 {
data, err := io.ReadAll(cfg.Stdin())
if err != nil {
return wrapper.Wrap(err)
}
if *pcfg.file != "" {
file := txtar.File{
Name: *pcfg.file,
Data: data,
}
a.Files = append(a.Files, file)
} else {
a = txtar.Parse(data)
}
}
// Do we have a secret name?
if *pcfg.secretName != "" {
a.Comment = []byte(*pcfg.secretName)
}
if len(a.Comment) == 0 {
// Use the first argument if not
if len(args) > 0 {
a.Comment = []byte(filepath.Base(args[0]))
} else {
err := fmt.Errorf("missing secret name from name, args, or txtar comment")
return wrapper.Wrap(err)
}
}
head, _, _ := bytes.Cut(a.Comment, []byte("\n"))
secretName := string(head)
// Add files from the filesystem to the archive
for _, name := range args {
if err := filepath.WalkDir(name, makeWalkFunc(a, name)); err != nil {
return wrapper.Wrap(err)
}
}
log := logger.FromContext(cmd.Context())
ctx := cmd.Context()
// Nothing to do?
if len(a.Files) == 0 {
log.WarnContext(ctx, "nothing to do")
return nil
}
// Create the secret.
secret, err := createSecret(ctx, cfg, pcfg, a, secretName)
if err != nil {
return wrapper.Wrap(err)
}
if *pcfg.dryRun {
data, err := yaml.Marshal(secret)
if err != nil {
return wrapper.Wrap(err)
}
cfg.Println(string(data))
return nil
}
// Make the API call
cs, err := newClientSet(cfg)
if err != nil {
return wrapper.Wrap(err)
}
secret, err = cs.CoreV1().Secrets(cfg.KVNamespace()).Create(ctx, secret, metav1.CreateOptions{})
if err != nil {
return wrapper.Wrap(err)
}
log.InfoContext(ctx, "created: "+secret.Name, "secret", secret.Name, "name", secretName, "namespace", secret.Namespace)
return nil
}
}
func createSecret(ctx context.Context, cfg *holos.Config, pcfg putConfig, a *txtar.Archive, secretName string) (*v1.Secret, error) {
secretData := make(map[string][]byte)
for _, file := range a.Files {
secretData[file.Name] = file.Data
}
labels := map[string]string{secret.NameLabel: secretName}
if owner := os.Getenv("USER"); owner != "" {
labels[secret.OwnerLabel] = owner
}
if cluster := cfg.ClusterName(); cluster != "" {
labels[secret.ClusterLabel] = cluster
}
secret := &v1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: secretName,
Labels: labels,
},
Data: secretData,
}
secretHash, err := hash.SecretHash(secret)
if err != nil {
return nil, wrapper.Wrap(err)
}
secret.Name = fmt.Sprintf("%s-%s", secret.Name, secretHash)
return secret, nil
}
func makeWalkFunc(a *txtar.Archive, rootDir string) fs.WalkDirFunc {
return func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
// Depth is the count of path separators from the root
depth := strings.Count(path[len(rootDir):], string(filepath.Separator))
if depth > 1 {
if d.IsDir() {
return filepath.SkipDir
}
}
if !d.IsDir() {
if file, err := file(path); err != nil {
return wrapper.Wrap(err)
} else {
file.Name = filepath.Base(path)
a.Files = append(a.Files, file)
}
}
return nil
}
}
func file(path string) (file txtar.File, err error) {
file.Name = path
file.Data, err = os.ReadFile(path)
return
}

35
pkg/cli/main.go Normal file
View File

@@ -0,0 +1,35 @@
package cli
import (
"context"
"errors"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/wrapper"
"log/slog"
)
// MakeMain makes a main function for the cli or tests.
func MakeMain(options ...holos.Option) func() int {
return func() (exitCode int) {
cfg := holos.New(options...)
slog.SetDefault(cfg.Logger())
ctx := context.Background()
if err := New(cfg).ExecuteContext(ctx); err != nil {
return handleError(ctx, err, cfg)
}
return 0
}
}
// handleError is the top level error handler that unwraps and logs errors.
func handleError(ctx context.Context, err error, hc *holos.Config) (exitCode int) {
log := hc.NewTopLevelLogger()
var errAt *wrapper.ErrorAt
const msg = "could not execute"
if ok := errors.As(err, &errAt); ok {
log.ErrorContext(ctx, msg, "err", errAt.Unwrap(), "loc", errAt.Source.Loc())
} else {
log.ErrorContext(ctx, msg, "err", err)
}
return 1
}

View File

@@ -1,15 +1,16 @@
package cli
package render
import (
"fmt"
"github.com/holos-run/holos/pkg/config"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/internal/builder"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
)
func makeRenderRunFunc(cfg *config.Config) runFunc {
func makeRenderRunFunc(cfg *holos.Config) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
if cfg.ClusterName() == "" {
return wrapper.Wrap(fmt.Errorf("missing cluster name"))
@@ -42,9 +43,9 @@ func makeRenderRunFunc(cfg *config.Config) runFunc {
}
}
// newRenderCmd returns the render subcommand for the root command
func newRenderCmd(cfg *config.Config) *cobra.Command {
cmd := newCmd("render [directory...]")
// New returns the render subcommand for the root command
func New(cfg *holos.Config) *cobra.Command {
cmd := command.New("render [directory...]")
cmd.Args = cobra.MinimumNArgs(1)
cmd.Short = "write kubernetes api objects to the filesystem"
cmd.Flags().SortFlags = false

View File

@@ -1,19 +1,21 @@
package cli
import (
"fmt"
"github.com/holos-run/holos/pkg/config"
"github.com/holos-run/holos/pkg/cli/build"
"github.com/holos-run/holos/pkg/cli/create"
"github.com/holos-run/holos/pkg/cli/get"
"github.com/holos-run/holos/pkg/cli/kv"
"github.com/holos-run/holos/pkg/cli/render"
"github.com/holos-run/holos/pkg/cli/txtar"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/version"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
"log/slog"
)
type runFunc func(c *cobra.Command, args []string) error
// New returns a new root *cobra.Command for command line execution.
func New(cfg *config.Config) *cobra.Command {
func New(cfg *holos.Config) *cobra.Command {
rootCmd := &cobra.Command{
Use: "holos",
Short: "holos manages a holistic integrated software development platform",
@@ -45,27 +47,16 @@ func New(cfg *config.Config) *cobra.Command {
rootCmd.PersistentFlags().AddGoFlagSet(cfg.LogFlagSet())
// subcommands
rootCmd.AddCommand(newBuildCmd(cfg))
rootCmd.AddCommand(newRenderCmd(cfg))
rootCmd.AddCommand(newKVRootCmd(cfg))
rootCmd.AddCommand(build.New(cfg))
rootCmd.AddCommand(render.New(cfg))
rootCmd.AddCommand(get.New(cfg))
rootCmd.AddCommand(create.New(cfg))
// Maybe not needed?
rootCmd.AddCommand(txtar.New(cfg))
// Deprecated, remove?
rootCmd.AddCommand(kv.New(cfg))
return rootCmd
}
// newCmd returns a new subcommand
func newCmd(name string) *cobra.Command {
cmd := &cobra.Command{
Use: name,
Version: version.Version,
Args: cobra.NoArgs,
CompletionOptions: cobra.CompletionOptions{
HiddenDefaultCmd: true,
},
RunE: func(c *cobra.Command, args []string) error {
return wrapper.Wrap(fmt.Errorf("could not run %v: not implemented", c.Name()))
},
SilenceUsage: true,
SilenceErrors: true,
}
return cmd
}

View File

@@ -2,7 +2,7 @@ package cli
import (
"bytes"
"github.com/holos-run/holos/pkg/config"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/version"
"github.com/spf13/cobra"
@@ -13,7 +13,7 @@ import (
func newCommand() (*cobra.Command, *bytes.Buffer) {
var b1, b2 bytes.Buffer
// discard stdout for now, it's a bunch of usage messages.
cmd := New(config.New(config.Stdout(&b1), config.Stderr(&b2)))
cmd := New(holos.New(holos.Stdout(&b1), holos.Stderr(&b2)))
return cmd, &b2
}
@@ -89,7 +89,7 @@ func TestInvalidArgs(t *testing.T) {
}
for _, args := range invalidArgs {
var b bytes.Buffer
cmd := New(config.New(config.Stdout(&b)))
cmd := New(holos.New(holos.Stdout(&b)))
cmd.SetArgs(args)
err := cmd.Execute()
if err == nil {
@@ -114,7 +114,7 @@ func TestLoggerFromContext(t *testing.T) {
func TestVersion(t *testing.T) {
var b bytes.Buffer
cmd := New(config.New(config.Stdout(&b)))
cmd := New(holos.New(holos.Stdout(&b)))
cmd.SetOut(&b)
cmd.SetArgs([]string{"--version"})
if err := cmd.Execute(); err != nil {

131
pkg/cli/secret/create.go Normal file
View File

@@ -0,0 +1,131 @@
package secret
import (
"fmt"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
"io/fs"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/kubectl/pkg/util/hash"
"os"
"path/filepath"
"sigs.k8s.io/yaml"
"strings"
)
func NewCreateCmd(hc *holos.Config) *cobra.Command {
cmd := command.New("secret NAME [--from-file=source]")
cmd.Aliases = []string{"secrets", "sec"}
cmd.Args = cobra.ExactArgs(1)
cmd.Short = "Create a holos secret from files or directories"
cfg, flagSet := newConfig()
flagSet.Var(&cfg.files, "from-file", "store files as keys in the secret")
cfg.dryRun = flagSet.Bool("dry-run", false, "dry run")
cfg.appendHash = flagSet.Bool("append-hash", true, "append hash to kubernetes secret name")
cmd.Flags().SortFlags = false
cmd.Flags().AddGoFlagSet(flagSet)
cmd.RunE = makeCreateRunFunc(hc, cfg)
return cmd
}
func makeCreateRunFunc(hc *holos.Config, cfg *config) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
log := logger.FromContext(ctx)
secretName := args[0]
secret := &v1.Secret{
TypeMeta: metav1.TypeMeta{
Kind: "Secret",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: secretName,
Labels: map[string]string{NameLabel: secretName},
},
Data: make(secretData),
}
if *cfg.cluster != "" {
clusterPrefix := fmt.Sprintf("%s-", *cfg.cluster)
if !strings.HasPrefix(secretName, clusterPrefix) {
const msg = "missing cluster name prefix"
log.WarnContext(ctx, msg, "have", secretName, "want", clusterPrefix)
}
}
for _, file := range cfg.files {
if err := filepath.WalkDir(file, makeWalkFunc(secret.Data, file)); err != nil {
return wrapper.Wrap(err)
}
}
if owner := os.Getenv("USER"); owner != "" {
secret.Labels[OwnerLabel] = owner
}
if *cfg.cluster != "" {
secret.Labels[ClusterLabel] = *cfg.cluster
}
if *cfg.appendHash {
if secretHash, err := hash.SecretHash(secret); err != nil {
return wrapper.Wrap(err)
} else {
secret.Name = fmt.Sprintf("%s-%s", secret.Name, secretHash)
}
}
if *cfg.dryRun {
out, err := yaml.Marshal(secret)
if err != nil {
return wrapper.Wrap(err)
}
hc.Write(out)
return nil
}
cs, err := hc.ProvisionerClientset()
if err != nil {
return wrapper.Wrap(err)
}
secret, err = cs.CoreV1().
Secrets(*cfg.namespace).
Create(ctx, secret, metav1.CreateOptions{})
if err != nil {
return wrapper.Wrap(err)
}
log.InfoContext(ctx, "created: "+secret.Name, "secret", secret.Name, "name", secretName, "namespace", secret.Namespace)
return nil
}
}
func makeWalkFunc(data secretData, root string) fs.WalkDirFunc {
return func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
// Depth is the count of path separators from the root
depth := strings.Count(path[len(root):], string(filepath.Separator))
if depth > 1 {
return filepath.SkipDir
}
if !d.IsDir() {
key := filepath.Base(path)
if data[key], err = os.ReadFile(path); err != nil {
return wrapper.Wrap(err)
}
}
return nil
}
}

147
pkg/cli/secret/get.go Normal file
View File

@@ -0,0 +1,147 @@
package secret
import (
"context"
"fmt"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"os"
"path/filepath"
"sort"
)
const printFlagName = "print-key"
func NewGetCmd(hc *holos.Config) *cobra.Command {
cmd := command.New("secrets NAME [--to-file=destination]")
cmd.Aliases = []string{"secret"}
cmd.Args = cobra.MinimumNArgs(0)
cmd.Short = "Get holos secrets from the provisioner cluster"
cfg, flagSet := newConfig()
flagSet.Var(&cfg.files, "to-file", "extract files from the secret")
cfg.printFile = flagSet.String(printFlagName, "", "print one key from the secret")
cfg.extract = flagSet.Bool("extract-all", false, "extract all files from the secret")
cfg.extractTo = flagSet.String("extract-to", ".", "extract to directory")
cmd.Flags().SortFlags = false
cmd.Flags().AddGoFlagSet(flagSet)
cmd.RunE = makeGetRunFunc(hc, cfg)
return cmd
}
func makeGetRunFunc(hc *holos.Config, cfg *config) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
namespace := *cfg.namespace
ctx := cmd.Context()
log := logger.FromContext(ctx).With("namespace", namespace)
cs, err := hc.ProvisionerClientset()
if err != nil {
return err
}
// List secrets if no arguments.
if len(args) == 0 {
return listSecrets(cmd.Context(), hc, namespace)
}
// Get each secret.
for _, secretName := range args {
log := log.With(NameLabel, secretName)
opts := metav1.ListOptions{
LabelSelector: fmt.Sprintf("%s=%s", NameLabel, secretName),
}
list, err := cs.CoreV1().Secrets(namespace).List(ctx, opts)
if err != nil {
return wrapper.Wrap(err)
}
log.DebugContext(ctx, "results", "len", len(list.Items))
if len(list.Items) < 1 {
continue
}
// Sort oldest first.
sort.Slice(list.Items, func(i, j int) bool {
return list.Items[i].CreationTimestamp.Before(&list.Items[j].CreationTimestamp)
})
// Get the most recent.
secret := list.Items[len(list.Items)-1]
log = log.With("secret", secret.Name)
// Extract the data keys (file names).
keys := make([]string, 0, len(secret.Data))
for k, v := range secret.Data {
keys = append(keys, k)
log.DebugContext(ctx, "data", "name", secret.Name, "key", k, "len", len(v))
}
// Extract specified files or all files.
toExtract := cfg.files
if *cfg.extract {
toExtract = keys
}
printFile := *cfg.printFile
if len(toExtract) == 0 {
if printFile == "" {
printFile = secretName
}
}
if printFile != "" {
if data, found := secret.Data[printFile]; found {
hc.Write(data)
} else {
err := fmt.Errorf("cannot print: want %s have %v: did you mean --extract-all or --%s=name", printFile, keys, printFlagName)
return wrapper.Wrap(err)
}
}
// Iterate over --to-file values.
for _, name := range toExtract {
data, found := secret.Data[name]
if !found {
err := fmt.Errorf("%s not found in %v", name, keys)
return wrapper.Wrap(err)
}
path := filepath.Join(*cfg.extractTo, name)
if err := os.WriteFile(path, data, 0666); err != nil {
return wrapper.Wrap(fmt.Errorf("could not write %s: %w", path, err))
}
log.InfoContext(ctx, "wrote: "+path, "name", name, "bytes", len(data))
}
}
return nil
}
}
// listSecrets lists holos secrets in the provisioner cluster
func listSecrets(ctx context.Context, hc *holos.Config, namespace string) error {
cs, err := hc.ProvisionerClientset()
if err != nil {
return err
}
selector := metav1.ListOptions{LabelSelector: NameLabel}
secrets, err := cs.CoreV1().Secrets(namespace).List(ctx, selector)
if err != nil {
return wrapper.Wrap(err)
}
secretNames := make(map[string]bool)
for _, secret := range secrets.Items {
if labelValue, ok := secret.Labels[NameLabel]; ok {
secretNames[labelValue] = true
}
}
for secretName := range secretNames {
hc.Println(secretName)
}
return nil
}

31
pkg/cli/secret/secret.go Normal file
View File

@@ -0,0 +1,31 @@
package secret
import (
"flag"
"github.com/holos-run/holos/pkg/holos"
)
const NameLabel = "holos.run/secret.name"
const OwnerLabel = "holos.run/owner.name"
const ClusterLabel = "holos.run/cluster.name"
type secretData map[string][]byte
type config struct {
files holos.StringSlice
printFile *string
extract *bool
dryRun *bool
appendHash *bool
cluster *string
namespace *string
extractTo *string
}
func newConfig() (*config, *flag.FlagSet) {
cfg := &config{}
flagSet := flag.NewFlagSet("", flag.ContinueOnError)
cfg.namespace = flagSet.String("namespace", holos.DefaultProvisionerNamespace, "namespace in the provisioner cluster")
cfg.cluster = flagSet.String("cluster-name", "", "cluster name selector")
return cfg, flagSet
}

View File

@@ -0,0 +1,81 @@
package secret_test
import (
"github.com/holos-run/holos/pkg/cli"
"github.com/holos-run/holos/pkg/holos"
"github.com/rogpeppe/go-internal/testscript"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/kubernetes/fake"
"testing"
"time"
)
const clientsetKey = "clientset"
var secret = v1.Secret{
TypeMeta: metav1.TypeMeta{
Kind: "Secret",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "k2-talos",
Namespace: "secrets",
Labels: map[string]string{
"holos.run/owner.name": "jeff",
"holos.run/secret.name": "k2-talos",
},
CreationTimestamp: metav1.Time{
Time: time.Date(2020, time.January, 1, 0, 0, 0, 0, time.UTC),
},
},
Data: map[string][]byte{
"secrets.yaml": []byte("content: secret\n"),
},
Type: "Opaque",
}
// cmdHolos executes the holos root command with a kubernetes.Interface that
// persists for the duration of the testscript. holos is NOT executed in a
// subprocess, the current working directory is not and should not be changed.
// Take care to read and write to $WORK in the test scripts using flags.
func cmdHolos(ts *testscript.TestScript, neg bool, args []string) {
clientset, ok := ts.Value(clientsetKey).(kubernetes.Interface)
if clientset == nil || !ok {
ts.Fatalf("missing kubernetes.Interface")
}
cfg := holos.New(
holos.ProvisionerClientset(clientset),
holos.Stdout(ts.Stdout()),
holos.Stderr(ts.Stderr()),
)
cmd := cli.New(cfg)
cmd.SetArgs(args)
err := cmd.Execute()
if neg {
if err == nil {
ts.Fatalf("want: error\nhave: %v", err)
} else {
ts.Logf("want: error\nhave: %v", err)
}
} else {
ts.Check(err)
}
}
func TestSecrets(t *testing.T) {
// Add TestWork: true to the Params to keep the $WORK directory around.
testscript.Run(t, testscript.Params{
Dir: "testdata",
Setup: func(env *testscript.Env) error {
env.Values[clientsetKey] = fake.NewSimpleClientset(&secret)
return nil
},
Cmds: map[string]func(ts *testscript.TestScript, neg bool, args []string){
"holos": cmdHolos,
},
})
}

View File

@@ -0,0 +1,21 @@
# Create the secret
holos create secret directory --from-file=$WORK/fixture --dry-run
# Want no warnings.
! stderr 'WRN'
# Want the data keys
stdout 'one.yaml: Y29udGVudDogb25lCg=='
stdout 'two.yaml: Y29udGVudDogdHdvCg=='
# Want the secret name label.
stdout 'holos.run/secret.name: directory'
# Want the TypeMeta
stdout 'kind: Secret'
stdout 'apiVersion: v1'
-- fixture/one.yaml --
content: one
-- fixture/two.yaml --
content: two

View File

@@ -0,0 +1,22 @@
# Create the secret
holos create secret directory --from-file=$WORK/want
stderr 'created: directory-..........'
stderr 'secret=directory-..........'
stderr 'name=directory'
stderr 'namespace=secrets'
! stderr 'WRN'
# Get the secret back
mkdir have
holos get secret directory --extract-all --extract-to=$WORK/have
stderr 'wrote: .*/have/one.yaml'
stderr 'wrote: .*/have/two.yaml'
# Compare the secrets
cmp want/one.yaml have/one.yaml
cmp want/two.yaml have/two.yaml
-- want/one.yaml --
content: one
-- want/two.yaml --
content: two

View File

@@ -0,0 +1,14 @@
# Create the secret.
holos create secret k3-talos --from-file $WORK/secrets.yaml
# Want info log attributes.
stderr 'created: k3-talos-..........'
stderr 'secret=k3-talos-..........'
stderr 'name=k3-talos'
stderr 'namespace=secrets'
# Want no warnings.
! stderr 'WRN'
-- secrets.yaml --
content: hello

View File

@@ -0,0 +1,14 @@
# Create the secret.
holos create secret k3-talos --namespace=jeff --from-file $WORK/secrets.yaml
stderr 'created: k3-talos-..........'
stderr 'secret=k3-talos-..........'
stderr 'name=k3-talos'
# Want specified namespace.
stderr 'namespace=jeff'
# Want no warnings.
! stderr 'WRN'
-- secrets.yaml --
content: hello

View File

@@ -0,0 +1,24 @@
# Create the secret
holos create secret directory --from-file=$WORK/want
# Get the secret back
mkdir have
holos get secret directory --extract-all --extract-to=$WORK/have
stderr 'wrote: .*/have/one.yaml'
stderr 'wrote: .*/have/two.yaml'
! stderr 'wrote: .*omit.yaml'
# Compare the secrets
cmp want/one.yaml have/one.yaml
cmp want/two.yaml have/two.yaml
# Want no files with depth > 1
! exists have/nope/omit.yaml
! exists have/omit.yaml
-- want/one.yaml --
content: one
-- want/two.yaml --
content: two
-- want/nope/omit.yaml --
content: not included

View File

@@ -0,0 +1,7 @@
# Want no hash appended
holos create secret test --namespace holos-system --from-file $WORK/test --append-hash=false
stderr ' created: test '
stderr ' secret=test '
-- test --
sekret

View File

@@ -0,0 +1,6 @@
# Want no hash appended
holos create secret test --namespace holos-system --from-file $WORK/test --append-hash=false --dry-run
stdout 'name: test$'
-- test --
sekret

View File

@@ -0,0 +1,11 @@
# Create the secret.
holos create secret k3-talos --cluster-name=k2 --from-file $WORK/secrets.yaml
stderr 'created: k3-talos-..........'
# Want a warning about the cluster name prefix.
stderr 'missing cluster name prefix'
stderr 'have=k3-talos'
stderr 'want=k2-'
-- secrets.yaml --
content: hello

View File

@@ -0,0 +1,10 @@
# Get and extract the secret
holos get secrets k2-talos --extract-all --extract-to=$WORK
! stdout .
stderr 'wrote: .*/secrets\.yaml'
# Check the secret keys
cmp want.secrets.yaml secrets.yaml
-- want.secrets.yaml --
content: secret

View File

@@ -0,0 +1,3 @@
holos get secrets k2-talos --print-key=secrets.yaml
stdout -count=1 '^content: secret$'
! stderr .

View File

@@ -0,0 +1,3 @@
holos get secrets
stdout '^k2-talos$'
! stderr .

95
pkg/cli/txtar/txtar.go Normal file
View File

@@ -0,0 +1,95 @@
package txtar
import (
"bytes"
"fmt"
"github.com/holos-run/holos/pkg/cli/command"
"github.com/holos-run/holos/pkg/holos"
"github.com/holos-run/holos/pkg/util"
"github.com/holos-run/holos/pkg/wrapper"
"github.com/spf13/cobra"
"golang.org/x/tools/txtar"
"io"
"log/slog"
"os"
"path/filepath"
)
// New returns a new txtar command.
func New(cfg *holos.Config) *cobra.Command {
cmd := command.New("txtar")
cmd.Short = "trivial text-based file archives"
cmd.Long = "writes arguments to stdout otherwise extracts"
cmd.Args = cobra.MinimumNArgs(0)
cmd.RunE = makeRunFunc(cfg)
cmd.Flags().SortFlags = false
cmd.Flags().AddGoFlagSet(cfg.TxtarFlagSet())
return cmd
}
func makeRunFunc(cfg *holos.Config) command.RunFunc {
return func(cmd *cobra.Command, args []string) error {
// extract an archive
if len(args) == 0 {
return extract(cfg)
}
// create an archive
a := &txtar.Archive{}
for _, name := range args {
if err := filepath.WalkDir(name, util.MakeWalkFunc(a)); err != nil {
return wrapper.Wrap(err)
}
}
if _, err := cfg.Stdout().Write(txtar.Format(a)); err != nil {
return wrapper.Wrap(err)
}
return nil
}
}
// extract files from the configured Stdin to Stdout or the filesystem.
func extract(cfg *holos.Config) error {
input, err := io.ReadAll(cfg.Stdin())
if err != nil {
return wrapper.Wrap(fmt.Errorf("could not read stdin: %w", err))
}
archive := txtar.Parse(input)
if idx := cfg.TxtarIndex(); idx != 0 {
return printFile(cfg.Stdout(), idx, archive)
}
return writeFiles(cfg.Logger(), archive)
}
// printFile prints one file from the txtar archive by index.
func printFile(w io.Writer, idx int, a *txtar.Archive) (err error) {
if idx == 0 {
return wrapper.Wrap(fmt.Errorf("idx cannot be 0"))
}
if idx > 0 {
_, err = w.Write(command.EnsureNewline(a.Files[idx-1].Data))
} else {
_, err = w.Write(command.EnsureNewline(a.Files[len(a.Files)+idx].Data))
}
return
}
// writeFiles writes all files in the archive.
func writeFiles(logger *slog.Logger, a *txtar.Archive) (err error) {
var header string
if h := bytes.Split(a.Comment, []byte{'\n'})[:1]; len(h) > 0 {
header = string(h[0])
}
for _, file := range a.Files {
log := logger.With("header", header, "path", file.Name, "bytes", len(file.Data))
path := filepath.Join(".", file.Name)
log.Info("writing: " + file.Name)
if err = os.MkdirAll(filepath.Dir(path), 0755); err != nil {
return wrapper.Wrap(fmt.Errorf("could not make directory: %w", err))
}
if err = os.WriteFile(path, file.Data, 0644); err != nil {
return wrapper.Wrap(fmt.Errorf("could not write file: %w", err))
}
}
return
}

View File

@@ -1,23 +1,36 @@
package config
package holos
import (
"flag"
"fmt"
"github.com/holos-run/holos/pkg/logger"
"github.com/holos-run/holos/pkg/wrapper"
"io"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
"log/slog"
"os"
"path/filepath"
)
const DefaultProvisionerNamespace = "secrets"
// An Option configures a Config using [functional
// options](https://commandcenter.blogspot.com/2014/01/self-referential-functions-and-design.html).
type Option func(o *options)
type options struct {
stdout io.Writer
stderr io.Writer
stdin io.Reader
stdout io.Writer
stderr io.Writer
provisionerClientset kubernetes.Interface
clientset kubernetes.Interface
}
// Stdin redirects standard input to r, useful for test capture.
func Stdin(r io.Reader) Option {
return func(o *options) { o.stdin = r }
}
// Stdout redirects standard output to w, useful for test capture.
@@ -30,9 +43,20 @@ func Stderr(w io.Writer) Option {
return func(o *options) { o.stderr = w }
}
// ProvisionerClientset sets the kubernetes Clientset, useful for test fake.
func ProvisionerClientset(clientset kubernetes.Interface) Option {
return func(o *options) { o.provisionerClientset = clientset }
}
// ClusterClientset sets the kubernetes Clientset, useful for test fake.
func ClusterClientset(clientset *kubernetes.Clientset) Option {
return func(o *options) { o.clientset = clientset }
}
// New returns a new top level cli Config.
func New(opts ...Option) *Config {
cfgOptions := &options{
stdin: os.Stdin,
stdout: os.Stdout,
stderr: os.Stderr,
}
@@ -42,14 +66,17 @@ func New(opts ...Option) *Config {
writeFlagSet := flag.NewFlagSet("", flag.ContinueOnError)
clusterFlagSet := flag.NewFlagSet("", flag.ContinueOnError)
kvFlagSet := flag.NewFlagSet("", flag.ContinueOnError)
txFlagSet := flag.NewFlagSet("", flag.ContinueOnError)
cfg := &Config{
logConfig: logger.NewConfig(),
writeTo: getenv("HOLOS_WRITE_TO", "deploy"),
clusterName: getenv("HOLOS_CLUSTER_NAME", ""),
writeFlagSet: writeFlagSet,
clusterFlagSet: clusterFlagSet,
options: cfgOptions,
kvFlagSet: kvFlagSet,
logConfig: logger.NewConfig(),
writeTo: getenv("HOLOS_WRITE_TO", "deploy"),
clusterName: getenv("HOLOS_CLUSTER_NAME", ""),
writeFlagSet: writeFlagSet,
clusterFlagSet: clusterFlagSet,
options: cfgOptions,
kvFlagSet: kvFlagSet,
txtarFlagSet: txFlagSet,
provisionerClientset: cfgOptions.provisionerClientset,
}
writeFlagSet.StringVar(&cfg.writeTo, "write-to", cfg.writeTo, "write to directory")
clusterFlagSet.StringVar(&cfg.clusterName, "cluster-name", cfg.clusterName, "cluster name")
@@ -57,8 +84,11 @@ func New(opts ...Option) *Config {
if home := homedir.HomeDir(); home != "" {
kvDefault = filepath.Join(home, ".holos", "kubeconfig.provisioner")
}
kvDefault = getenv("HOLOS_KUBECONFIG_PROVISIONER", kvDefault)
cfg.kvKubeconfig = kvFlagSet.String("kubeconfig-provisioner", kvDefault, "absolute path to the provisioner cluster kubeconfig file")
kvDefault = getenv("HOLOS_PROVISIONER_KUBECONFIG", kvDefault)
cfg.kvKubeconfig = kvFlagSet.String("provisioner-kubeconfig", kvDefault, "absolute path to the provisioner cluster kubeconfig file")
ns := getenv("HOLOS_PROVISIONER_NAMESPACE", DefaultProvisionerNamespace)
cfg.kvNamespace = kvFlagSet.String("provisioner-namespace", ns, "namespace in the provisioner cluster")
cfg.txtarIndex = txFlagSet.Int("index", 0, "file number to print if not 0")
return cfg
}
@@ -66,16 +96,20 @@ func New(opts ...Option) *Config {
// should be initialized early at a well known location in the program lifecycle
// then remain immutable.
type Config struct {
logConfig *logger.Config
writeTo string
clusterName string
logger *slog.Logger
options *options
finalized bool
writeFlagSet *flag.FlagSet
clusterFlagSet *flag.FlagSet
kvKubeconfig *string
kvFlagSet *flag.FlagSet
logConfig *logger.Config
writeTo string
clusterName string
logger *slog.Logger
options *options
finalized bool
writeFlagSet *flag.FlagSet
clusterFlagSet *flag.FlagSet
kvKubeconfig *string
kvNamespace *string
kvFlagSet *flag.FlagSet
txtarIndex *int
txtarFlagSet *flag.FlagSet
provisionerClientset kubernetes.Interface
}
// LogFlagSet returns the logging *flag.FlagSet for use by the command handler.
@@ -98,6 +132,11 @@ func (c *Config) KVFlagSet() *flag.FlagSet {
return c.kvFlagSet
}
// TxtarFlagSet returns the *flag.FlagSet for txtar related commands.
func (c *Config) TxtarFlagSet() *flag.FlagSet {
return c.txtarFlagSet
}
// Finalize validates the config and finalizes the startup lifecycle based on user configuration.
func (c *Config) Finalize() error {
if c.finalized {
@@ -132,9 +171,9 @@ func (c *Config) NewTopLevelLogger() *slog.Logger {
return c.logConfig.NewTopLevelLogger(c.options.stderr)
}
// Stderr should be used instead of os.Stderr to capture output for tests.
func (c *Config) Stderr() io.Writer {
return c.options.stderr
// Stdin should be used instead of os.Stdin to capture input from tests.
func (c *Config) Stdin() io.Reader {
return c.options.stdin
}
// Stdout should be used instead of os.Stdout to capture output for tests.
@@ -142,6 +181,11 @@ func (c *Config) Stdout() io.Writer {
return c.options.stdout
}
// Stderr should be used instead of os.Stderr to capture output for tests.
func (c *Config) Stderr() io.Writer {
return c.options.stderr
}
// WriteTo returns the write to path configured by flags.
func (c *Config) WriteTo() string {
return c.writeTo
@@ -181,6 +225,38 @@ func (c *Config) KVKubeconfig() string {
return *c.kvKubeconfig
}
// KVNamespace returns the configured namespace to operate against in the provisioner cluster.
func (c *Config) KVNamespace() string {
if c.kvNamespace == nil {
return DefaultProvisionerNamespace
}
return *c.kvNamespace
}
// TxtarIndex returns the
func (c *Config) TxtarIndex() int {
if c.txtarIndex == nil {
return 0
}
return *c.txtarIndex
}
// ProvisionerClientset returns a kubernetes client set for the provisioner cluster.
func (c *Config) ProvisionerClientset() (kubernetes.Interface, error) {
if c.provisionerClientset == nil {
kcfg, err := clientcmd.BuildConfigFromFlags("", c.KVKubeconfig())
if err != nil {
return nil, wrapper.Wrap(err)
}
clientset, err := kubernetes.NewForConfig(kcfg)
if err != nil {
return nil, wrapper.Wrap(err)
}
c.provisionerClientset = clientset
}
return c.provisionerClientset, nil
}
// getenv is equivalent to os.LookupEnv with a default value.
func getenv(key, defaultValue string) string {
if value, exists := os.LookupEnv(key); exists {

View File

@@ -1,4 +1,4 @@
package config
package holos
import (
"bytes"

22
pkg/holos/types.go Normal file
View File

@@ -0,0 +1,22 @@
package holos
import (
"fmt"
"strings"
)
// StringSlice represents zero or more flag values.
type StringSlice []string
// String implements the flag.Value interface.
func (i *StringSlice) String() string {
return fmt.Sprint(*i)
}
// Set implements the flag.Value interface.
func (i *StringSlice) Set(value string) error {
for _, str := range strings.Split(value, ",") {
*i = append(*i, str)
}
return nil
}

32
pkg/util/txtar.go Normal file
View File

@@ -0,0 +1,32 @@
package util
import (
"github.com/holos-run/holos/pkg/wrapper"
"golang.org/x/tools/txtar"
"io/fs"
"os"
)
func MakeWalkFunc(a *txtar.Archive) fs.WalkDirFunc {
return func(path string, d os.DirEntry, err error) error {
if err != nil {
return wrapper.Wrap(err)
}
if !d.IsDir() {
if file, err := file(path); err != nil {
return wrapper.Wrap(err)
} else {
a.Files = append(a.Files, file)
}
}
return nil
}
}
func file(path string) (file txtar.File, err error) {
file.Name = path
file.Data, err = os.ReadFile(path)
return
}

View File

@@ -1 +1 @@
43
46