Compare commits

...

27 Commits

Author SHA1 Message Date
Jeff McCune
a689c53a9c (#47) v0.62.1 - Projects v1alpha1 milestone complete 2024-04-03 15:32:34 -07:00
Jeff McCune
58cdda1d35 Merge pull request #100 from holos-run/jeff/47-iam-v2
(#47) Remove the prod-iam-zitadel namespace
2024-04-03 15:23:48 -07:00
Jeff McCune
bcb02b5c5c (#47) Remove the prod-iam-zitadel namespace
No longer needed, cluster has moved to prod-iam namespace.
2024-04-03 15:10:30 -07:00
Jeff McCune
0736c7de1a (#47) Bind ALL VirtualServices to the default gateway
Problem:
The VirtualService that catches auth routes for paths, e.g.
`/holos/authproxy/istio-ingress` is bound to the default gateway which
no longer exists because it has no hosts.

Solution:
It's unnecessary and complicated to create a Gateway for every project.
Instead, put all server entries into one `default` gateway and
consolidate the list using CUE.

Result:
It's easier to reason about this system.  There is only one ingress
gateway, `default` and everything gets added to it.  VirtualServices
need only bind to this gateway, which has a hosts entry appropriately
namespaced for the project.
2024-04-03 14:56:40 -07:00
Jeff McCune
28be9f9fbb (#47) Use the project specific Gateway
The login service is unavailable because the wrong gateway is used.
When using projects the VS needs to attach to the correct Gateway.
2024-04-03 12:59:48 -07:00
Jeff McCune
647681de38 (#99) Restore backups from prod-iam namespace
This patch configures the standby cluster to restore backups from the
prod-iam namespace instead of the prod-iam-zitadel namespace.
2024-04-03 12:30:12 -07:00
Jeff McCune
81beb5c539 (#47) Restore ZITADEL from existing backups
Problem:
The ZITADEL database isn't restoring into the prod-iam namespace after
moving from prod-iam-zitadel because no backup exists at the bucket
path.

Solution:
Hard-code the path to the old namespace to restore the database.  We'll
figure out how to move the backups to the new location in a follow up
change.
2024-04-03 11:44:16 -07:00
Jeff McCune
5c1e0a29c8 (#47) Have Ceph depend on secret stores
Another kustomization reconciling too early.
2024-04-03 11:22:15 -07:00
Jeff McCune
01ac5276a9 (#47) Have Gateway depend on secret stores
The `prod-platform-gateway` kustomization is reconciling early:

ExternalSecret/istio-ingress/argocd.ois.run dry-run failed: failed to
get API group resources: unable to retrieve the complete list of server
APIs: external-secrets.io/v1beta1: the server could not find the
requested resource
2024-04-03 11:20:15 -07:00
Jeff McCune
e40594ad8e (#47) Move ZITADEL to prod-iam project namespace
This patch moves ZITADEL from the prod-iam-zitadel namespace to the
projects managed prod-iam namespace, which is the prod environment of
the prod stage of the iam project.
2024-04-03 11:06:55 -07:00
Jeff McCune
bc9c6a622a (#97) Increase ZITADEL pgdata volume to 20Gi
Problem:

```
❯ k exec zitadel-pgha1-4npq-0 -it -- bash
Defaulted container "database" out of: database, replication-cert-copy, pgbackrest, pgbackrest-config, postgres-startup (init), nss-wrapper-init (init)
bash-4.4$ df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay         119G   51G   68G  43% /
tmpfs            64M     0   64M   0% /dev
/dev/rbd3       9.8G  9.8G     0 100% /pgdata
/dev/sda6       119G   51G   68G  43% /tmp
tmpfs            16G   24K   16G   1% /pgconf/tls
tmpfs            16G   24K   16G   1% /etc/database-containerinfo
tmpfs            16G   16K   16G   1% /etc/patroni
tmpfs            16G     0   16G   0% /dev/shm
tmpfs            16G   28K   16G   1% /etc/pgbackrest/conf.d
tmpfs            16G   12K   16G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           7.9G     0  7.9G   0% /proc/acpi
tmpfs           7.9G     0  7.9G   0% /proc/scsi
tmpfs           7.9G     0  7.9G   0% /sys/firmware
```
2024-04-03 10:09:49 -07:00
Jeff McCune
17f22199b7 (#86) ArgoCD - Disable Dex
Not needed
2024-04-02 15:47:22 -07:00
Jeff McCune
7e93fe4535 (#86) ArgoCD
Using the Helm chart so we can inject the istio sidecar with a kustomize
patch and tweak the configs for OIDC integration.

Login works, istio sidecar is injected.  ArgoCD can only be configured
with one domain unfortunately, it's not accessible at argocd.ois.run,
only argocd.k2.ois.run (or whatever cluster it's installed into).

Ideally it would use the Host header but it does not.

RBAC is not implemented but the User Info endpoint does have group
membership so this shouldn't be a problem to implement.
2024-04-02 15:33:47 -07:00
Jeff McCune
2e98df3572 (#86) ArgoCD in prod-platform project namespace
Deploys using the official release yaml.
2024-04-02 13:34:03 -07:00
Jeff McCune
3b561de413 (#93) Custom AuthPolicy rules for vault
This patch defines a #AuthPolicyRules struct which excludes hosts from
the blanket auth policy and includes them in specialized auth policies.
The purpose is to handle special cases like vault requests which have an
`X-Vault-Token` and `X-Vault-Request` header.

Vault does not use jwts so we cannot verify them in the mesh, have to
pass them along to the backend.

Closes: #93
2024-04-02 12:54:31 -07:00
Jeff McCune
0d0dae8742 (#89) Disable project auth proxies by default
Focus on the ingress gateway auth proxy for now and see how far it gets
us.
2024-04-01 21:48:08 -07:00
Jeff McCune
61b4b5bd17 (#89) Refactor auth proxy callbacks
The ingress gateway auth proxy callback conflicts with the project stage
auth proxy callback for the same backend Host: header value.

This patch disambiguates them by the namespace the auth proxy resides
in.
2024-04-01 21:37:52 -07:00
Jeff McCune
0060740b76 (#82) ingress gateway AuthorizationPolicy
This patch adds a `RequestAuthentication` and `AuthorizationPolicy` rule
to protect all requests flowing through the default ingress gateway.

Consider a browser request for httpbin.k2.example.com representing any
arbitrary host with a valid destination inside the service mesh.  The
default ingress gateway will check if there is already an
x-oidc-id-token header, and if so validate the token is issued by
ZITADEL and the aud value contains the ZITADEL project number.

If the header is not present, the request is forwarded to oauth2-proxy
in the istio-ingress namespace.  This auth proxy is configured to start
the oidc auth flow with a redirect back to /holos/oidc/callback of the
Host: value originally provided in the browser request.

Closes: #82
2024-04-01 20:37:34 -07:00
Jeff McCune
bf8a4af579 (#82) ingressgateway ExtAuthzHttp provider
This patch adds an ingress gateway extauthz provider.  Because ZITADEL
returns all applications associated with a ZITADEL project in the aud
claim, it makes sense to have one ingress auth proxy at the initial
ingress gateway so we can get the ID token in the request header for
backend namespaces to match using `RequestAuthentication` and
`AuthorizationPolicy`.

This change likely makes the additional per-stage auth proxies
unnecessary and over-engineered.  Backend namespaces will have access to
the ID token.
2024-04-01 16:53:11 -07:00
Jeff McCune
dc057fe39d (#89) Add platform project hosts for argocd, grafana, and prometheus
Certificates are issued by the provisioner and synced to the workload
clusters.
2024-04-01 13:09:46 -07:00
Jeff McCune
9877ab131a (#89) Platform Project
This patch manages a platform project to host platform level services
like ArgoCD, Kube Prom Stack, Kiali, etc...
2024-04-01 11:46:02 -07:00
Jeff McCune
13aba64cb7 (#66) Move CUSTOM AuthorizationPolicy to env namespace
It doesn't make sense to link the stage ext authz provider to the
ingress gateway because there can be only one provider per workload.

Link it instead to the backend environment and use the
`security.holos.run/authproxy` label to match the workload.
2024-03-31 18:56:14 -07:00
Jeff McCune
fe9bc2dbfc (#81) Istio 1.21.0 2024-03-31 12:51:56 -07:00
Jeff McCune
c53b682852 (#66) Use x-oidc-id-token instead of authorization header
Problem:
Backend services and web apps expect to place their own credentials into
the Authorization header.  oauth2-proxy writes over the authorization
header creating a conflict.

Solution:
Use the alpha configuration to place the id token into the
x-oidc-id-token header and configure the service mesh to authenticate
requests that have this header in place.

Note: ZITADEL does not use a JWT for an access token, unlike Keycloak
and Dex.  The access token is not compatible with a
RequestAuthentication jwt rule so we must use the id token.
2024-03-31 11:41:23 -07:00
Jeff McCune
3aca6a9e4c (#66) configure auth proxies to set Authorization: Bearer header
Without this patch the istio RequestAuthentication resources fail to
match because the access token from ZITADEL returned by oauth2-proxy in
the x-auth-request-access-token header is not a proper jwt.

The error is:

```
Jwt is not in the form of Header.Payload.Signature with two dots and 3 sections
```

This patch works around the problem by configuring oauth2-proxy to set
the ID token, which is guaranteed to be a proper JWT in the
authorization response headers.

Unfortunately, oauth2-proxy will only place the ID token in the
Authorization header response, which will write over any header set by a
client application.  This is likely to cause problems with single page
apps.

We'll probably need to work around this issue by using the alpha
configuration to set the id token in some out-of-the-way header.  We've
done this before, it'll just take some work to setup the ConfigMap and
translate the config again.
2024-03-30 16:15:27 -07:00
Jeff McCune
40fdfc0317 (#66) Fix auth proxy provider name, stage is always first
dev-holos-authproxy not authproxy-dev-holos
2024-03-30 14:05:50 -07:00
Jeff McCune
25d9415b0a (#66) Fix redis not able to write to /data
Without this patch redis cannot write to the /data directory, which
causes oauth2-proxy to fail with a 500 server error.
2024-03-30 13:40:34 -07:00
32 changed files with 12736 additions and 182 deletions

View File

@@ -0,0 +1,37 @@
package holos
import ap "security.istio.io/authorizationpolicy/v1"
// #AuthPolicyRules represents AuthorizationPolicy rules for hosts that need specialized treatment. Entries in this struct are exclused from the blank ingressauth AuthorizationPolicy governing the ingressgateway and included in a spcialized policy
#AuthPolicyRules: {
// AuthProxySpec represents the identity provider configuration
AuthProxySpec: #AuthProxySpec & #Platform.authproxy
// Hosts are hosts that need specialized treatment
hosts: {
[Name=_]: {
// name is the fully qualifed hostname, a Host: header value.
name: Name
// slug is the resource name prefix
slug: string
// Refer to https://istio.io/latest/docs/reference/config/security/authorization-policy/#Rule
spec: ap.#AuthorizationPolicySpec & {
action: "CUSTOM"
provider: name: AuthProxySpec.provider
selector: matchLabels: istio: "ingressgateway"
}
}
}
objects: #APIObjects & {
for Host in hosts {
apiObjects: {
AuthorizationPolicy: "\(Host.slug)-custom": {
metadata: namespace: "istio-ingress"
metadata: name: "\(Host.slug)-custom"
spec: Host.spec
}
}
}
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,189 @@
// Code generated by timoni. DO NOT EDIT.
//timoni:generate timoni vendor crd -f /home/jeff/workspace/holos-run/holos-infra/deploy/clusters/k2/components/prod-platform-argocd/prod-platform-argocd.gen.yaml
package v1alpha1
import "strings"
// AppProject provides a logical grouping of applications,
// providing controls for: * where the apps may deploy to
// (cluster whitelist) * what may be deployed (repository
// whitelist, resource whitelist/blacklist) * who can access
// these applications (roles, OIDC group claims bindings) * and
// what they can do (RBAC policies) * automation access to these
// roles (JWT tokens)
#AppProject: {
// APIVersion defines the versioned schema of this representation
// of an object. Servers should convert recognized schemas to the
// latest internal value, and may reject unrecognized values.
// More info:
// https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
apiVersion: "argoproj.io/v1alpha1"
// Kind is a string value representing the REST resource this
// object represents. Servers may infer this from the endpoint
// the client submits requests to. Cannot be updated. In
// CamelCase. More info:
// https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
kind: "AppProject"
metadata: {
name!: strings.MaxRunes(253) & strings.MinRunes(1) & {
string
}
namespace!: strings.MaxRunes(63) & strings.MinRunes(1) & {
string
}
labels?: {
[string]: string
}
annotations?: {
[string]: string
}
}
// AppProjectSpec is the specification of an AppProject
spec!: #AppProjectSpec
}
// AppProjectSpec is the specification of an AppProject
#AppProjectSpec: {
// ClusterResourceBlacklist contains list of blacklisted cluster
// level resources
clusterResourceBlacklist?: [...{
group: string
kind: string
}]
// ClusterResourceWhitelist contains list of whitelisted cluster
// level resources
clusterResourceWhitelist?: [...{
group: string
kind: string
}]
// Description contains optional project description
description?: string
// Destinations contains list of destinations available for
// deployment
destinations?: [...{
// Name is an alternate way of specifying the target cluster by
// its symbolic name. This must be set if Server is not set.
name?: string
// Namespace specifies the target namespace for the application's
// resources. The namespace will only be set for namespace-scoped
// resources that have not set a value for .metadata.namespace
namespace?: string
// Server specifies the URL of the target cluster's Kubernetes
// control plane API. This must be set if Name is not set.
server?: string
}]
// NamespaceResourceBlacklist contains list of blacklisted
// namespace level resources
namespaceResourceBlacklist?: [...{
group: string
kind: string
}]
// NamespaceResourceWhitelist contains list of whitelisted
// namespace level resources
namespaceResourceWhitelist?: [...{
group: string
kind: string
}]
// OrphanedResources specifies if controller should monitor
// orphaned resources of apps in this project
orphanedResources?: {
// Ignore contains a list of resources that are to be excluded
// from orphaned resources monitoring
ignore?: [...{
group?: string
kind?: string
name?: string
}]
// Warn indicates if warning condition should be created for apps
// which have orphaned resources
warn?: bool
}
// PermitOnlyProjectScopedClusters determines whether destinations
// can only reference clusters which are project-scoped
permitOnlyProjectScopedClusters?: bool
// Roles are user defined RBAC roles associated with this project
roles?: [...{
// Description is a description of the role
description?: string
// Groups are a list of OIDC group claims bound to this role
groups?: [...string]
// JWTTokens are a list of generated JWT tokens bound to this role
jwtTokens?: [...{
exp?: int
iat: int
id?: string
}]
// Name is a name for this role
name: string
// Policies Stores a list of casbin formatted strings that define
// access policies for the role in the project
policies?: [...string]
}]
// SignatureKeys contains a list of PGP key IDs that commits in
// Git must be signed with in order to be allowed for sync
signatureKeys?: [...{
// The ID of the key in hexadecimal notation
keyID: string
}]
// SourceNamespaces defines the namespaces application resources
// are allowed to be created in
sourceNamespaces?: [...string]
// SourceRepos contains list of repository URLs which can be used
// for deployment
sourceRepos?: [...string]
// SyncWindows controls when syncs can be run for apps in this
// project
syncWindows?: [...{
// Applications contains a list of applications that the window
// will apply to
applications?: [...string]
// Clusters contains a list of clusters that the window will apply
// to
clusters?: [...string]
// Duration is the amount of time the sync window will be open
duration?: string
// Kind defines if the window allows or blocks syncs
kind?: string
// ManualSync enables manual syncs when they would otherwise be
// blocked
manualSync?: bool
// Namespaces contains a list of namespaces that the window will
// apply to
namespaces?: [...string]
// Schedule is the time the window will begin, specified in cron
// format
schedule?: string
// TimeZone of the sync that will be applied to the schedule
timeZone?: string
}]
}

View File

@@ -18,7 +18,10 @@ import "encoding/yaml"
Issuer?: [Name=_]: #Issuer & {metadata: name: Name}
Gateway?: [Name=_]: #Gateway & {metadata: name: Name}
ConfigMap?: [Name=_]: #ConfigMap & {metadata: name: Name}
Deployment?: [_]: #Deployment
Deployment?: [_]: #Deployment
RequestAuthentication?: [_]: #RequestAuthentication
AuthorizationPolicy?: [_]: #AuthorizationPolicy
}
// apiObjectMap holds the marshalled representation of apiObjects

View File

@@ -6,9 +6,10 @@ package holos
// clusterName is the value of the --cluster-name flag, the cluster currently being manged / rendered.
clusterName: string | *#ClusterName
extensionProviderMap: [Name=_]: {
name: Name
}
// for extAuthzHttp extension providers
extensionProviderMap: [Name=_]: #ExtAuthzProxy & {name: Name}
// for other extension providers like zipkin
extensionProviderExtraMap: [Name=_]: {name: Name}
config: {
accessLogEncoding: string | *"JSON"
@@ -21,7 +22,10 @@ package holos
enablePrometheusMerge: false | *true
rootNamespace: string | *"istio-system"
trustDomain: string | *"cluster.local"
extensionProviders: [for x in extensionProviderMap {x}]
extensionProviders: [
for x in extensionProviderMap {x},
for y in extensionProviderExtraMap {y},
]
}
}
@@ -36,9 +40,7 @@ package holos
headersToUpstreamOnAllow: [
"authorization",
"path",
"x-auth-request-user",
"x-auth-request-email",
"x-auth-request-access-token",
"x-oidc-id-token",
]
includeAdditionalHeadersInCheck: "X-Auth-Request-Redirect": "%REQ(x-forwarded-proto)%://%REQ(:authority)%%REQ(:path)%%REQ(:query)%"
includeRequestHeadersInCheck: [

View File

@@ -1,5 +1,21 @@
package holos
#PlatformServers: {
for cluster in #Platform.clusters {
(cluster.name): {
"https-istio-ingress-httpbin": {
let cert = #PlatformCerts[cluster.name+"-httpbin"]
hosts: [for host in cert.spec.dnsNames {"istio-ingress/\(host)"}]
port: name: "https-istio-ingress-httpbin"
port: number: 443
port: protocol: "HTTPS"
tls: credentialName: cert.spec.secretName
tls: mode: "SIMPLE"
}
}
}
}
#PlatformCerts: {
// Globally scoped platform services are defined here.
login: #PlatformCert & {

View File

@@ -4,4 +4,4 @@ package holos
#InputKeys: project: "iam"
// Shared dependencies for all components in this collection.
#DependsOn: namespaces: name: "\(#StageName)-secrets-namespaces"
#DependsOn: namespaces: name: "\(#StageName)-secrets-stores"

View File

@@ -37,7 +37,7 @@ spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
metadata: name: "prod-iam-postgres"
_dependsOn: "prod-secrets-namespaces": _
_dependsOn: "prod-secrets-stores": _
_dependsOn: "prod-iam-postgres-certs": _
apiObjectMap: OBJECTS.apiObjectMap
},
@@ -68,7 +68,7 @@ let OBJECTS = #APIObjects & {
replicas: 2
dataVolumeClaimSpec: {
accessModes: ["ReadWriteOnce"]
resources: requests: storage: "10Gi"
resources: requests: storage: "20Gi"
}
}]
standby: {
@@ -124,7 +124,7 @@ let OBJECTS = #APIObjects & {
"\(BucketRepoName)-cipher-type": "aes-256-cbc"
// "The convention we recommend for setting this variable is /pgbackrest/$NAMESPACE/$CLUSTER_NAME/repoN"
// Ref: https://access.crunchydata.com/documentation/postgres-operator/latest/tutorials/backups-disaster-recovery/backups#understanding-backup-configuration-and-basic-operations
"\(BucketRepoName)-path": "/pgbackrest/\(#TargetNamespace)/\(metadata.name)/\(manual.repoName)"
"\(BucketRepoName)-path": "/pgbackrest/\(metadata.namespace)/\(metadata.name)/\(manual.repoName)"
}
repos: [
{
@@ -165,7 +165,7 @@ let HighlyAvailable = {
replicas: 2
dataVolumeClaimSpec: {
accessModes: ["ReadWriteOnce"]
resources: requests: storage: string | *"10Gi"
resources: requests: storage: string | *"20Gi"
}
affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: [{
weight: 1

View File

@@ -1,7 +1,9 @@
package holos
#InstancePrefix: "prod-iam"
#TargetNamespace: #InstancePrefix + "-zitadel"
#InstancePrefix: "prod-iam"
// The namespace is managed by a project.
#TargetNamespace: _Projects.iam.environments.prod.namespace
// _DBName is the database name used across multiple holos components in this project
_DBName: "zitadel"

View File

@@ -0,0 +1,84 @@
package holos
import "encoding/yaml"
let ArgoCD = "argocd"
let Namespace = "prod-platform"
spec: components: HelmChartList: [
#HelmChart & {
_dependsOn: "prod-secrets-stores": _
namespace: Namespace
metadata: name: "\(namespace)-\(ArgoCD)"
chart: {
name: "argo-cd"
release: "argocd"
version: "6.7.8"
repository: {
name: "argocd"
url: "https://argoproj.github.io/argo-helm"
}
}
_values: #ArgoCDValues & {
kubeVersionOverride: "1.29.0"
global: domain: "argocd.\(#ClusterName).\(#Platform.org.domain)"
dex: enabled: false
// for integration with istio
configs: params: "server.insecure": true
configs: cm: {
"admin.enabled": false
"oidc.config": yaml.Marshal(OIDCConfig)
}
}
// Holos overlay objects
apiObjectMap: OBJECTS.apiObjectMap
},
]
let OBJECTS = #APIObjects & {
apiObjects: {
// ExternalSecret: "deploy-key": _
VirtualService: (ArgoCD): {
metadata: name: ArgoCD
metadata: namespace: Namespace
spec: hosts: [
ArgoCD + ".\(#Platform.org.domain)",
ArgoCD + ".\(#ClusterName).\(#Platform.org.domain)",
]
spec: gateways: ["istio-ingress/default"]
spec: http: [{route: [{destination: {
host: "argocd-server.\(Namespace).svc.cluster.local"
port: number: 80
}}]}]
}
}
}
let IstioInject = [{op: "add", path: "/spec/template/metadata/labels/sidecar.istio.io~1inject", value: "true"}]
#Kustomize: _patches: {
mesh: {
target: {
group: "apps"
version: "v1"
kind: "Deployment"
name: "argocd-server"
}
patch: yaml.Marshal(IstioInject)
}
}
// Probably shouldn't use the authproxy struct and should instead define an identity provider struct.
let AuthProxySpec = #AuthProxySpec & #Platform.authproxy
let OIDCConfig = {
name: "Holos Platform"
issuer: AuthProxySpec.issuer
clientID: #Platform.argocd.clientID
requestedIDTokenClaims: groups: essential: true
requestedScopes: ["openid", "profile", "email", "groups", "urn:zitadel:iam:org:domain:primary:\(AuthProxySpec.orgDomain)"]
enablePKCEAuthentication: true
}

File diff suppressed because it is too large Load Diff

View File

@@ -8,7 +8,7 @@ spec: components: HelmChartList: [
namespace: "istio-system"
chart: {
name: "base"
version: "1.20.3"
version: #IstioVersion
repository: {
name: "istio"
url: "https://istio-release.storage.googleapis.com/charts"

View File

@@ -7,48 +7,53 @@ let Name = "gateway"
#InputKeys: component: Name
#TargetNamespace: "istio-ingress"
let LoginCert = #PlatformCerts.login
spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
_dependsOn: "prod-secrets-namespaces": _
_dependsOn: "prod-mesh-istio-base": _
_dependsOn: "prod-mesh-ingress": _
_dependsOn: "prod-secrets-stores": _
_dependsOn: "prod-mesh-istio-base": _
_dependsOn: "prod-mesh-ingress": _
metadata: name: "\(#InstancePrefix)-\(Name)"
apiObjectMap: OBJECTS.apiObjectMap
},
]
// GatewayServers represents all hosts for all VirtualServices in the cluster attached to Gateway/default
// NOTE: This is a critical structure because the default Gateway should be used in most cases.
let GatewayServers = {
for Project in _Projects {
for server in (#ProjectTemplate & {project: Project}).ClusterGatewayServers {
(server.port.name): server
}
}
for k, svc in #OptionalServices {
if svc.enabled && list.Contains(svc.clusterNames, #ClusterName) {
for server in svc.servers {
(server.port.name): server
}
}
}
if #PlatformServers[#ClusterName] != _|_ {
for server in #PlatformServers[#ClusterName] {
(server.port.name): server
}
}
}
let OBJECTS = #APIObjects & {
apiObjects: {
ExternalSecret: login: #ExternalSecret & {
_name: "login"
}
Gateway: default: #Gateway & {
metadata: name: "default"
metadata: namespace: #TargetNamespace
spec: selector: istio: "ingressgateway"
spec: servers: [
{
hosts: [for dnsName in LoginCert.spec.dnsNames {"prod-iam-zitadel/\(dnsName)"}]
port: name: "https-prod-iam-login"
port: number: 443
port: protocol: "HTTPS"
tls: credentialName: LoginCert.spec.secretName
tls: mode: "SIMPLE"
},
]
spec: servers: [for x in GatewayServers {x}]
}
for k, svc in #OptionalServices {
if svc.enabled && list.Contains(svc.clusterNames, #ClusterName) {
Gateway: "\(svc.name)": #Gateway & {
metadata: name: svc.name
metadata: namespace: #TargetNamespace
spec: selector: istio: "ingressgateway"
spec: servers: [for s in svc.servers {s}]
}
for k, s in svc.servers {
ExternalSecret: "\(s.tls.credentialName)": _
}

View File

@@ -3,7 +3,6 @@ package holos
let Name = "httpbin"
let ComponentName = "\(#InstancePrefix)-\(Name)"
let SecretName = #InputKeys.cluster + "-" + Name
let MatchLabels = {
app: Name
"app.kubernetes.io/instance": ComponentName
@@ -18,7 +17,7 @@ let Metadata = {
#TargetNamespace: "istio-ingress"
let Cert = #PlatformCerts[SecretName]
let Cert = #PlatformCerts["\(#ClusterName)-httpbin"]
spec: components: KubernetesObjectsList: [
#KubernetesObjects & {
@@ -63,24 +62,10 @@ let OBJECTS = #APIObjects & {
{port: 80, targetPort: 8080, protocol: "TCP", name: "http"},
]
}
Gateway: httpbin: #Gateway & {
metadata: Metadata
spec: selector: istio: "ingressgateway"
spec: servers: [
{
hosts: [for host in Cert.spec.dnsNames {"\(#TargetNamespace)/\(host)"}]
port: name: "https-\(ComponentName)"
port: number: 443
port: protocol: "HTTPS"
tls: credentialName: Cert.spec.secretName
tls: mode: "SIMPLE"
},
]
}
VirtualService: httpbin: #VirtualService & {
metadata: Metadata
spec: hosts: [for host in Cert.spec.dnsNames {host}]
spec: gateways: ["\(#TargetNamespace)/\(Name)"]
spec: gateways: ["istio-ingress/default"]
spec: http: [{route: [{destination: host: Name}]}]
}
}

View File

@@ -52,6 +52,12 @@ spec: components: HelmChartList: [
}
}
apiObjectMap: OBJECTS.apiObjectMap
// Auth Proxy
apiObjectMap: _IngressAuthProxy.Deployment.apiObjectMap
// Auth Policy
apiObjectMap: _IngressAuthProxy.Policy.apiObjectMap
// Auth Policy Exclusions
apiObjectMap: _AuthPolicyRules.objects.apiObjectMap
},
]

View File

@@ -2,7 +2,7 @@ package holos
#HelmChart: {
chart: {
version: "1.20.3"
version: #IstioVersion
repository: {
name: "istio"
url: "https://istio-release.storage.googleapis.com/charts"

View File

@@ -1,5 +1,9 @@
package holos
// Ingress Gateway default auth proxy
let Provider = _IngressAuthProxy.AuthProxySpec.provider
let Service = _IngressAuthProxy.service
#MeshConfig: extensionProviderMap: (Provider): envoyExtAuthzHttp: service: Service
// Istio meshconfig
// TODO: Generate per-project extauthz providers.
_MeshConfig: (#MeshConfig & {projects: _Projects}).config

View File

@@ -126,7 +126,7 @@ package holos
hub: "docker.io/istio"
// Default tag for Istio images.
tag: "1.20.3"
tag: #IstioVersion
// Variant of the image to use.
// Currently supported are: [debug, distroless]

View File

@@ -1,3 +1,323 @@
package holos
import "encoding/yaml"
#InstancePrefix: "prod-mesh"
#IstioVersion: "1.21.0"
// The ingress gateway auth proxy is used by multiple cue instances.
// AUTHPROXY configures one oauth2-proxy deployment for each host in each stage of a project. Multiple deployments per stage are used to narrow down the cookie domain.
_IngressAuthProxy: {
Name: "authproxy"
Namespace: "istio-ingress"
service: "\(Name).\(Namespace).svc.cluster.local"
AuthProxySpec: #AuthProxySpec & #Platform.authproxy
Domains: [DOMAIN=string]: {name: DOMAIN}
Domains: (#Platform.org.domain): _
Domains: "\(#ClusterName).\(#Platform.org.domain)": _
let Metadata = {
name: string
namespace: Namespace
labels: "app.kubernetes.io/name": name
labels: "app.kubernetes.io/part-of": "istio-ingressgateway"
...
}
let ProxyMetadata = Metadata & {name: Name}
let RedisMetadata = Metadata & {name: Name + "-redis"}
// Deployment represents the oauth2-proxy deployment
Deployment: #APIObjects & {
apiObjects: {
// oauth2-proxy
ExternalSecret: (Name): metadata: ProxyMetadata
// Place the ID token in a header that does not conflict with the Authorization header.
// Refer to: https://github.com/oauth2-proxy/oauth2-proxy/issues/1877#issuecomment-1364033723
ConfigMap: (Name): {
metadata: ProxyMetadata
data: "config.yaml": yaml.Marshal(AuthProxyConfig)
let AuthProxyConfig = {
injectResponseHeaders: [{
name: "x-oidc-id-token"
values: [{claim: "id_token"}]
}]
providers: [{
id: "Holos Platform"
name: "Holos Platform"
provider: "oidc"
scope: "openid profile email groups offline_access urn:zitadel:iam:org:domain:primary:\(AuthProxySpec.orgDomain)"
clientID: AuthProxySpec.clientID
clientSecretFile: "/dev/null"
code_challenge_method: "S256"
loginURLParameters: [{
default: ["force"]
name: "approval_prompt"
}]
oidcConfig: {
issuerURL: AuthProxySpec.issuer
audienceClaims: ["aud"]
emailClaim: "email"
groupsClaim: "groups"
userIDClaim: "sub"
}
}]
server: BindAddress: ":4180"
upstreamConfig: upstreams: [{
id: "static://200"
path: "/"
static: true
staticCode: 200
}]
}
}
Deployment: (Name): #Deployment & {
metadata: ProxyMetadata
spec: {
replicas: 1
selector: matchLabels: ProxyMetadata.labels
template: {
metadata: labels: ProxyMetadata.labels
metadata: labels: #IstioSidecar
spec: {
securityContext: seccompProfile: type: "RuntimeDefault"
containers: [{
image: "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0"
imagePullPolicy: "IfNotPresent"
name: "oauth2-proxy"
volumeMounts: [{
name: "config"
mountPath: "/config"
readOnly: true
}]
args: [
// callback url is proxy prefix + /callback
"--proxy-prefix=" + AuthProxySpec.proxyPrefix,
"--email-domain=*",
"--session-store-type=redis",
"--redis-connection-url=redis://\(RedisMetadata.name):6379",
"--cookie-refresh=12h",
"--cookie-expire=2160h",
"--cookie-secure=true",
"--cookie-name=__Secure-\(#ClusterName)-ingress-\(Name)",
"--cookie-samesite=lax",
for domain in Domains {"--cookie-domain=.\(domain.name)"},
for domain in Domains {"--cookie-domain=\(domain.name)"},
for domain in Domains {"--whitelist-domain=.\(domain.name)"},
for domain in Domains {"--whitelist-domain=\(domain.name)"},
"--cookie-csrf-per-request=true",
"--cookie-csrf-expire=120s",
// will skip authentication for OPTIONS requests
"--skip-auth-preflight=true",
"--real-client-ip-header=X-Forwarded-For",
"--skip-provider-button=true",
"--auth-logging",
"--alpha-config=/config/config.yaml",
]
env: [{
name: "OAUTH2_PROXY_COOKIE_SECRET"
// echo '{"cookiesecret":"'$(LC_ALL=C tr -dc "[:alpha:]" </dev/random | tr '[:upper:]' '[:lower:]' | head -c 32)'"}' | holos create secret -n istio-ingress --append-hash=false --data-stdin authproxy
valueFrom: secretKeyRef: {
key: "cookiesecret"
name: Name
}
}]
ports: [{
containerPort: 4180
protocol: "TCP"
}]
securityContext: {
seccompProfile: type: "RuntimeDefault"
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 8192
runAsGroup: 8192
capabilities: drop: ["ALL"]
}
}]
volumes: [{name: "config", configMap: name: Name}]
}
}
}
}
Service: (Name): #Service & {
metadata: ProxyMetadata
spec: selector: ProxyMetadata.labels
spec: ports: [
{port: 4180, targetPort: 4180, protocol: "TCP", name: "http"},
]
}
VirtualService: (Name): #VirtualService & {
metadata: ProxyMetadata
spec: hosts: ["*"]
spec: gateways: ["istio-ingress/default"]
spec: http: [{
match: [{uri: prefix: AuthProxySpec.proxyPrefix}]
route: [{
destination: host: Name
destination: port: number: 4180
}]
}]
}
// redis
ConfigMap: (RedisMetadata.name): {
metadata: RedisMetadata
data: "redis.conf": """
maxmemory 128mb
maxmemory-policy allkeys-lru
"""
}
Deployment: (RedisMetadata.name): {
metadata: RedisMetadata
spec: {
selector: matchLabels: RedisMetadata.labels
template: {
metadata: labels: RedisMetadata.labels
metadata: labels: #IstioSidecar
spec: securityContext: seccompProfile: type: "RuntimeDefault"
spec: {
containers: [{
command: [
"redis-server",
"/redis-master/redis.conf",
]
env: [{
name: "MASTER"
value: "true"
}]
image: "quay.io/holos/redis:7.2.4"
livenessProbe: {
initialDelaySeconds: 15
tcpSocket: port: "redis"
}
name: "redis"
ports: [{
containerPort: 6379
name: "redis"
}]
readinessProbe: {
exec: command: [
"redis-cli",
"ping",
]
initialDelaySeconds: 5
}
resources: limits: cpu: "0.5"
securityContext: {
seccompProfile: type: "RuntimeDefault"
allowPrivilegeEscalation: false
capabilities: drop: ["ALL"]
runAsNonRoot: true
runAsUser: 999
runAsGroup: 999
}
volumeMounts: [{
mountPath: "/redis-master-data"
name: "data"
}, {
mountPath: "/redis-master"
name: "config"
}]
}]
volumes: [{
emptyDir: {}
name: "data"
}, {
configMap: name: RedisMetadata.name
name: "config"
}]
}
}
}
}
Service: (RedisMetadata.name): #Service & {
metadata: RedisMetadata
spec: selector: RedisMetadata.labels
spec: type: "ClusterIP"
spec: ports: [{
name: "redis"
port: 6379
protocol: "TCP"
targetPort: 6379
}]
}
}
}
// Policy represents the AuthorizationPolicy and RequestAuthentication policy
Policy: #APIObjects & {
apiObjects: {
RequestAuthentication: (Name): #RequestAuthentication & {
metadata: Metadata & {name: Name}
spec: jwtRules: [{
audiences: ["\(AuthProxySpec.projectID)"]
forwardOriginalToken: true
fromHeaders: [{name: AuthProxySpec.idTokenHeader}]
issuer: AuthProxySpec.issuer
}]
spec: selector: matchLabels: istio: "ingressgateway"
}
AuthorizationPolicy: "\(Name)-custom": {
_description: "Route all requests through the auth proxy by default"
metadata: Metadata & {name: "\(Name)-custom"}
spec: {
action: "CUSTOM"
provider: name: AuthProxySpec.provider
rules: [
{
to: [{
operation: notHosts: [
// Never send requests for the login service through the authorizer, would block login.
AuthProxySpec.issuerHost,
// Exclude hosts with specialized rules from the catch-all.
for x in _AuthPolicyRules.hosts {x.name},
]
}]
when: [
{
// bypass the external authorizer when the id token is already in the request.
// the RequestAuthentication rule will verify the token.
key: "request.headers[\(AuthProxySpec.idTokenHeader)]"
notValues: ["*"]
},
]
},
]
selector: matchLabels: istio: "ingressgateway"
}
}
}
}
}
_AuthPolicyRules: #AuthPolicyRules & {
hosts: {
let Vault = "vault.core.ois.run"
(Vault): {
slug: "vault"
// Rules for when to route requests through the auth proxy
spec: rules: [
{
to: [{
operation: hosts: [Vault]
operation: paths: ["/ui", "/ui/*"]
}]
},
{
to: [{
operation: hosts: [Vault]
}]
when: [{
key: "request.headers[x-vault-request]"
notValues: ["true"]
}]
},
]
}
}
}

View File

@@ -172,20 +172,14 @@ package holos
enabled: true
// Indicates whether to enable WebAssembly runtime for stats filter.
wasmEnabled: false
// overrides stats EnvoyFilter configuration.
configOverride: {
gateway: {}
inboundSidecar: {}
outboundSidecar: {}
}
}
// stackdriver filter settings.
stackdriver: {
enabled: false
logging: false
monitoring: false
topology: false // deprecated. setting this to true will have no effect, as this option is no longer supported.
disableOutbound: false
enabled: false
logging: false
monitoring: false
topology: false // deprecated. setting this to true will have no effect, as this option is no longer supported.
// configOverride parts give you the ability to override the low level configuration params passed to envoy filter.
configOverride: {}
@@ -248,7 +242,7 @@ package holos
// Dev builds from prow are on gcr.io
hub: string | *"docker.io/istio"
// Default tag for Istio images.
tag: string | *"1.20.3"
tag: #IstioVersion
// Variant of the image to use.
// Currently supported are: [debug, distroless]
variant: string | *""

View File

@@ -10,7 +10,7 @@ package holos
spec: components: HelmChartList: [
#HelmChart & {
_dependsOn: "prod-secrets-namespaces": _
_dependsOn: "prod-secrets-stores": _
metadata: name: "prod-metal-ceph"

View File

@@ -67,7 +67,7 @@ let OBJECTS = #APIObjects & {
metadata: name: Name
metadata: namespace: #TargetNamespace
spec: hosts: [for cert in Vault.certs {cert.spec.commonName}]
spec: gateways: ["istio-ingress/\(Name)"]
spec: gateways: ["istio-ingress/default"]
spec: http: [
{
route: [

View File

@@ -2,15 +2,27 @@ package holos
#Project: authProxyOrgDomain: "openinfrastructure.co"
let ZitadelProjectID = 257713952794870157
_Projects: #Projects & {
// The platform project is required and where platform services reside. ArgoCD, Grafana, Prometheus, etc...
platform: {
resourceId: ZitadelProjectID
// platform level services typically run in the core cluster pair.
clusters: core1: _
clusters: core2: _
// for development, probably wouldn't run these services in the workload clusters.
clusters: k2: _
// Services hosted in the platform project
hosts: argocd: _
hosts: grafana: _
hosts: prometheus: _
}
holos: {
resourceId: 260446255245690199
clusters: {
k1: _
k2: _
}
stages: dev: authProxyClientID: "260505543108527218@holos"
stages: prod: authProxyClientID: "260506079325128023@holos"
resourceId: ZitadelProjectID
clusters: k1: _
clusters: k2: _
environments: {
prod: stage: "prod"
dev: stage: "dev"
@@ -21,13 +33,12 @@ _Projects: #Projects & {
}
iam: {
resourceId: 260582480954787159
resourceId: ZitadelProjectID
hosts: login: _
clusters: {
core1: _
core2: _
}
stages: dev: authProxyClientID: "260582521186616432@iam"
stages: prod: authProxyClientID: "260582633862399090@iam"
}
}

View File

@@ -8,7 +8,7 @@ package holos
// Refer to [Using Cert Manager to Deploy TLS for Postgres on Kubernetes](https://www.crunchydata.com/blog/using-cert-manager-to-deploy-tls-for-postgres-on-kubernetes)
#TargetNamespace: "prod-iam-zitadel"
#TargetNamespace: _Projects.iam.environments.prod.namespace
#InputKeys: component: "postgres-certs"
let DBName = "zitadel"

View File

@@ -4,7 +4,7 @@ package holos
projects: _
clusterName: _
extensionProviderMap: {
extensionProviderExtraMap: {
"cluster-trace": {
zipkin: {
maxTagLength: 56
@@ -25,8 +25,7 @@ package holos
for Project in projects {
if Project.clusters[clusterName] != _|_ {
for Stage in Project.stages {
let Name = "authproxy-\(Stage.slug)"
extensionProviderMap: (Name): #ExtAuthzProxy & {
extensionProviderMap: (Stage.extAuthzProviderName): #ExtAuthzProxy & {
envoyExtAuthzHttp: service: "authproxy.\(Stage.namespace).svc.cluster.local"
}
}

View File

@@ -22,7 +22,6 @@ let Privileged = {
{name: "istio-ingress"} & Restricted,
{name: "cert-manager"},
{name: "argocd"},
{name: "prod-iam-zitadel"},
{name: "arc-system"},
{name: "arc-runner"},
// https://github.com/CrunchyData/postgres-operator-examples/blob/main/kustomize/install/namespace/namespace.yaml

View File

@@ -1,5 +1,7 @@
package holos
import "encoding/yaml"
// Platform level definition of a project.
#Project: {
name: string
@@ -53,6 +55,18 @@ package holos
}
}
// ClusterGatewayServers provides a struct of Gateway servers for the current cluster.
// This is intended for Gateway/default to add all servers to the default gateway.
ClusterGatewayServers: {
if project.clusters[#ClusterName] != _|_ {
for Stage in project.stages {
for server in GatewayServers[Stage.name] {
(server.port.name): server
}
}
}
}
workload: resources: {
// Provide resources only if the project is managed on --cluster-name
if project.clusters[#ClusterName] != _|_ {
@@ -62,10 +76,6 @@ package holos
// Istio Gateway
"\(stage.slug)-gateway": #KubernetesObjects & {
apiObjectMap: (#APIObjects & {
apiObjects: Gateway: (stage.slug): #Gateway & {
spec: servers: [for server in GatewayServers[stage.name] {server}]
}
for host in GatewayServers[stage.name] {
apiObjects: ExternalSecret: (host.tls.credentialName): metadata: namespace: "istio-ingress"
}
@@ -73,19 +83,32 @@ package holos
}
// Manage auth-proxy in each stage
"\(stage.slug)-authproxy": #KubernetesObjects & {
apiObjectMap: (#APIObjects & {
apiObjects: (AUTHPROXY & {stage: Stage, project: Project, servers: GatewayServers[stage.name]}).apiObjects
}).apiObjectMap
if project.features.authproxy.enabled {
"\(stage.slug)-authproxy": #KubernetesObjects & {
apiObjectMap: (#APIObjects & {
apiObjects: (AUTHPROXY & {stage: Stage, project: Project, servers: GatewayServers[stage.name]}).apiObjects
}).apiObjectMap
}
for Env in project.environments if Env.stage == stage.name {
"\(Env.slug)-authpolicy": #KubernetesObjects & {
// Manage auth policy in each env
apiObjectMap: (#APIObjects & {
apiObjects: (AUTHPOLICY & {env: Env, project: Project, servers: GatewayServers[stage.name]}).apiObjects
}).apiObjectMap
}
}
}
// Manage httpbin in each environment
for Env in project.environments if Env.stage == stage.name {
"\(Env.slug)-httpbin": #KubernetesObjects & {
apiObjectMap: (#APIObjects & {
if project.features.httpbin.enabled {
for Env in project.environments if Env.stage == stage.name {
"\(Env.slug)-httpbin": #KubernetesObjects & {
let Project = project
apiObjects: (HTTPBIN & {env: Env, project: Project}).apiObjects
}).apiObjectMap
apiObjectMap: (#APIObjects & {
apiObjects: (HTTPBIN & {env: Env, project: Project}).apiObjects
}).apiObjectMap
}
}
}
}
@@ -123,12 +146,19 @@ let HTTPBIN = {
project: #Project
env: #Environment
let Name = name
let Stage = project.stages[env.stage]
let Metadata = {
name: Name
namespace: env.namespace
labels: app: name
}
let Labels = {
"app.kubernetes.io/name": Name
"app.kubernetes.io/instance": env.slug
"app.kubernetes.io/part-of": env.project
"security.holos.run/authproxy": Stage.extAuthzProviderName
}
apiObjects: {
Deployment: (Name): #Deployment & {
@@ -136,7 +166,7 @@ let HTTPBIN = {
spec: selector: matchLabels: Metadata.labels
spec: template: {
metadata: labels: Metadata.labels & #IstioSidecar
metadata: labels: Metadata.labels & #IstioSidecar & Labels
spec: securityContext: seccompProfile: type: "RuntimeDefault"
spec: containers: [{
name: Name
@@ -164,7 +194,7 @@ let HTTPBIN = {
let Project = project
let Env = env
spec: hosts: [for host in (#EnvHosts & {project: Project, env: Env}).hosts {host.name}]
spec: gateways: ["istio-ingress/\(env.stageSlug)"]
spec: gateways: ["istio-ingress/default"]
spec: http: [{route: [{destination: host: Name}]}]
}
}
@@ -180,6 +210,14 @@ let AUTHPROXY = {
let Project = project
let Stage = stage
let AuthProxySpec = #AuthProxySpec & {
namespace: stage.namespace
projectID: project.resourceId
clientID: stage.authProxyClientID
orgDomain: project.authProxyOrgDomain
provider: stage.extAuthzProviderName
}
let Metadata = {
name: Name
namespace: stage.namespace
@@ -203,6 +241,45 @@ let AUTHPROXY = {
apiObjects: {
// oauth2-proxy
ExternalSecret: (Name): metadata: Metadata
// Place the ID token in a header that does not conflict with the Authorization header.
// Refer to: https://github.com/oauth2-proxy/oauth2-proxy/issues/1877#issuecomment-1364033723
ConfigMap: (Name): {
metadata: Metadata
data: "config.yaml": yaml.Marshal(AuthProxyConfig)
let AuthProxyConfig = {
injectResponseHeaders: [{
name: AuthProxySpec.idTokenHeader
values: [{claim: "id_token"}]
}]
providers: [{
id: "Holos Platform"
name: "Holos Platform"
provider: "oidc"
scope: "openid profile email groups offline_access urn:zitadel:iam:org:domain:primary:\(AuthProxySpec.orgDomain)"
clientID: AuthProxySpec.clientID
clientSecretFile: "/dev/null"
code_challenge_method: "S256"
loginURLParameters: [{
default: ["force"]
name: "approval_prompt"
}]
oidcConfig: {
issuerURL: AuthProxySpec.issuer
audienceClaims: ["aud"]
emailClaim: "email"
groupsClaim: "groups"
userIDClaim: "sub"
}
}]
server: BindAddress: ":4180"
upstreamConfig: upstreams: [{
id: "static://200"
path: "/"
static: true
staticCode: 200
}]
}
}
Deployment: (Name): #Deployment & {
metadata: Metadata
@@ -219,65 +296,64 @@ let AUTHPROXY = {
template: {
metadata: labels: Metadata.labels
metadata: labels: #IstioSidecar
spec: securityContext: seccompProfile: type: "RuntimeDefault"
spec: containers: [{
image: "quay.io/oauth2-proxy/oauth2-proxy:v7.4.0"
imagePullPolicy: "IfNotPresent"
name: "oauth2-proxy"
args: [
// callback url is proxy prefix + /callback
"--proxy-prefix=" + project.authProxyPrefix,
"--email-domain=*",
"--session-store-type=redis",
"--redis-connection-url=redis://\(RedisMetadata.name):6379",
"--cookie-refresh=12h",
"--cookie-expire=2160h",
"--cookie-secure=true",
"--cookie-name=__Secure-\(Name)-\(stage.slug)",
"--cookie-samesite=lax",
for domain in StageDomains {"--cookie-domain=.\(domain.name)"},
for domain in StageDomains {"--whitelist-domain=.\(domain.name)"},
"--cookie-csrf-per-request=true",
"--cookie-csrf-expire=120s",
"--set-authorization-header=false",
"--set-xauthrequest=true",
"--pass-access-token=true",
"--pass-authorization-header=true",
"--upstream=static://200",
"--reverse-proxy",
"--real-client-ip-header=X-Forwarded-For",
"--skip-provider-button=true",
"--auth-logging",
"--provider=oidc",
"--scope=openid profile email groups offline_access urn:zitadel:iam:org:domain:primary:\(project.authProxyOrgDomain)",
"--client-id=" + stage.authProxyClientID,
"--client-secret-file=/dev/null",
"--oidc-issuer-url=\(project.authProxyIssuer)",
"--code-challenge-method=S256",
"--http-address=0.0.0.0:4180",
// "--allowed-group=\(project.resourceId):\(stage.name)-access",
]
env: [{
name: "OAUTH2_PROXY_COOKIE_SECRET"
// echo '{"cookiesecret":"'$(LC_ALL=C tr -dc "[:alpha:]" </dev/random | tr '[:upper:]' '[:lower:]' | head -c 32)'"}' | holos create secret -n dev-holos-system --append-hash=false --data-stdin authproxy
valueFrom: secretKeyRef: {
key: "cookiesecret"
name: Name
spec: {
securityContext: seccompProfile: type: "RuntimeDefault"
containers: [{
image: "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0"
imagePullPolicy: "IfNotPresent"
name: "oauth2-proxy"
volumeMounts: [{
name: "config"
mountPath: "/config"
readOnly: true
}]
args: [
// callback url is proxy prefix + /callback
"--proxy-prefix=" + AuthProxySpec.proxyPrefix,
"--email-domain=*",
"--session-store-type=redis",
"--redis-connection-url=redis://\(RedisMetadata.name):6379",
"--cookie-refresh=12h",
"--cookie-expire=2160h",
"--cookie-secure=true",
"--cookie-name=__Secure-\(stage.slug)-\(Name)",
"--cookie-samesite=lax",
for domain in StageDomains {"--cookie-domain=.\(domain.name)"},
for domain in StageDomains {"--cookie-domain=\(domain.name)"},
for domain in StageDomains {"--whitelist-domain=.\(domain.name)"},
for domain in StageDomains {"--whitelist-domain=\(domain.name)"},
"--cookie-csrf-per-request=true",
"--cookie-csrf-expire=120s",
// will skip authentication for OPTIONS requests
"--skip-auth-preflight=true",
"--real-client-ip-header=X-Forwarded-For",
"--skip-provider-button=true",
"--auth-logging",
"--alpha-config=/config/config.yaml",
]
env: [{
name: "OAUTH2_PROXY_COOKIE_SECRET"
// echo '{"cookiesecret":"'$(LC_ALL=C tr -dc "[:alpha:]" </dev/random | tr '[:upper:]' '[:lower:]' | head -c 32)'"}' | holos create secret -n dev-holos-system --append-hash=false --data-stdin authproxy
valueFrom: secretKeyRef: {
key: "cookiesecret"
name: Name
}
}]
ports: [{
containerPort: 4180
protocol: "TCP"
}]
securityContext: {
seccompProfile: type: "RuntimeDefault"
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 8192
runAsGroup: 8192
capabilities: drop: ["ALL"]
}
}]
ports: [{
containerPort: 4180
protocol: "TCP"
}]
securityContext: {
seccompProfile: type: "RuntimeDefault"
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 8192
runAsGroup: 8192
capabilities: drop: ["ALL"]
}
}]
volumes: [{name: "config", configMap: name: Name}]
}
}
}
}
@@ -291,9 +367,9 @@ let AUTHPROXY = {
VirtualService: (Name): #VirtualService & {
metadata: Metadata
spec: hosts: ["*"]
spec: gateways: ["istio-ingress/\(stage.slug)"]
spec: gateways: ["istio-ingress/default"]
spec: http: [{
match: [{uri: prefix: project.authProxyPrefix}]
match: [{uri: prefix: AuthProxySpec.proxyPrefix}]
route: [{
destination: host: Name
destination: port: number: 4180
@@ -349,6 +425,9 @@ let AUTHPROXY = {
seccompProfile: type: "RuntimeDefault"
allowPrivilegeEscalation: false
capabilities: drop: ["ALL"]
runAsNonRoot: true
runAsUser: 999
runAsGroup: 999
}
volumeMounts: [{
mountPath: "/redis-master-data"
@@ -382,3 +461,79 @@ let AUTHPROXY = {
}
}
}
// AUTHPOLICY configures the baseline AuthorizationPolicy and RequestAuthentication policy for each stage of each project.
let AUTHPOLICY = {
project: #Project
env: #Environment
let Name = "\(stage.slug)-authproxy"
let Project = project
let stage = project.stages[env.stage]
let Env = env
let AuthProxySpec = #AuthProxySpec & {
namespace: stage.namespace
projectID: project.resourceId
clientID: stage.authProxyClientID
orgDomain: project.authProxyOrgDomain
provider: stage.extAuthzProviderName
}
let Metadata = {
name: string
namespace: env.namespace
labels: {
"app.kubernetes.io/name": name
"app.kubernetes.io/instance": stage.name
"app.kubernetes.io/part-of": stage.project
}
}
// Collect all the hosts associated with the stage
let Hosts = {
for HOST in (#EnvHosts & {project: Project, env: Env}).hosts {
(HOST.name): HOST
}
}
// HostList is a list of hosts for AuthorizationPolicy rules
let HostList = [
for host in Hosts {host.name},
for host in Hosts {host.name + ":*"},
]
let MatchLabels = {"security.holos.run/authproxy": AuthProxySpec.provider}
apiObjects: {
RequestAuthentication: (Name): #RequestAuthentication & {
metadata: Metadata & {name: Name}
spec: jwtRules: [{
audiences: [AuthProxySpec.clientID]
forwardOriginalToken: true
fromHeaders: [{name: AuthProxySpec.idTokenHeader}]
issuer: AuthProxySpec.issuer
}]
spec: selector: matchLabels: MatchLabels
}
AuthorizationPolicy: "\(Name)-custom": {
metadata: Metadata & {name: "\(Name)-custom"}
spec: {
action: "CUSTOM"
// send the request to the auth proxy
provider: name: AuthProxySpec.provider
rules: [{
to: [{operation: hosts: HostList}]
when: [
{
key: "request.headers[\(AuthProxySpec.idTokenHeader)]"
notValues: ["*"]
},
{
key: "request.headers[host]"
notValues: [AuthProxySpec.issuerHost]
},
]}]
selector: matchLabels: MatchLabels
}
}
}
}

View File

@@ -7,6 +7,9 @@ import "strings"
// #Projects is a map of all the projects in the platform.
#Projects: [Name=_]: #Project & {name: Name}
// The platform project is required and where platform services reside. ArgoCD, Grafana, Prometheus, etc...
#Projects: platform: _
#Project: {
name: string
// resourceId is the zitadel project Resource ID
@@ -23,8 +26,6 @@ import "strings"
}
domain: string | *#Platform.org.domain
// authProxyPrefix is the path routed to the ext auth proxy.
authProxyPrefix: string | *"/holos/oidc"
// authProxyOrgDomain is the primary org domain for zitadel.
authProxyOrgDomain: string | *#Platform.org.domain
// authProxyIssuer is the issuer url
@@ -60,6 +61,8 @@ import "strings"
// features is YAGNI maybe?
features: [Name=string]: #Feature & {name: Name}
features: authproxy: _
features: httpbin: _
}
// #Cluster defines a cluster
@@ -117,12 +120,14 @@ import "strings"
stageSegments: [...string] | *[name]
// authProxyClientID is the ClientID registered with the oidc issuer.
authProxyClientID: string
// extAuthzProviderName is the provider name in the mesh config
extAuthzProviderName: "\(slug)-authproxy"
}
#Feature: {
name: string
description: string
enabled: *true | false
enabled: true | *false
}
#ProjectTemplate: {

View File

@@ -13,6 +13,8 @@ import (
crt "cert-manager.io/certificate/v1"
gw "networking.istio.io/gateway/v1beta1"
vs "networking.istio.io/virtualservice/v1beta1"
ra "security.istio.io/requestauthentication/v1"
ap "security.istio.io/authorizationpolicy/v1"
pg "postgres-operator.crunchydata.com/postgrescluster/v1beta1"
)
@@ -63,19 +65,21 @@ _apiVersion: "holos.run/v1alpha1"
#ClusterRoleBinding: #ClusterObject & rbacv1.#ClusterRoleBinding
#ClusterIssuer: #ClusterObject & ci.#ClusterIssuer & {...}
#Issuer: #NamespaceObject & is.#Issuer
#Role: #NamespaceObject & rbacv1.#Role
#RoleBinding: #NamespaceObject & rbacv1.#RoleBinding
#ConfigMap: #NamespaceObject & corev1.#ConfigMap
#ServiceAccount: #NamespaceObject & corev1.#ServiceAccount
#Pod: #NamespaceObject & corev1.#Pod
#Service: #NamespaceObject & corev1.#Service
#Job: #NamespaceObject & batchv1.#Job
#CronJob: #NamespaceObject & batchv1.#CronJob
#Deployment: #NamespaceObject & appsv1.#Deployment
#VirtualService: #NamespaceObject & vs.#VirtualService
#Certificate: #NamespaceObject & crt.#Certificate
#PostgresCluster: #NamespaceObject & pg.#PostgresCluster
#Issuer: #NamespaceObject & is.#Issuer
#Role: #NamespaceObject & rbacv1.#Role
#RoleBinding: #NamespaceObject & rbacv1.#RoleBinding
#ConfigMap: #NamespaceObject & corev1.#ConfigMap
#ServiceAccount: #NamespaceObject & corev1.#ServiceAccount
#Pod: #NamespaceObject & corev1.#Pod
#Service: #NamespaceObject & corev1.#Service
#Job: #NamespaceObject & batchv1.#Job
#CronJob: #NamespaceObject & batchv1.#CronJob
#Deployment: #NamespaceObject & appsv1.#Deployment
#VirtualService: #NamespaceObject & vs.#VirtualService
#RequestAuthentication: #NamespaceObject & ra.#RequestAuthentication
#AuthorizationPolicy: #NamespaceObject & ap.#AuthorizationPolicy
#Certificate: #NamespaceObject & crt.#Certificate
#PostgresCluster: #NamespaceObject & pg.#PostgresCluster
#Gateway: #NamespaceObject & gw.#Gateway & {
metadata: namespace: string | *"istio-ingress"
@@ -108,7 +112,7 @@ _apiVersion: "holos.run/v1alpha1"
_name: string
metadata: {
name: _name
namespace: #TargetNamespace
namespace: string | *#TargetNamespace
}
spec: {
refreshInterval: string | *"1h"
@@ -221,6 +225,32 @@ _apiVersion: "holos.run/v1alpha1"
services: [ID=_]: {
name: string & ID
}
// authproxy configures the auth proxy attached to the default ingress gateway in the istio-ingress namespace.
authproxy: #AuthProxySpec & {
namespace: "istio-ingress"
provider: "ingressauth"
}
}
#AuthProxySpec: {
// projectID is the zitadel project resource id.
projectID: number
// clientID is the zitadel application client id.
clientID: string
// namespace is the namespace
namespace: string
// provider is the istio extension provider name in the mesh config.
provider: string
// orgDomain is the zitadel organization domain for logins.
orgDomain: string | *#Platform.org.domain
// issuerHost is the Host: header value of the oidc issuer
issuerHost: string | *"login.\(#Platform.org.domain)"
// issuer is the oidc identity provider issuer url
issuer: string | *"https://\(issuerHost)"
// path is the oauth2-proxy --proxy-prefix value. The default callback url is the Host: value with a path of /holos/oidc/callback
proxyPrefix: string | *"/holos/authproxy/\(namespace)"
// idTokenHeader represents the header where the id token is placed
idTokenHeader: string | *"x-oidc-id-token"
}
// ManagedNamespace is a namespace to manage across all clusters in the holos platform.

View File

@@ -1 +1 @@
61
62

View File

@@ -1 +1 @@
4
1