Compare commits

...

31 Commits

Author SHA1 Message Date
tym83
ba3ecf6893 docs: add SECURITY.md
Signed-off-by: tym83 <6355522@gmail.com>
2026-03-17 02:57:52 +05:00
Andrei Kvapil
9fb9354fd2 [kubernetes] Fix CiliumNetworkPolicy endpointSelector for multi-node RWX volumes (#2227)
## What this PR does

When an NFS-backed RWX volume is published to multiple VMs, the
`CiliumNetworkPolicy` `endpointSelector.matchLabels` only included the
first VM. Subsequent `ControllerPublishVolume` calls added
`ownerReferences` but never broadened the selector, causing Cilium to
block NFS egress — mounts hang on all nodes except the first.

This PR switches from `matchLabels` to `matchExpressions` (`operator:
In`) so the selector can list multiple VM names, and rebuilds it
whenever ownerReferences are added or removed.

### Release note

```release-note
[kubernetes] Fixed CiliumNetworkPolicy endpointSelector not being updated when NFS-backed RWX volumes are published to multiple VMs, which caused NFS mounts to hang on all nodes except the first.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* KubeVirt CSI driver now supports selecting and targeting multiple
virtual machines for volume publishing.

* **Improvements**
* Network policy targets are rebuilt automatically when VM ownership
references change, improving correctness and lifecycle handling in
multi-VM scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-16 17:21:46 +01:00
Andrei Kvapil
23bc8525be [monitoring] Add inlineScrapeConfig support to tenant vmagent (#2200)
## Summary

- Adds `inlineScrapeConfig` support to the tenant `VMAgent` resource in
`packages/system/monitoring/templates/vm/vmagent.yaml`
- Adds commented usage example in both
`packages/system/monitoring/values.yaml` and
`packages/extra/monitoring/values.yaml`

## Problem

The tenant VMAgent resource does not support custom scrape
configurations. Using `additionalScrapeConfigs` (which references a
Kubernetes Secret) is not viable because tenant users have no access to
create or read Secrets — they can only manage resources under
`apps.cozystack.io`. Instead, `inlineScrapeConfig` accepts inline YAML
directly through Helm values, which is consistent with tenant
permissions.

Relates to #2194

## Usage

```yaml
vmagent:
  inlineScrapeConfig: |
    - job_name: "custom"
      static_configs:
        - targets: ["my-service:9090"]
```

## Test plan

- [ ] `helm template` monitoring with `inlineScrapeConfig` set — block
rendered
- [ ] `helm template` monitoring without it — no `inlineScrapeConfig` in
output
- [ ] Deploy and verify custom scrape targets are picked up by vmagent

```release-note
Add inlineScrapeConfig support to tenant vmagent
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Support for including an inline scrape configuration into VMAgent
monitoring when provided.

* **Documentation**
* Added a commented example showing how to supply an inline scrape job
and targets.
* Reordered remote-write URL entries in monitoring configuration for
clearer ordering.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-16 14:56:15 +01:00
Andrei Kvapil
3fef40e9f7 [dashboard] Add keycloakInternalUrl for backend-to-backend OIDC requests (#2224)
## What this PR does

Adds `authentication.oidc.keycloakInternalUrl` platform value that
allows oauth2-proxy
in the dashboard to route backend calls (token exchange, JWKS, userinfo,
logout) through
an internal cluster URL while keeping browser redirects on the external
Keycloak URL.

When set, oauth2-proxy uses `--skip-oidc-discovery` and explicit
endpoint URLs pointing
to the internal Keycloak service. This avoids external DNS lookups and
TLS overhead for
pod-to-pod communication.

Fully backward-compatible: when the value is empty (default), behavior
is unchanged.

### Release note

```release-note
[dashboard] Added `authentication.oidc.keycloakInternalUrl` platform value to route oauth2-proxy backend requests through internal Keycloak service URL, bypassing external DNS and TLS.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added configuration support for an internal KeyCloak URL, enabling
backend authentication requests to be routed through an alternative
endpoint while maintaining existing external URLs for browser
interactions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-16 14:54:41 +01:00
mattia-eleuteri
cc5ec0b7a3 [kubernetes] Fix CiliumNetworkPolicy endpointSelector for multi-node RWX volumes
When an NFS-backed RWX volume is published to multiple VMs, the
CiliumNetworkPolicy egress rule only allowed traffic from the first VM.
The endpointSelector.matchLabels was set once on creation and never
broadened, causing NFS mounts to hang on all nodes except the first.

Switch from matchLabels to matchExpressions (operator: In) so the
selector can list multiple VM names. Rebuild the selector whenever
ownerReferences are added or removed.

Signed-off-by: mattia-eleuteri <mattia@hidora.io>
2026-03-16 14:45:04 +01:00
Andrey Kolkov
38d58a77dc Fix/backups (#2149)
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
[backups] Added fix to roles and changed backupstrategy-controller location
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Updated backup controller namespace configuration.
* Enhanced backup controller permissions for leader election and event
recording capabilities.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-16 17:31:06 +04:00
Andrei Kvapil
47dbbb9538 feat(linstor): add linstor-csi patches for clone/snapshot relocation (#2133)
## Summary

- Remove merged RWX validation patch (`001-rwx-validation.diff`)
- Add new patch (`001-relocate-after-clone-restore.diff`) that includes:
  - Randomized node selection for snapshot restore
- `linstor.csi.linbit.com/relocateAfterClone` StorageClass parameter to
relocate replicas to optimal nodes after clone
- `snap.linstor.csi.linbit.com/relocate-after-restore`
VolumeSnapshotClass parameter to relocate replicas to optimal nodes
after snapshot restore
- Both parameters are **disabled by default**
- Placing the snapshot restore parameter in VolumeSnapshotClass prevents
unwanted relocation when Velero creates temporary PVCs during data mover
backup

Upstream PRs:
- https://github.com/piraeusdatastore/linstor-csi/pull/418
- https://github.com/piraeusdatastore/linstor-csi/pull/419

## Test plan

- [x] Clone a PVC and verify relocation logic executes
- [x] Restore a PVC from snapshot and verify replicas get migrated to
optimal nodes
- [x] Verified on dev5 cluster (3-node) — snapshot restore triggered
actual migration (node0 → node2)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **New Features**
* Automatic resource relocation after clone and restore operations
optimizes storage placement and load balancing across nodes.
* RWX block attachment validation with optional disable flag ensures
proper multi-pod access control.

* **Chores**
  * Updated CDI clone strategy to use CSI-clone for improved efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-16 13:04:24 +01:00
Andrei Kvapil
37050922f2 docs: add changelog for v1.1.2 (#2220)
This PR adds the changelog for release `v1.1.2`.

 Changelog has been automatically generated in
`docs/changelogs/v1.1.2.md`.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
  * Fixed S3 Manager endpoint alignment with BucketInfo secrets
  * Resolved spurious OpenAPI post-processing errors on startup

* **Documentation**
  * Added troubleshooting guidance for DependenciesNotReady
  * Enhanced installation documentation with Ansible guide
  * Added CA rotation operations documentation
  * Improved backup and recovery guidance
  * Expanded metrics and architecture references
  * Reorganized operator-first installation guidance

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-16 09:38:53 +01:00
Kirill Ilin
689c2a5e4a feat(dashboard): add keycloakInternalUrl for backend-to-backend OIDC requests
When set, oauth2-proxy skips OIDC discovery and routes all backend calls
(token exchange, JWKS, userinfo, logout) through the internal cluster URL
while keeping browser redirects on the external URL. This avoids external
DNS lookups and TLS overhead for pod-to-pod communication with Keycloak.

Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
2026-03-16 11:03:55 +05:00
cozystack-bot
7e0a059d34 docs: add changelog for v1.1.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-03-16 01:43:05 +00:00
Andrei Kvapil
ee8533647b docs: add changelog for v1.0.5 (#2221)
This PR adds the changelog for release `v1.0.5`.

 Changelog has been automatically generated in
`docs/changelogs/v1.0.5.md`.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Resolved spurious error messages in OpenAPI post-processing for
certain configurations.

* **Documentation**
  * Enhanced troubleshooting guides and installation instructions.
  * Expanded operational procedures for backups and CA rotation.
* Added custom metrics collection guidance and architecture
documentation.
  * Completed comprehensive v1 documentation refresh.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-13 18:30:18 +01:00
cozystack-bot
b3f356a5ed docs: add changelog for v1.0.5
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-03-13 15:41:31 +00:00
Andrei Kvapil
ffd6e628e2 fix(api): skip OpenAPI post-processor for non-apps group versions (#2212)
## What this PR does

The OpenAPI `PostProcessSpec` callback is invoked for every registered
group-version (apps, core, version, etc.), but the Application schema
cloning logic only applies to `apps.cozystack.io`. When called for other
GVs the base Application schemas are absent, producing a spurious error
on every API server start:

```
ERROR klog Failed to build OpenAPI v3 for group version, "base Application* schemas not found"
```

This PR changes the post-processor (both v2 and v3) to return early
when the base schemas are not found, instead of returning an error.

### Release note

```release-note
[platform] Fix spurious "base Application* schemas not found" error logged on cozystack-api startup
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved error handling for missing OpenAPI schema components. The
system now gracefully continues processing instead of halting when
certain base schemas are unavailable.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-13 16:23:18 +01:00
Andrei Kvapil
22f2e4f82a [bucket] Fix s3manager endpoint mismatch with COSI credentials (#2211)
## What this PR does

Fixes s3manager UI deployment to use the actual S3 endpoint from
BucketInfo (COSI) instead of constructing it from the tenant namespace
host.

The deployment was using `s3.<tenant>.<cluster-domain>` while
credentials issued by COSI point to the root-level S3 endpoint. This
mismatch caused "invalid credentials" errors on login even with correct
credentials from the bucket secret.

Falls back to the constructed namespace host on first deploy before
BucketAccess secrets exist.

### Release note

```release-note
[bucket] Fix s3manager endpoint mismatch causing "invalid credentials" errors in login mode
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Deployment configuration now supports per-user endpoint customization.
Endpoints are dynamically retrieved from account-specific settings,
enabling flexible configurations while maintaining backward
compatibility for standard deployments without custom settings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-13 16:23:03 +01:00
Andrei Kvapil
39df52542b [kubernetes] Fixed k8s<1.32 creation (#2209)
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does

This PR adds version specific ubuntu base images to fix errors when base
image has new deb packages of kubeadm and kubelet installed, but at
runtime it was downgraded by replacing just binaries. Now update by
replacing binaries works as intended - latest patch version of minor
version used.

Core issue was in kubeadm<1.32 expecting conntrack binary in its
preflight checks but it was not found. It happened because kubelet deb
package dropped conntrack dependency since 1.32 (actually it absent in
1.31.14 too).
So now status of supported tenant k8s versions is:
- 1.30 - works because kubelet package provided conntrack, also
conntrack preflight check ignored (see 1.31).
- 1.31 - works because conntrack preflight check ignored (for some
reason kubelet 1.31.14 did't provide conntrack dependency, unlike
1.31.13 did).
- \>=1.32 - works because conntrack preflight check removed from
`kubeadm init` entirely.

Conntrack preflight check ignoring is legit for tenant kubernetes
clusters because until 1.32 it was used in kube-proxy but cozystack k8s
approach doesn't use kube-proxy (replaced with cilium).

Issue with conntrack may be mitigated with only `ignorePreflightErrors`,
but I think proper base image build will help to avoid similar bugs in
future.

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
[kubernetes] Fixed tenant k8s older than 1.32 creation by adding version specific ubuntu base images
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added multi-version Kubernetes support with version-specific container
images.
* Enhanced compatibility with newer Kubernetes releases, including
version 1.31.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-13 08:46:32 +01:00
Andrei Kvapil
2b60c010dd Revert "fix(operator): requeue packages when dependencies are not ready"
This reverts commit f906a0d8ad.

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-13 00:56:27 +01:00
Andrei Kvapil
f906a0d8ad fix(operator): requeue packages when dependencies are not ready
When dependencies are not ready the reconciler returned without
requeueing, relying solely on watch events to re-trigger. If a watch
event was missed (controller restart, race condition, dependency already
ready before watch setup), the package would stay stuck in
DependenciesNotReady forever.

Add RequeueAfter: 30s so dependencies are periodically rechecked.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-13 00:31:56 +01:00
Andrei Kvapil
ee83aaa82e fix(api): skip OpenAPI post-processor for non-apps group versions
The OpenAPI PostProcessSpec callback is invoked for every group-version
(apps, core, version, etc.), but the Application schema cloning logic
only applies to apps.cozystack.io. When called for other GVs the base
Application schemas are absent, causing a spurious error log on every
API server start.

Return early instead of erroring when the base schemas are not found.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-12 23:58:55 +01:00
IvanHunters
f647cfd7b9 [bucket] Fix s3manager endpoint to use actual S3 endpoint from BucketInfo
The deployment template was constructing the S3 endpoint from the tenant's
namespace host (e.g. s3.freedom.infra.example.com), while COSI credentials
are issued for the actual SeaweedFS endpoint (e.g. s3.infra.example.com).
This mismatch caused 'invalid credentials' errors when users tried to log
in with valid credentials from the bucket secret.

Now the endpoint is resolved from BucketInfo (same source as credentials),
with a fallback to the constructed namespace host for first-time deploys
before BucketAccess secrets are created.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-13 01:08:13 +03:00
Andrei Kvapil
450e2e8ec4 refactor(linstor): hardcode relocate defaults, remove SC/VSC parameters
Hardcode relocateAfterClone=true and relocateAfterRestore=false as
defaults in the CSI driver patch instead of exposing them via
StorageClass/VolumeSnapshotClass parameters.

Remove the extra linstor-snapshots-ephemeral VolumeSnapshotClass (Velero)
and the relocateAfterRestore parameter from the default VolumeSnapshotClass.

This is a temporary measure while upstream linstor-server is deciding on
the interface for the rebalance feature (see LINBIT/linstor-server#487).
Once upstream provides native rebalance support, these hardcoded defaults
will be replaced by the proper upstream mechanism.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-12 13:51:02 +01:00
Andrei Kvapil
941fb02cd1 [cozystack-scheduler] Add custom scheduler as an optional system package (#2205)
## What this PR does

Adds the cozystack-scheduler as an optional system package, vendored
from https://github.com/cozystack/cozystack-scheduler. The scheduler
extends the default kube-scheduler with SchedulingClass-aware affinity
plugins, allowing platform operators to define cluster-wide scheduling
constraints via a SchedulingClass CRD. Pods opt in via the
`scheduler.cozystack.io/scheduling-class` annotation.

The package includes:
- Helm chart with RBAC, ConfigMap, Deployment, and CRD
- PackageSource definition for the cozystack package system
- Optional inclusion in the platform system bundle

### Release note

```release-note
[cozystack-scheduler] Add cozystack-scheduler as an optional system
package. The custom scheduler supports SchedulingClass CRDs for
cluster-wide node affinity, pod affinity, and topology spread constraints.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added cozystack-scheduler component as an optional system package.
* Introduced SchedulingClass custom resource for advanced scheduling
configurations.
* Scheduler supports node affinity, pod affinity, pod anti-affinity, and
topology spread constraints.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-12 13:38:25 +01:00
Myasnikov Daniil
93992ef00b [backups] Added fix to roles and backupstrategy-controller location
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-03-12 13:44:16 +05:00
Myasnikov Daniil
f82f13bf32 [kubernetes] Fixed k8s<1.32 creation
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-03-12 11:11:25 +05:00
Mattia Eleuteri
f5d8c89ddf [monitoring] Add inlineScrapeConfig support to tenant vmagent
Signed-off-by: Mattia Eleuteri <mattia.eleuteri@hidora.io>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
2026-03-11 10:40:20 +01:00
Timofei Larkin
1dd27f6b23 [cozystack-scheduler] Add custom scheduler as an optional system package
## What this PR does

Adds the cozystack-scheduler as an optional system package, vendored from
https://github.com/cozystack/cozystack-scheduler. The scheduler extends
the default kube-scheduler with SchedulingClass-aware affinity plugins,
allowing platform operators to define cluster-wide scheduling constraints
via a SchedulingClass CRD. Pods opt in via the
`scheduler.cozystack.io/scheduling-class` annotation.

The package includes:
- Helm chart with RBAC, ConfigMap, Deployment, and CRD
- PackageSource definition for the cozystack package system
- Optional inclusion in the platform system bundle

### Release note

```release-note
[cozystack-scheduler] Add cozystack-scheduler as an optional system
package. The custom scheduler supports SchedulingClass CRDs for
cluster-wide node affinity, pod affinity, and topology spread constraints.
```

Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-03-10 22:43:41 +03:00
Andrei Kvapil
37a6816492 style(linstor): regenerate patch with gofmt fix
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-10 12:42:02 +01:00
Andrei Kvapil
1c2c9f723a fix(linstor): use camelCase for relocateAfterRestore parameter
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-10 12:42:02 +01:00
Andrei Kvapil
d4abb86092 feat(linstor): configure VolumeSnapshotClasses for relocation and Velero
Enable relocate-after-restore on the default linstor-snapshots class
so that PVCs restored from snapshots get replicas relocated to optimal
nodes.

Add a separate linstor-snapshots-velero class with the Velero
annotation and without relocation, so Velero's temporary data mover
PVCs are not relocated unnecessarily.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-10 12:42:01 +01:00
Andrei Kvapil
42778cf4b1 feat(cdi): switch clone strategy to csi-clone
Use CSI clone instead of host-assisted copy for CDI DataVolume
cloning. This leverages LINSTOR's native rd-clone mechanism which
is significantly faster than pod-based data copying, and works
together with the new relocateAfterClone parameter.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-10 12:42:01 +01:00
Andrei Kvapil
374e12e1c4 refactor(linstor): move relocate-after-restore to VolumeSnapshotClass
Move snapshot restore relocation parameter from StorageClass to
VolumeSnapshotClass to avoid unwanted relocation during Velero
data mover backup. Change relocateAfterClone default to false.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-10 12:42:01 +01:00
Andrei Kvapil
4dcba2fe5a feat(linstor): update linstor-csi patches
Remove merged RWX validation patch and add new patch that includes:
- Randomized node selection for snapshot restore
- Relocate replicas to optimal nodes after clone and snapshot restore

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-03-10 12:42:01 +01:00
39 changed files with 1981 additions and 745 deletions

102
SECURITY.md Normal file
View File

@@ -0,0 +1,102 @@
# Security Policy
## Scope
This policy applies to the [`cozystack/cozystack`](https://github.com/cozystack/cozystack) repository and to release artifacts produced from it, including Cozystack core components, operators, packaged manifests, container images, and installation assets published by the project.
Cozystack integrates and ships many upstream cloud native components. If you believe a vulnerability originates in an upstream project rather than in Cozystack-specific code, packaging, defaults, or integration logic, please report it to the upstream project as well. If you are unsure, report it to Cozystack first and we will help route or coordinate the issue.
## Supported Versions
As of March 17, 2026, the Cozystack project maintains multiple release lines. Security fixes are prioritized for the latest stable release line and, when needed, backported to other supported lines.
| Version line | Status | Notes |
| --- | --- | --- |
| `v1.1.x` | Supported | Current stable release line. |
| `v1.0.x` | Supported | Previous stable release line; receives security and important maintenance fixes. |
| `v0.41.x` | Limited support | Legacy pre-v1 line during the v0 to v1 transition; critical security and upgrade-blocking fixes may be backported at maintainer discretion. |
| `< v0.41` | Not supported | Please upgrade to a supported release line before requesting a security fix. |
| `alpha`, `beta`, `rc` releases | Not supported | Pre-release builds are for testing and evaluation only. |
Supported versions may change over time as new release lines are cut. The authoritative source for current releases is the GitHub Releases page:
<https://github.com/cozystack/cozystack/releases>
## Reporting a Vulnerability
Please do **not** report security vulnerabilities through public GitHub issues, discussions, pull requests, Telegram, Slack, or other public community channels.
At the moment, this repository does not publish a dedicated private security mailbox in-tree. If you need to report a vulnerability:
1. Contact one of the project maintainers listed in `CODEOWNERS` using an existing private channel you already have.
2. If you do not already have a private maintainer contact, use a public community channel only to request a private contact path, without disclosing any vulnerability details.
Please do not include exploit details, credentials, tokens, private keys, customer data, or other sensitive material in any public message.
When reporting a vulnerability, please include as much of the following as possible:
- affected Cozystack version, tag, or commit
- affected component or package, for example operator, API server, dashboard, installer, or a packaged system component
- deployment environment and provider, for example bare metal, Hetzner, Oracle Cloud, or other infrastructure
- prerequisites and exact reproduction steps
- impact, attack scenario, and expected blast radius
- whether authentication, tenant access, cluster-admin access, or network adjacency is required
- known mitigations or workarounds
- whether you believe the issue also affects an upstream dependency
## What to Expect
The maintainers will aim to:
- acknowledge receipt within 3 business days
- perform an initial triage and severity assessment within 7 business days
- keep the reporter informed as the fix and disclosure plan are developed
Resolution timelines depend on severity, complexity, release branch applicability, and whether coordination with upstream projects is required.
## Disclosure Process
The Cozystack project follows a coordinated disclosure model.
- We ask reporters to keep details private until a fix or mitigation is available and users have had a reasonable opportunity to upgrade.
- When appropriate, maintainers may use GitHub Security Advisories or equivalent coordinated disclosure tooling to manage remediation and public disclosure.
- If appropriate, the project may request or publish a GHSA and/or CVE as part of the disclosure process.
- Fixes will normally be released in the supported version lines affected by the issue, subject to severity and feasibility.
Public disclosure will typically happen through one or more of the following:
- GitHub Releases and release notes
- project changelogs and documentation updates
- GitHub Security Advisories, when used for coordinated disclosure
## Project Security Practices
Security is part of the normal Cozystack development and release process. Current project practices include:
- maintainer-owned review through pull requests and `CODEOWNERS`
- automated pull request checks, including pre-commit validation, unit tests, builds, and end-to-end testing
- release automation with patch releases, release branches, and backport workflows
- ongoing maintenance of packaged dependencies and platform integrations across supported release lines
Because Cozystack is an integration-heavy platform, some vulnerabilities may require coordination across multiple repositories or with upstream maintainers before a public fix can be released.
## Security Fixes and Announcements
Security fixes are published in normal release artifacts whenever possible. Users should monitor:
- GitHub Releases: <https://github.com/cozystack/cozystack/releases>
- project changelogs in this repository
- the Cozystack website and documentation: <https://cozystack.io>
## Out of Scope
The following are generally out of scope for private security reporting unless there is a clear Cozystack-specific impact:
- vulnerabilities in unsupported or end-of-life Cozystack versions
- issues that require access already equivalent to cluster-admin, node root, or direct infrastructure administrator privileges, unless they bypass an expected Cozystack security boundary
- vulnerabilities that exist only in an upstream dependency and are not introduced or materially worsened by Cozystack packaging, configuration, or defaults
- requests for security best-practice advice without a concrete vulnerability
## Credits
We appreciate responsible disclosure and will credit reporters in public advisories or release notes unless anonymous disclosure is requested.

29
docs/changelogs/v1.0.5.md Normal file
View File

@@ -0,0 +1,29 @@
<!--
https://github.com/cozystack/cozystack/releases/tag/v1.0.5
-->
## Fixes
* **[api] Fix spurious OpenAPI post-processing errors for non-apps group versions**: The API server no longer logs false errors while generating OpenAPI specs for core and other non-`apps.cozystack.io` group versions. The post-processor now exits early when the base `Application` schemas are absent, reducing noisy startup logs without affecting application schema generation ([**@kvaps**](https://github.com/kvaps) in #2212, #2216).
## Documentation
* **[website] Add `DependenciesNotReady` troubleshooting and correct packages management build target**: Added a troubleshooting guide for packages stuck in `DependenciesNotReady`, including how to inspect operator logs and identify missing dependencies, and fixed the outdated `make image-cozystack` command to `make image-packages` in the packages management guide ([**@kvaps**](https://github.com/kvaps) in cozystack/website#450).
* **[website] Clarify operator-first installation order**: Reordered the platform installation guide and tutorial so users install the Cozystack operator before preparing and applying the Platform Package, matching the rest of the installation docs and reducing setup confusion during fresh installs ([**@sircthulhu**](https://github.com/sircthulhu) in cozystack/website#449).
* **[website] Add automated installation guide for Ansible**: Added end-to-end documentation for deploying Cozystack with the `cozystack.installer` Ansible collection, including inventory examples, distro-specific playbooks, configuration reference, verification steps, and explicit version pinning guidance to help operators automate installs safely ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#442).
* **[website] Expand CA rotation operations guide**: Completed the CA rotation documentation with separate Talos and Kubernetes certificate rotation procedures, dry-run preview steps, and post-rotation guidance for fetching updated `talosconfig` and `kubeconfig` files after certificate changes ([**@kvaps**](https://github.com/kvaps) in cozystack/website#406).
* **[website] Improve backup operations documentation**: Enhanced the operator backup and recovery guide with clearer Velero enablement steps, concrete provider and bucket examples, and more useful commands for inspecting backups, schedules, restores, CRD status, and logs ([**@androndo**](https://github.com/androndo) in cozystack/website#440).
* **[website] Add custom metrics collection guide**: Added a monitoring guide showing how tenants can expose their own Prometheus exporters through `VMServiceScrape` and `VMPodScrape`, including namespace labeling requirements, example manifests, verification steps, and troubleshooting advice ([**@IvanHunters**](https://github.com/IvanHunters) in cozystack/website#444).
* **[website] Document PackageSource and Package architecture**: Added a Key Concepts reference covering `PackageSource` and `Package` reconciliation flow, dependency handling, update propagation, rollback behavior, FluxPlunger recovery, and the `cozypkg` CLI for package management ([**@IvanHunters**](https://github.com/IvanHunters) in cozystack/website#445).
* **[website] Refresh v1 application and platform documentation**: Fixed the documentation auto-update flow and published a broad v1 documentation refresh covering newly documented applications, updated naming and navigation, virtualization and platform content updates, and reorganized versioned docs pages ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#439).
---
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.0.4...v1.0.5

25
docs/changelogs/v1.1.2.md Normal file
View File

@@ -0,0 +1,25 @@
<!--
https://github.com/cozystack/cozystack/releases/tag/v1.1.2
-->
## Fixes
* **[bucket] Fix S3 Manager endpoint mismatch with COSI credentials**: The S3 Manager UI previously constructed an `s3.<tenant>.<cluster-domain>` endpoint even though COSI-issued bucket credentials point to the root-level S3 endpoint. This caused login failures with "invalid credentials" despite valid secrets. The deployment now uses the actual endpoint from `BucketInfo`, with the old namespace-based endpoint kept only as a fallback before `BucketAccess` secrets exist ([**@IvanHunters**](https://github.com/IvanHunters) in #2211, #2215).
* **[platform] Fix spurious OpenAPI post-processing errors on cozystack-api startup**: The OpenAPI post-processor was being invoked for non-`apps.cozystack.io` group versions where the base `Application*` schemas do not exist, producing noisy startup errors on every API server launch. It now skips those non-apps group versions gracefully instead of returning an error ([**@kvaps**](https://github.com/kvaps) in #2212, #2217).
## Documentation
* **[website] Add troubleshooting for packages stuck in `DependenciesNotReady`**: Added an operations guide that explains how to diagnose missing package dependencies in operator logs and corrected the packages management development docs to use the current `make image-packages` target ([**@kvaps**](https://github.com/kvaps) in cozystack/website#450).
* **[website] Reorder installation docs to install the operator before the platform package**: Updated the platform installation guide and tutorial so the setup sequence consistently installs the Cozystack operator first, then prepares and applies the Platform Package, matching the rest of the documentation set ([**@sircthulhu**](https://github.com/sircthulhu) in cozystack/website#449).
* **[website] Add automated installation guide for the Ansible collection**: Added a full guide for deploying Cozystack with the `cozystack.installer` collection, including inventory examples, distro-specific playbooks, configuration reference, and explicit version pinning guidance ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#442).
* **[website] Expand monitoring and platform architecture reference docs**: Added a tenant custom metrics collection guide for `VMServiceScrape` and `VMPodScrape`, and documented `PackageSource`/`Package` architecture, reconciliation flow, rollback behavior, and the `cozypkg` workflow in Key Concepts ([**@IvanHunters**](https://github.com/IvanHunters) in cozystack/website#444, cozystack/website#445).
* **[website] Improve operations guides for CA rotation and Velero backups**: Completed the CA rotation documentation with dry-run and post-rotation credential retrieval steps, and expanded the backup configuration guide with concrete examples, verification commands, and clearer operator procedures ([**@kvaps**](https://github.com/kvaps) in cozystack/website#406; [**@androndo**](https://github.com/androndo) in cozystack/website#440).
---
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.1.1...v1.1.2

View File

@@ -1,4 +1,4 @@
KUBERNETES_VERSION = v1.35
KUBERNETES_VERSIONS = $(shell awk -F'"' '{print $$2}' files/versions.yaml)
KUBERNETES_PKG_TAG = $(shell awk '$$1 == "version:" {print $$2}' Chart.yaml)
include ../../../hack/common-envs.mk
@@ -15,17 +15,19 @@ update:
image: image-ubuntu-container-disk image-kubevirt-cloud-provider image-kubevirt-csi-driver image-cluster-autoscaler
image-ubuntu-container-disk:
docker buildx build images/ubuntu-container-disk \
--build-arg KUBERNETES_VERSION=${KUBERNETES_VERSION} \
--tag $(REGISTRY)/ubuntu-container-disk:$(call settag,$(KUBERNETES_VERSION)) \
--tag $(REGISTRY)/ubuntu-container-disk:$(call settag,$(KUBERNETES_VERSION)-$(TAG)) \
--cache-from type=registry,ref=$(REGISTRY)/ubuntu-container-disk:latest \
--cache-to type=inline \
--metadata-file images/ubuntu-container-disk.json \
$(BUILDX_ARGS)
echo "$(REGISTRY)/ubuntu-container-disk:$(call settag,$(KUBERNETES_VERSION))@$$(yq e '."containerimage.digest"' images/ubuntu-container-disk.json -o json -r)" \
> images/ubuntu-container-disk.tag
rm -f images/ubuntu-container-disk.json
$(foreach ver,$(KUBERNETES_VERSIONS), \
docker buildx build images/ubuntu-container-disk \
--build-arg KUBERNETES_VERSION=$(ver) \
--tag $(REGISTRY)/ubuntu-container-disk:$(call settag,$(ver)) \
--tag $(REGISTRY)/ubuntu-container-disk:$(call settag,$(ver)-$(TAG)) \
--cache-from type=registry,ref=$(REGISTRY)/ubuntu-container-disk:$(call settag,$(ver)) \
--cache-to type=inline \
--metadata-file images/ubuntu-container-disk-$(ver).json \
$(BUILDX_ARGS) && \
echo "$(REGISTRY)/ubuntu-container-disk:$(call settag,$(ver))@$$(yq e '."containerimage.digest"' images/ubuntu-container-disk-$(ver).json -o json -r)" \
> images/ubuntu-container-disk-$(ver).tag && \
rm -f images/ubuntu-container-disk-$(ver).json; \
)
image-kubevirt-cloud-provider:
docker buildx build images/kubevirt-cloud-provider \

View File

@@ -317,11 +317,7 @@ func (w *WrappedControllerService) ControllerPublishVolume(ctx context.Context,
"ownerReferences": []interface{}{vmiOwnerRef},
},
"spec": map[string]interface{}{
"endpointSelector": map[string]interface{}{
"matchLabels": map[string]interface{}{
"kubevirt.io/vm": vmName,
},
},
"endpointSelector": buildEndpointSelector([]string{vmName}),
"egress": []interface{}{
map[string]interface{}{
"toEndpoints": []interface{}{
@@ -441,6 +437,13 @@ func (w *WrappedControllerService) addCNPOwnerReference(ctx context.Context, nam
if err := unstructured.SetNestedSlice(existing.Object, ownerRefs, "metadata", "ownerReferences"); err != nil {
return status.Errorf(codes.Internal, "failed to set ownerReferences: %v", err)
}
// Rebuild endpointSelector to include all VMs
selector := buildEndpointSelector(vmNamesFromOwnerRefs(ownerRefs))
if err := unstructured.SetNestedField(existing.Object, selector, "spec", "endpointSelector"); err != nil {
return status.Errorf(codes.Internal, "failed to set endpointSelector: %v", err)
}
if _, err := w.dynamicClient.Resource(ciliumNetworkPolicyGVR).Namespace(namespace).Update(ctx, existing, metav1.UpdateOptions{}); err != nil {
return err
}
@@ -486,6 +489,13 @@ func (w *WrappedControllerService) removeCNPOwnerReference(ctx context.Context,
if err := unstructured.SetNestedSlice(existing.Object, remaining, "metadata", "ownerReferences"); err != nil {
return status.Errorf(codes.Internal, "failed to set ownerReferences: %v", err)
}
// Rebuild endpointSelector from remaining VMs
selector := buildEndpointSelector(vmNamesFromOwnerRefs(remaining))
if err := unstructured.SetNestedField(existing.Object, selector, "spec", "endpointSelector"); err != nil {
return status.Errorf(codes.Internal, "failed to set endpointSelector: %v", err)
}
if _, err := w.dynamicClient.Resource(ciliumNetworkPolicyGVR).Namespace(namespace).Update(ctx, existing, metav1.UpdateOptions{}); err != nil {
return err
}
@@ -494,6 +504,37 @@ func (w *WrappedControllerService) removeCNPOwnerReference(ctx context.Context,
})
}
// buildEndpointSelector returns an endpointSelector using matchExpressions
// so that multiple VMs can be listed in a single selector.
func buildEndpointSelector(vmNames []string) map[string]interface{} {
values := make([]interface{}, len(vmNames))
for i, name := range vmNames {
values[i] = name
}
return map[string]interface{}{
"matchExpressions": []interface{}{
map[string]interface{}{
"key": "kubevirt.io/vm",
"operator": "In",
"values": values,
},
},
}
}
// vmNamesFromOwnerRefs extracts VM names from ownerReferences.
func vmNamesFromOwnerRefs(ownerRefs []interface{}) []string {
var names []string
for _, ref := range ownerRefs {
if refMap, ok := ref.(map[string]interface{}); ok {
if name, ok := refMap["name"].(string); ok {
names = append(names, name)
}
}
}
return names
}
func hasRWXAccessMode(pvc *corev1.PersistentVolumeClaim) bool {
for _, mode := range pvc.Spec.AccessModes {
if mode == corev1.ReadWriteMany {

View File

@@ -0,0 +1 @@
ttl.sh/rjfkdsjflsk/ubuntu-container-disk:v1.30@sha256:8c2276f68beb67edf5bf76d6c97b271dd9303b336e1d5850ae2b91a590c9bb57

View File

@@ -0,0 +1 @@
ttl.sh/rjfkdsjflsk/ubuntu-container-disk:v1.31@sha256:2b631cd227bc9b1bae16de033830e756cd6590b512dc0d2b13367ee626f3e4ca

View File

@@ -0,0 +1 @@
ttl.sh/rjfkdsjflsk/ubuntu-container-disk:v1.32@sha256:600d6ce7df4eaa8cc79c7d6d1b01ecac43e7696beb84eafce752d9210a16455f

View File

@@ -0,0 +1 @@
ttl.sh/rjfkdsjflsk/ubuntu-container-disk:v1.33@sha256:243e55d6f2887a4f6ce8526de52fd083b7b88194d5c7f3eaa51b87efb557ac88

View File

@@ -0,0 +1 @@
ttl.sh/rjfkdsjflsk/ubuntu-container-disk:v1.34@sha256:ad8377d5644ba51729dc69dff4c9f6b4a48957075d054a58c61a45d0bb41f6af

View File

@@ -0,0 +1 @@
ttl.sh/rjfkdsjflsk/ubuntu-container-disk:v1.35@sha256:1c2f2430383a9b9882358c60c194465c1b6092b4aa77536a0343cf74155c0067

View File

@@ -1 +0,0 @@
ghcr.io/cozystack/cozystack/ubuntu-container-disk:v1.35@sha256:39f626c802dd84f95720ffb54fcd80dfb8a58ac280498870d0a1aa30d4252f94

View File

@@ -74,7 +74,7 @@ spec:
volumes:
- name: system
containerDisk:
image: "{{ $.Files.Get "images/ubuntu-container-disk.tag" | trim }}"
image: "{{ $.Files.Get (printf "images/ubuntu-container-disk-%s.tag" $.Values.version) | trim }}"
- name: ephemeral
emptyDisk:
capacity: {{ .group.ephemeralStorage | default "20Gi" }}
@@ -249,6 +249,9 @@ spec:
joinConfiguration:
nodeRegistration:
kubeletExtraArgs: {}
# Ignore this for 1.31
ignorePreflightErrors:
- FileExisting-conntrack
discovery:
bootstrapToken:
apiServerEndpoint: {{ $.Release.Name }}.{{ $.Release.Namespace }}.svc:6443

View File

@@ -18,5 +18,5 @@ spec:
path: system/backupstrategy-controller
install:
privileged: true
namespace: cozy-backupstrategy-controller
namespace: cozy-backup-controller
releaseName: backupstrategy-controller

View File

@@ -0,0 +1,19 @@
---
apiVersion: cozystack.io/v1alpha1
kind: PackageSource
metadata:
name: cozystack.cozystack-scheduler
spec:
sourceRef:
kind: OCIRepository
name: cozystack-packages
namespace: cozy-system
path: /
variants:
- name: default
components:
- name: cozystack-scheduler
path: system/cozystack-scheduler
install:
namespace: kube-system
releaseName: cozystack-scheduler

View File

@@ -26,6 +26,7 @@ stringData:
oidc-enabled: {{ .Values.authentication.oidc.enabled | quote }}
oidc-insecure-skip-verify: {{ .Values.authentication.oidc.insecureSkipVerify | quote }}
extra-keycloak-redirect-uri-for-dashboard: {{ index .Values.authentication.oidc.keycloakExtraRedirectUri | quote }}
keycloak-internal-url: {{ .Values.authentication.oidc.keycloakInternalUrl | quote }}
expose-services: {{ .Values.publishing.exposedServices | join "," | quote }}
expose-ingress: {{ .Values.publishing.ingressName | quote }}
expose-external-ips: {{ .Values.publishing.externalIPs | join "," | quote }}

View File

@@ -156,5 +156,6 @@
{{include "cozystack.platform.package.default" (list "cozystack.bootbox" $) }}
{{- end }}
{{include "cozystack.platform.package.optional.default" (list "cozystack.hetzner-robotlb" $) }}
{{include "cozystack.platform.package.optional.default" (list "cozystack.cozystack-scheduler" $) }}
{{- end }}

View File

@@ -54,6 +54,11 @@ authentication:
enabled: false
insecureSkipVerify: false
keycloakExtraRedirectUri: ""
# Internal URL to access KeyCloak realm for backend-to-backend requests (bypasses external DNS).
# When set, oauth2-proxy uses --skip-oidc-discovery and routes all backend calls (token, jwks,
# userinfo, logout) through this URL while keeping browser redirects on the external URL.
# Example: http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy
keycloakInternalUrl: ""
# Pod scheduling configuration
scheduling:
globalAppTopologySpreadConstraints: ""

View File

@@ -178,3 +178,7 @@ vmagent:
urls:
- http://vminsert-shortterm:8480/insert/0/prometheus
- http://vminsert-longterm:8480/insert/0/prometheus
## inlineScrapeConfig: |
## - job_name: "custom"
## static_configs:
## - targets: ["my-service:9090"]

View File

@@ -14,3 +14,10 @@ rules:
- apiGroups: ["backups.cozystack.io"]
resources: ["backupjobs"]
verbs: ["create", "get", "list", "watch"]
# Leader election (--leader-elect)
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "patch"]

View File

@@ -30,6 +30,10 @@ rules:
- apiGroups: ["velero.io"]
resources: ["backups", "restores"]
verbs: ["create", "get", "list", "watch", "update", "patch"]
# Events from Recorder.Event() calls
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "patch"]
# Leader election (--leader-elect)
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]

View File

@@ -1,3 +1,12 @@
{{- $endpoint := printf "s3.%s" .Values._namespace.host }}
{{- range $name, $user := .Values.users }}
{{- $secretName := printf "%s-%s" $.Values.bucketName $name }}
{{- $existingSecret := lookup "v1" "Secret" $.Release.Namespace $secretName }}
{{- if $existingSecret }}
{{- $bucketInfo := fromJson (b64dec (index $existingSecret.data "BucketInfo")) }}
{{- $endpoint = trimPrefix "https://" (index $bucketInfo.spec.secretS3 "endpoint") }}
{{- end }}
{{- end }}
apiVersion: apps/v1
kind: Deployment
metadata:
@@ -17,6 +26,6 @@ spec:
image: "{{ $.Files.Get "images/s3manager.tag" | trim }}"
env:
- name: ENDPOINT
value: "s3.{{ .Values._namespace.host }}"
value: {{ $endpoint | quote }}
- name: SKIP_SSL_VERIFICATION
value: "true"

View File

@@ -0,0 +1,3 @@
apiVersion: v2
name: cozy-cozystack-scheduler
version: 0.1.0

View File

@@ -0,0 +1,10 @@
export NAME=cozystack-scheduler
export NAMESPACE=kube-system
include ../../../hack/package.mk
update:
rm -rf crds templates values.yaml Chart.yaml
tag=$$(git ls-remote --tags --sort="v:refname" https://github.com/cozystack/cozystack-scheduler | awk -F'[/^]' 'END{print $$3}') && \
curl -sSL https://github.com/cozystack/cozystack-scheduler/archive/refs/tags/$${tag}.tar.gz | \
tar xzvf - --strip 2 cozystack-scheduler-$${tag#*v}/chart

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,9 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cozystack-scheduler
rules:
- apiGroups: ["cozystack.io"]
resources:
- schedulingclasses
verbs: ["get", "list", "watch"]

View File

@@ -0,0 +1,38 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cozystack-scheduler:kube-scheduler
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-scheduler
subjects:
- kind: ServiceAccount
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cozystack-scheduler:volume-scheduler
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:volume-scheduler
subjects:
- kind: ServiceAccount
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cozystack-scheduler
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cozystack-scheduler
subjects:
- kind: ServiceAccount
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}

View File

@@ -0,0 +1,54 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: cozystack-scheduler-config
namespace: {{ .Release.Namespace }}
data:
scheduler-config.yaml: |
apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
leaderElection:
leaderElect: true
resourceNamespace: {{ .Release.Namespace }}
resourceName: cozystack-scheduler
profiles:
- schedulerName: cozystack-scheduler
plugins:
preFilter:
disabled:
- name: InterPodAffinity
- name: NodeAffinity
- name: PodTopologySpread
enabled:
- name: CozystackInterPodAffinity
- name: CozystackNodeAffinity
- name: CozystackPodTopologySpread
- name: CozystackSchedulingClass
filter:
disabled:
- name: InterPodAffinity
- name: NodeAffinity
- name: PodTopologySpread
enabled:
- name: CozystackInterPodAffinity
- name: CozystackNodeAffinity
- name: CozystackPodTopologySpread
- name: CozystackSchedulingClass
preScore:
disabled:
- name: InterPodAffinity
- name: NodeAffinity
- name: PodTopologySpread
enabled:
- name: CozystackInterPodAffinity
- name: CozystackNodeAffinity
- name: CozystackPodTopologySpread
score:
disabled:
- name: InterPodAffinity
- name: NodeAffinity
- name: PodTopologySpread
enabled:
- name: CozystackInterPodAffinity
- name: CozystackNodeAffinity
- name: CozystackPodTopologySpread

View File

@@ -0,0 +1,37 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}
spec:
replicas: {{ .Values.replicas }}
selector:
matchLabels:
app: cozystack-scheduler
template:
metadata:
labels:
app: cozystack-scheduler
spec:
serviceAccountName: cozystack-scheduler
containers:
- name: cozystack-scheduler
image: {{ .Values.image }}
command:
- /cozystack-scheduler
- --config=/etc/kubernetes/scheduler-config.yaml
livenessProbe:
httpGet:
path: /healthz
port: 10259
scheme: HTTPS
initialDelaySeconds: 15
volumeMounts:
- name: config
mountPath: /etc/kubernetes/scheduler-config.yaml
subPath: scheduler-config.yaml
readOnly: true
volumes:
- name: config
configMap:
name: cozystack-scheduler-config

View File

@@ -0,0 +1,40 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cozystack-scheduler:extension-apiserver-authentication-reader
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: cozystack-scheduler:leader-election
namespace: {{ .Release.Namespace }}
rules:
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["create", "get", "list", "update", "watch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leasecandidates"]
verbs: ["create", "get", "list", "update", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cozystack-scheduler:leader-election
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: cozystack-scheduler:leader-election
subjects:
- kind: ServiceAccount
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}

View File

@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: cozystack-scheduler
namespace: {{ .Release.Namespace }}

View File

@@ -0,0 +1,2 @@
image: ghcr.io/cozystack/cozystack/cozystack-scheduler:v0.1.0@sha256:5f7150c82177478467ff80628acb5a400291aff503364aa9e26fc346d79a73cf
replicas: 1

View File

@@ -1,6 +1,7 @@
{{- $host := index .Values._cluster "root-host" }}
{{- $oidcEnabled := index .Values._cluster "oidc-enabled" }}
{{- $oidcInsecureSkipVerify := index .Values._cluster "oidc-insecure-skip-verify" }}
{{- $keycloakInternalUrl := index .Values._cluster "keycloak-internal-url" | default "" }}
apiVersion: apps/v1
kind: Deployment
@@ -55,7 +56,16 @@ spec:
- --http-address=0.0.0.0:8000
- --redirect-url=https://dashboard.{{ $host }}/oauth2/callback
- --oidc-issuer-url=https://keycloak.{{ $host }}/realms/cozy
{{- if $keycloakInternalUrl }}
- --skip-oidc-discovery
- --login-url=https://keycloak.{{ $host }}/realms/cozy/protocol/openid-connect/auth
- --redeem-url={{ $keycloakInternalUrl }}/protocol/openid-connect/token
- --oidc-jwks-url={{ $keycloakInternalUrl }}/protocol/openid-connect/certs
- --validate-url={{ $keycloakInternalUrl }}/protocol/openid-connect/userinfo
- --backend-logout-url={{ $keycloakInternalUrl }}/protocol/openid-connect/logout?id_token_hint={id_token}
{{- else }}
- --backend-logout-url=https://keycloak.{{ $host }}/realms/cozy/protocol/openid-connect/logout?id_token_hint={id_token}
{{- end }}
- --whitelist-domain=keycloak.{{ $host }}
- --email-domain=*
- --pass-access-token=true

View File

@@ -3,7 +3,7 @@ kind: CDI
metadata:
name: cdi
spec:
cloneStrategyOverride: copy
cloneStrategyOverride: csi-clone
config:
{{- with .Values.uploadProxyURL }}
uploadProxyURLOverride: {{ quote . }}

View File

@@ -0,0 +1,358 @@
diff --git a/pkg/client/linstor.go b/pkg/client/linstor.go
index f544493..98e7fde 100644
--- a/pkg/client/linstor.go
+++ b/pkg/client/linstor.go
@@ -181,7 +181,7 @@ func LogLevel(s string) func(*Linstor) error {
func (s *Linstor) ListAllWithStatus(ctx context.Context) ([]volume.VolumeStatus, error) {
var vols []volume.VolumeStatus
- resourcesByName := make(map[string][]lapi.Resource)
+ resourcesByName := make(map[string][]lapi.ResourceWithVolumes)
resDefs, err := s.client.ResourceDefinitions.GetAll(ctx, lapi.RDGetAllRequest{WithVolumeDefinitions: true})
if err != nil {
@@ -194,7 +194,7 @@ func (s *Linstor) ListAllWithStatus(ctx context.Context) ([]volume.VolumeStatus,
}
for i := range allResources {
- resourcesByName[allResources[i].Name] = append(resourcesByName[allResources[i].Name], allResources[i].Resource)
+ resourcesByName[allResources[i].Name] = append(resourcesByName[allResources[i].Name], allResources[i])
}
for _, rd := range resDefs {
@@ -462,6 +462,14 @@ func (s *Linstor) Clone(ctx context.Context, vol, src *volume.Info, params *volu
return err
}
+ if params.RelocateAfterClone {
+ logger.Debug("relocate resources to optimal nodes")
+
+ if err := s.relocateResources(ctx, vol.ID, rGroup.Name); err != nil {
+ logger.WithError(err).Warn("resource relocation failed, volume is still usable")
+ }
+ }
+
logger.Debug("reconcile extra properties")
err = s.client.ResourceDefinitions.Modify(ctx, vol.ID, lapi.GenericPropsModify{OverrideProps: vol.Properties})
@@ -1280,6 +1288,14 @@ func (s *Linstor) VolFromSnap(ctx context.Context, snap *volume.Snapshot, vol *v
return err
}
+ if snapParams != nil && snapParams.RelocateAfterRestore {
+ logger.Debug("relocate resources to optimal nodes")
+
+ if err := s.relocateResources(ctx, vol.ID, rGroup.Name); err != nil {
+ logger.WithError(err).Warn("resource relocation failed, volume is still usable")
+ }
+ }
+
logger.Debug("reconcile extra properties")
err = s.client.ResourceDefinitions.Modify(ctx, vol.ID, lapi.GenericPropsModify{OverrideProps: vol.Properties})
@@ -1470,9 +1486,8 @@ func (s *Linstor) reconcileSnapshotResources(ctx context.Context, snapshot *volu
return fmt.Errorf("snapshot '%s' not deployed on any node", snap.Name)
}
- // Optimize the node we use to restore. It should be one of the preferred nodes, or just the first with a snapshot
- // if no preferred nodes match.
- var selectedNode string
+ // Collect available snapshot nodes, preferring those matching the topology hints.
+ var preferred, available []string
for _, snapNode := range snap.Nodes {
if err := s.NodeAvailable(ctx, snapNode); err != nil {
logger.WithField("selected node candidate", snapNode).WithError(err).Debug("node is not available")
@@ -1480,13 +1495,23 @@ func (s *Linstor) reconcileSnapshotResources(ctx context.Context, snapshot *volu
}
if slices.Contains(preferredNodes, snapNode) {
- // We found a perfect candidate.
- selectedNode = snapNode
- break
- } else if selectedNode == "" {
- // Set a fallback if we have no candidate yet.
- selectedNode = snapNode
+ preferred = append(preferred, snapNode)
}
+
+ available = append(available, snapNode)
+ }
+
+ // Pick a random node from preferred candidates, falling back to any available node.
+ // Randomization distributes restore load across snapshot nodes, preventing all
+ // clones of the same source from concentrating on a single node.
+ candidates := preferred
+ if len(candidates) == 0 {
+ candidates = available
+ }
+
+ var selectedNode string
+ if len(candidates) > 0 {
+ selectedNode = candidates[rand.Intn(len(candidates))]
}
if selectedNode == "" {
@@ -1679,6 +1704,114 @@ func (s *Linstor) reconcileResourcePlacement(ctx context.Context, vol *volume.In
return nil
}
+const propertyRelocationTriggered = "Aux/csi-relocation-triggered"
+
+// relocateResources migrates replicas from their current nodes to the nodes that
+// LINSTOR's autoplacer considers optimal. This is a best-effort, fire-and-forget
+// operation: migrate-disk API is asynchronous and the volume remains usable on its
+// current nodes even if relocation fails.
+func (s *Linstor) relocateResources(ctx context.Context, volID, rgName string) error {
+ logger := s.log.WithFields(logrus.Fields{
+ "volume": volID,
+ "resourceGroup": rgName,
+ })
+
+ // Check if relocation was already triggered (idempotency guard).
+ rd, err := s.client.ResourceDefinitions.Get(ctx, volID)
+ if err != nil {
+ return fmt.Errorf("failed to get resource definition: %w", err)
+ }
+
+ if rd.Props[propertyRelocationTriggered] != "" {
+ logger.Debug("relocation already triggered, skipping")
+ return nil
+ }
+
+ // Get current diskful nodes.
+ resources, err := s.client.Resources.GetAll(ctx, volID)
+ if err != nil {
+ return fmt.Errorf("failed to list resources: %w", err)
+ }
+
+ currentNodes := util.DeployedDiskfullyNodes(resources)
+
+ // Query optimal placement from LINSTOR autoplacer.
+ sizeInfo, err := s.client.ResourceGroups.QuerySizeInfo(ctx, rgName, lapi.QuerySizeInfoRequest{})
+ if err != nil {
+ return fmt.Errorf("failed to query size info: %w", err)
+ }
+
+ if sizeInfo.SpaceInfo == nil || len(sizeInfo.SpaceInfo.NextSpawnResult) == 0 {
+ logger.Debug("no spawn result from query-size-info, skipping relocation")
+ return nil
+ }
+
+ // Build a map of optimal node -> storage pool.
+ optimalPools := make(map[string]string, len(sizeInfo.SpaceInfo.NextSpawnResult))
+ for _, r := range sizeInfo.SpaceInfo.NextSpawnResult {
+ optimalPools[r.NodeName] = r.StorPoolName
+ }
+
+ // Compute nodes to remove (current but not optimal) and nodes to add (optimal but not current).
+ var nodesToRemove, nodesToAdd []string
+
+ for _, node := range currentNodes {
+ if _, ok := optimalPools[node]; !ok {
+ nodesToRemove = append(nodesToRemove, node)
+ }
+ }
+
+ for _, r := range sizeInfo.SpaceInfo.NextSpawnResult {
+ if !slices.Contains(currentNodes, r.NodeName) {
+ nodesToAdd = append(nodesToAdd, r.NodeName)
+ }
+ }
+
+ // Only migrate min(remove, add) pairs.
+ pairs := min(len(nodesToRemove), len(nodesToAdd))
+ if pairs == 0 {
+ logger.Debug("no relocation needed, placement is already optimal")
+ return nil
+ }
+
+ logger.Infof("relocating %d replica(s): %v -> %v", pairs, nodesToRemove[:pairs], nodesToAdd[:pairs])
+
+ // Mark relocation as triggered before starting migrations.
+ err = s.client.ResourceDefinitions.Modify(ctx, volID, lapi.GenericPropsModify{
+ OverrideProps: map[string]string{propertyRelocationTriggered: "true"},
+ })
+ if err != nil {
+ return fmt.Errorf("failed to set relocation property: %w", err)
+ }
+
+ // Migrate each pair sequentially (LINSTOR serializes at the RD level anyway).
+ for i := range pairs {
+ fromNode := nodesToRemove[i]
+ toNode := nodesToAdd[i]
+ storPool := optimalPools[toNode]
+
+ logger := logger.WithFields(logrus.Fields{"from": fromNode, "to": toNode, "storagePool": storPool})
+
+ logger.Info("creating diskless resource on target node")
+
+ err := s.client.Resources.MakeAvailable(ctx, volID, toNode, lapi.ResourceMakeAvailable{})
+ if err != nil {
+ logger.WithError(err).Warn("failed to create diskless on target, skipping this pair")
+ continue
+ }
+
+ logger.Info("initiating migrate-disk")
+
+ err = s.client.Resources.Migrate(ctx, volID, fromNode, toNode, storPool)
+ if err != nil {
+ logger.WithError(err).Warn("migrate-disk failed, skipping this pair")
+ continue
+ }
+ }
+
+ return nil
+}
+
// FindSnapsByID searches the snapshot in the backend
func (s *Linstor) FindSnapsByID(ctx context.Context, id string) ([]*volume.Snapshot, error) {
snapshotId, err := volume.ParseSnapshotId(id)
@@ -2173,7 +2306,7 @@ func (s *Linstor) Status(ctx context.Context, volId string) ([]string, *csi.Volu
"volume": volId,
}).Debug("getting assignments")
- ress, err := s.client.Resources.GetAll(ctx, volId)
+ ress, err := s.client.Resources.GetResourceView(ctx, &lapi.ListOpts{Resource: []string{volId}})
if err != nil {
return nil, nil, fmt.Errorf("failed to list resources for '%s': %w", volId, err)
}
@@ -2261,13 +2394,20 @@ func GetSnapshotRemoteAndReadiness(snap *lapi.Snapshot) (string, bool, error) {
return "", slices.Contains(snap.Flags, lapiconsts.FlagSuccessful), nil
}
-func NodesAndConditionFromResources(ress []lapi.Resource) ([]string, *csi.VolumeCondition) {
+func NodesAndConditionFromResources(ress []lapi.ResourceWithVolumes) ([]string, *csi.VolumeCondition) {
var allNodes, abnormalNodes []string
for i := range ress {
res := &ress[i]
- allNodes = append(allNodes, res.NodeName)
+ // A resource is a CSI publish target if any of its volumes were created
+ // by ControllerPublishVolume, identified by the temporary-diskless-attach property.
+ if slices.ContainsFunc(res.Volumes, func(v lapi.Volume) bool {
+ createdFor, ok := v.Props[linstor.PropertyCreatedFor]
+ return ok && createdFor == linstor.CreatedForTemporaryDisklessAttach
+ }) {
+ allNodes = append(allNodes, res.NodeName)
+ }
if res.State == nil {
abnormalNodes = append(abnormalNodes, res.NodeName)
diff --git a/pkg/volume/parameter.go b/pkg/volume/parameter.go
index 39acd95..aed18ab 100644
--- a/pkg/volume/parameter.go
+++ b/pkg/volume/parameter.go
@@ -50,6 +50,7 @@ const (
nfsservicename
nfssquash
nfsrecoveryvolumesize
+ relocateafterclone
)
// Parameters configuration for linstor volumes.
@@ -118,6 +119,8 @@ type Parameters struct {
// NfsRecoveryVolumeSize sets the volume size (in bytes) of the recovery volume used by the NFS server.
// Defaults to 300MiB.
NfsRecoveryVolumeBytes int64
+ // RelocateAfterClone triggers asynchronous relocation of replicas to optimal nodes after cloning.
+ RelocateAfterClone bool
}
const DefaultDisklessStoragePoolName = "DfltDisklessStorPool"
@@ -140,6 +143,7 @@ func NewParameters(params map[string]string, topologyPrefix string) (Parameters,
NfsServiceName: "linstor-csi-nfs",
NfsSquash: "no_root_squash",
NfsRecoveryVolumeBytes: 300 * 1024 * 1024,
+ RelocateAfterClone: true,
}
for k, v := range params {
@@ -287,6 +291,13 @@ func NewParameters(params map[string]string, topologyPrefix string) (Parameters,
}
p.NfsRecoveryVolumeBytes = s.Value()
+ case relocateafterclone:
+ val, err := strconv.ParseBool(v)
+ if err != nil {
+ return p, err
+ }
+
+ p.RelocateAfterClone = val
}
}
diff --git a/pkg/volume/paramkey_enumer.go b/pkg/volume/paramkey_enumer.go
index 5474963..2eb8a7c 100644
--- a/pkg/volume/paramkey_enumer.go
+++ b/pkg/volume/paramkey_enumer.go
@@ -7,11 +7,11 @@ import (
"strings"
)
-const _paramKeyName = "allowremotevolumeaccessautoplaceclientlistdisklessonremainingdisklessstoragepooldonotplacewithregexencryptionfsoptslayerlistmountoptsnodelistplacementcountplacementpolicyreplicasondifferentreplicasonsamesizekibstoragepoolpostmountxfsoptsresourcegroupusepvcnameoverprovisionxreplicasondifferentnfsconfigtemplatenfsservicenamenfssquashnfsrecoveryvolumesize"
+const _paramKeyName = "allowremotevolumeaccessautoplaceclientlistdisklessonremainingdisklessstoragepooldonotplacewithregexencryptionfsoptslayerlistmountoptsnodelistplacementcountplacementpolicyreplicasondifferentreplicasonsamesizekibstoragepoolpostmountxfsoptsresourcegroupusepvcnameoverprovisionxreplicasondifferentnfsconfigtemplatenfsservicenamenfssquashnfsrecoveryvolumesizerelocateafterclone"
-var _paramKeyIndex = [...]uint16{0, 23, 32, 42, 61, 80, 99, 109, 115, 124, 133, 141, 155, 170, 189, 203, 210, 221, 237, 250, 260, 273, 293, 310, 324, 333, 354}
+var _paramKeyIndex = [...]uint16{0, 23, 32, 42, 61, 80, 99, 109, 115, 124, 133, 141, 155, 170, 189, 203, 210, 221, 237, 250, 260, 273, 293, 310, 324, 333, 354, 372}
-const _paramKeyLowerName = "allowremotevolumeaccessautoplaceclientlistdisklessonremainingdisklessstoragepooldonotplacewithregexencryptionfsoptslayerlistmountoptsnodelistplacementcountplacementpolicyreplicasondifferentreplicasonsamesizekibstoragepoolpostmountxfsoptsresourcegroupusepvcnameoverprovisionxreplicasondifferentnfsconfigtemplatenfsservicenamenfssquashnfsrecoveryvolumesize"
+const _paramKeyLowerName = "allowremotevolumeaccessautoplaceclientlistdisklessonremainingdisklessstoragepooldonotplacewithregexencryptionfsoptslayerlistmountoptsnodelistplacementcountplacementpolicyreplicasondifferentreplicasonsamesizekibstoragepoolpostmountxfsoptsresourcegroupusepvcnameoverprovisionxreplicasondifferentnfsconfigtemplatenfsservicenamenfssquashnfsrecoveryvolumesizerelocateafterclone"
func (i paramKey) String() string {
if i < 0 || i >= paramKey(len(_paramKeyIndex)-1) {
@@ -50,9 +50,10 @@ func _paramKeyNoOp() {
_ = x[nfsservicename-(23)]
_ = x[nfssquash-(24)]
_ = x[nfsrecoveryvolumesize-(25)]
+ _ = x[relocateafterclone-(26)]
}
-var _paramKeyValues = []paramKey{allowremotevolumeaccess, autoplace, clientlist, disklessonremaining, disklessstoragepool, donotplacewithregex, encryption, fsopts, layerlist, mountopts, nodelist, placementcount, placementpolicy, replicasondifferent, replicasonsame, sizekib, storagepool, postmountxfsopts, resourcegroup, usepvcname, overprovision, xreplicasondifferent, nfsconfigtemplate, nfsservicename, nfssquash, nfsrecoveryvolumesize}
+var _paramKeyValues = []paramKey{allowremotevolumeaccess, autoplace, clientlist, disklessonremaining, disklessstoragepool, donotplacewithregex, encryption, fsopts, layerlist, mountopts, nodelist, placementcount, placementpolicy, replicasondifferent, replicasonsame, sizekib, storagepool, postmountxfsopts, resourcegroup, usepvcname, overprovision, xreplicasondifferent, nfsconfigtemplate, nfsservicename, nfssquash, nfsrecoveryvolumesize, relocateafterclone}
var _paramKeyNameToValueMap = map[string]paramKey{
_paramKeyName[0:23]: allowremotevolumeaccess,
@@ -107,6 +108,8 @@ var _paramKeyNameToValueMap = map[string]paramKey{
_paramKeyLowerName[324:333]: nfssquash,
_paramKeyName[333:354]: nfsrecoveryvolumesize,
_paramKeyLowerName[333:354]: nfsrecoveryvolumesize,
+ _paramKeyName[354:372]: relocateafterclone,
+ _paramKeyLowerName[354:372]: relocateafterclone,
}
var _paramKeyNames = []string{
@@ -136,6 +139,7 @@ var _paramKeyNames = []string{
_paramKeyName[310:324],
_paramKeyName[324:333],
_paramKeyName[333:354],
+ _paramKeyName[354:372],
}
// paramKeyString retrieves an enum value from the enum constants string name.
diff --git a/pkg/volume/snapshot_params.go b/pkg/volume/snapshot_params.go
index d167cb8..50b70fb 100644
--- a/pkg/volume/snapshot_params.go
+++ b/pkg/volume/snapshot_params.go
@@ -35,6 +35,7 @@ type SnapshotParameters struct {
LinstorTargetUrl string `json:"linstor-target-url,omitempty"`
LinstorTargetClusterID string `json:"linstor-target-cluster-id,omitempty"`
LinstorTargetStoragePool string `json:"linstor-target-storage-pool,omitempty"`
+ RelocateAfterRestore bool `json:"relocate-after-restore,omitempty"`
}
func NewSnapshotParameters(params, secrets map[string]string) (*SnapshotParameters, error) {
@@ -91,6 +92,13 @@ func NewSnapshotParameters(params, secrets map[string]string) (*SnapshotParamete
p.LinstorTargetClusterID = v
case "/linstor-target-storage-pool":
p.LinstorTargetStoragePool = v
+ case "/relocateAfterRestore":
+ b, err := strconv.ParseBool(v)
+ if err != nil {
+ return nil, err
+ }
+
+ p.RelocateAfterRestore = b
default:
log.WithField("key", k).Warn("ignoring unknown snapshot parameter key")
}

View File

@@ -1,718 +0,0 @@
diff --git a/cmd/linstor-csi/linstor-csi.go b/cmd/linstor-csi/linstor-csi.go
index 143f6cee..bd28e06e 100644
--- a/cmd/linstor-csi/linstor-csi.go
+++ b/cmd/linstor-csi/linstor-csi.go
@@ -41,22 +41,23 @@ import (
func main() {
var (
- lsEndpoint = flag.String("linstor-endpoint", "", "Controller API endpoint for LINSTOR")
- lsSkipTLSVerification = flag.Bool("linstor-skip-tls-verification", false, "If true, do not verify tls")
- csiEndpoint = flag.String("csi-endpoint", "unix:///var/lib/kubelet/plugins/linstor.csi.linbit.com/csi.sock", "CSI endpoint")
- node = flag.String("node", "", "Node ID to pass to node service")
- logLevel = flag.String("log-level", "info", "Enable debug log output. Choose from: panic, fatal, error, warn, info, debug")
- rps = flag.Float64("linstor-api-requests-per-second", 0, "Maximum allowed number of LINSTOR API requests per second. Default: Unlimited")
- burst = flag.Int("linstor-api-burst", 1, "Maximum number of API requests allowed before being limited by requests-per-second. Default: 1 (no bursting)")
- bearerTokenFile = flag.String("bearer-token", "", "Read the bearer token from the given file and use it for authentication.")
- propNs = flag.String("property-namespace", linstor.NamespcAuxiliary, "Limit the reported topology keys to properties from the given namespace.")
- labelBySP = flag.Bool("label-by-storage-pool", true, "Set to false to disable labeling of nodes based on their configured storage pools.")
- nodeCacheTimeout = flag.Duration("node-cache-timeout", 1*time.Minute, "Duration for which the results of node and storage pool related API responses should be cached.")
- resourceCacheTimeout = flag.Duration("resource-cache-timeout", 30*time.Second, "Duration for which the results of resource related API responses should be cached.")
- resyncAfter = flag.Duration("resync-after", 5*time.Minute, "Duration after which reconciliations (such as for VolumeSnapshotClasses) should be rerun. Set to 0 to disable.")
- enableRWX = flag.Bool("enable-rwx", false, "Enable RWX support via NFS (requires running in Kubernetes).")
- namespace = flag.String("nfs-service-namespace", "", "The namespace the NFS service is running in.")
- reactorConfigMapName = flag.String("nfs-reactor-config-map-name", "linstor-csi-nfs-reactor-config", "Name of the config map used to store promoter configuration")
+ lsEndpoint = flag.String("linstor-endpoint", "", "Controller API endpoint for LINSTOR")
+ lsSkipTLSVerification = flag.Bool("linstor-skip-tls-verification", false, "If true, do not verify tls")
+ csiEndpoint = flag.String("csi-endpoint", "unix:///var/lib/kubelet/plugins/linstor.csi.linbit.com/csi.sock", "CSI endpoint")
+ node = flag.String("node", "", "Node ID to pass to node service")
+ logLevel = flag.String("log-level", "info", "Enable debug log output. Choose from: panic, fatal, error, warn, info, debug")
+ rps = flag.Float64("linstor-api-requests-per-second", 0, "Maximum allowed number of LINSTOR API requests per second. Default: Unlimited")
+ burst = flag.Int("linstor-api-burst", 1, "Maximum number of API requests allowed before being limited by requests-per-second. Default: 1 (no bursting)")
+ bearerTokenFile = flag.String("bearer-token", "", "Read the bearer token from the given file and use it for authentication.")
+ propNs = flag.String("property-namespace", linstor.NamespcAuxiliary, "Limit the reported topology keys to properties from the given namespace.")
+ labelBySP = flag.Bool("label-by-storage-pool", true, "Set to false to disable labeling of nodes based on their configured storage pools.")
+ nodeCacheTimeout = flag.Duration("node-cache-timeout", 1*time.Minute, "Duration for which the results of node and storage pool related API responses should be cached.")
+ resourceCacheTimeout = flag.Duration("resource-cache-timeout", 30*time.Second, "Duration for which the results of resource related API responses should be cached.")
+ resyncAfter = flag.Duration("resync-after", 5*time.Minute, "Duration after which reconciliations (such as for VolumeSnapshotClasses) should be rerun. Set to 0 to disable.")
+ enableRWX = flag.Bool("enable-rwx", false, "Enable RWX support via NFS (requires running in Kubernetes).")
+ namespace = flag.String("nfs-service-namespace", "", "The namespace the NFS service is running in.")
+ reactorConfigMapName = flag.String("nfs-reactor-config-map-name", "linstor-csi-nfs-reactor-config", "Name of the config map used to store promoter configuration")
+ disableRWXBlockValidation = flag.Bool("disable-rwx-block-validation", false, "Disable KubeVirt VM ownership validation for RWX block volumes.")
)
flag.Var(&volume.DefaultRemoteAccessPolicy, "default-remote-access-policy", "")
@@ -169,6 +170,10 @@ func main() {
opts = append(opts, driver.ConfigureRWX(*namespace, *reactorConfigMapName))
}
+ if *disableRWXBlockValidation {
+ opts = append(opts, driver.DisableRWXBlockValidation())
+ }
+
drv, err := driver.NewDriver(opts...)
if err != nil {
log.Fatal(err)
diff --git a/pkg/driver/driver.go b/pkg/driver/driver.go
index bea69a8b..a39674b6 100644
--- a/pkg/driver/driver.go
+++ b/pkg/driver/driver.go
@@ -83,6 +83,8 @@ type Driver struct {
topologyPrefix string
// resyncAfter is the interval after which reconciliations should be retried
resyncAfter time.Duration
+ // disableRWXBlockValidation disables KubeVirt VM ownership validation for RWX block volumes
+ disableRWXBlockValidation bool
// Embed for forward compatibility.
csi.UnimplementedIdentityServer
@@ -300,6 +302,17 @@ func ResyncAfter(resyncAfter time.Duration) func(*Driver) error {
}
}
+// DisableRWXBlockValidation disables the KubeVirt VM ownership validation for RWX block volumes.
+// When disabled, the driver will not check if multiple pods using the same RWX block volume
+// belong to the same VM. This may be needed in environments where the validation causes issues
+// or when using RWX block volumes outside of KubeVirt.
+func DisableRWXBlockValidation() func(*Driver) error {
+ return func(d *Driver) error {
+ d.disableRWXBlockValidation = true
+ return nil
+ }
+}
+
// GetPluginInfo https://github.com/container-storage-interface/spec/blob/v1.9.0/spec.md#getplugininfo
func (d Driver) GetPluginInfo(ctx context.Context, req *csi.GetPluginInfoRequest) (*csi.GetPluginInfoResponse, error) {
return &csi.GetPluginInfoResponse{
@@ -751,6 +764,14 @@ func (d Driver) ControllerPublishVolume(ctx context.Context, req *csi.Controller
// ReadWriteMany block volume
rwxBlock := req.VolumeCapability.AccessMode.GetMode() == csi.VolumeCapability_AccessMode_MULTI_NODE_MULTI_WRITER && req.VolumeCapability.GetBlock() != nil
+ // Validate RWX block attachment to prevent misuse of allow-two-primaries
+ if rwxBlock && !d.disableRWXBlockValidation {
+ if _, err := utils.ValidateRWXBlockAttachment(ctx, d.kubeClient, d.log, req.GetVolumeId()); err != nil {
+ return nil, status.Errorf(codes.FailedPrecondition,
+ "ControllerPublishVolume failed for %s: %v", req.GetVolumeId(), err)
+ }
+ }
+
devPath, err := d.Assignments.Attach(ctx, req.GetVolumeId(), req.GetNodeId(), rwxBlock)
if err != nil {
return nil, status.Errorf(codes.Internal,
diff --git a/pkg/utils/rwx_validation.go b/pkg/utils/rwx_validation.go
new file mode 100644
index 00000000..9fe82768
--- /dev/null
+++ b/pkg/utils/rwx_validation.go
@@ -0,0 +1,263 @@
+/*
+CSI Driver for Linstor
+Copyright © 2018 LINBIT USA, LLC
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+package utils
+
+import (
+ "context"
+ "fmt"
+
+ "github.com/sirupsen/logrus"
+ metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+ "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+ "k8s.io/apimachinery/pkg/runtime/schema"
+ "k8s.io/client-go/dynamic"
+)
+
+// KubeVirtVMLabel is the label that KubeVirt adds to pods to identify the VM they belong to.
+const KubeVirtVMLabel = "vm.kubevirt.io/name"
+
+// KubeVirtHotplugDiskLabel is the label that KubeVirt adds to hotplug disk pods.
+const KubeVirtHotplugDiskLabel = "kubevirt.io"
+
+// PodGVR is the GroupVersionResource for pods.
+var PodGVR = schema.GroupVersionResource{Group: "", Version: "v1", Resource: "pods"}
+
+// PVGVR is the GroupVersionResource for persistent volumes.
+var PVGVR = schema.GroupVersionResource{Group: "", Version: "v1", Resource: "persistentvolumes"}
+
+// ValidateRWXBlockAttachment checks that RWX block volumes are only used by pods belonging to the same VM.
+// This prevents misuse of allow-two-primaries while still permitting live migration.
+// Returns the VM name if validation passes, or an error if:
+// - Multiple pods from different VMs are trying to use the same volume
+// - A pod without the KubeVirt VM label is trying to use a volume already attached elsewhere (strict mode)
+// Returns empty string for VM name when no pods are using the volume or validation is skipped.
+func ValidateRWXBlockAttachment(ctx context.Context, kubeClient dynamic.Interface, log *logrus.Entry, volumeID string) (string, error) {
+ log.WithField("volumeID", volumeID).Info("validateRWXBlockAttachment called")
+
+ if kubeClient == nil {
+ // Not running in Kubernetes, skip validation
+ log.Warn("validateRWXBlockAttachment: kubeClient is nil, skipping validation")
+ return "", nil
+ }
+
+ // Get PV to find PVC reference
+ pv, err := kubeClient.Resource(PVGVR).Get(ctx, volumeID, metav1.GetOptions{})
+ if err != nil {
+ log.WithError(err).Warn("cannot validate RWX attachment: failed to get PV")
+ return "", nil
+ }
+
+ // Verify that PV's volumeHandle matches the volumeID
+ volumeHandle, found, err := unstructured.NestedString(pv.Object, "spec", "csi", "volumeHandle")
+ if err != nil {
+ log.WithError(err).Warnf("cannot validate RWX attachment: failed to read volumeHandle for PV %s", volumeID)
+
+ return "", nil
+ }
+
+ if !found {
+ log.Warnf("cannot validate RWX attachment: volumeHandle not found for PV %s", volumeID)
+
+ return "", nil
+ }
+
+ if volumeHandle != volumeID {
+ log.WithFields(logrus.Fields{
+ "volumeID": volumeID,
+ "volumeHandle": volumeHandle,
+ }).Warn("cannot validate RWX attachment: PV volumeHandle does not match volumeID")
+
+ return "", nil
+ }
+
+ // Extract claimRef from PV
+ claimRef, found, _ := unstructured.NestedMap(pv.Object, "spec", "claimRef")
+ if !found {
+ log.Warn("cannot validate RWX attachment: PV has no claimRef")
+ return "", nil
+ }
+
+ pvcName, _, _ := unstructured.NestedString(claimRef, "name")
+ pvcNamespace, _, _ := unstructured.NestedString(claimRef, "namespace")
+
+ if pvcNamespace == "" || pvcName == "" {
+ log.Warn("cannot validate RWX attachment: PVC name or namespace is empty in claimRef")
+ return "", nil
+ }
+
+ // List all pods in the namespace
+ podList, err := kubeClient.Resource(PodGVR).Namespace(pvcNamespace).List(ctx, metav1.ListOptions{})
+ if err != nil {
+ return "", fmt.Errorf("failed to list pods in namespace %s: %w", pvcNamespace, err)
+ }
+
+ // Filter pods that use this PVC and are in a running/pending state
+ type podInfo struct {
+ name string
+ vmName string
+ }
+
+ var podsUsingPVC []podInfo
+
+ for _, item := range podList.Items {
+ // Get pod phase from status
+ phase, _, _ := unstructured.NestedString(item.Object, "status", "phase")
+ if phase == "Succeeded" || phase == "Failed" {
+ continue
+ }
+
+ // Check if pod uses the PVC
+ volumes, found, _ := unstructured.NestedSlice(item.Object, "spec", "volumes")
+ if !found {
+ continue
+ }
+
+ for _, vol := range volumes {
+ volMap, ok := vol.(map[string]interface{})
+ if !ok {
+ continue
+ }
+
+ claimName, found, _ := unstructured.NestedString(volMap, "persistentVolumeClaim", "claimName")
+ if !found || claimName != pvcName {
+ continue
+ }
+
+ // Extract VM name, handling both regular and hotplug disk pods
+ vmName, err := GetVMNameFromPod(ctx, kubeClient, log, &item)
+ if err != nil {
+ log.WithError(err).WithField("pod", item.GetName()).Warn("failed to get VM name from pod")
+ // Continue with empty vmName - will be caught by strict mode check
+ vmName = ""
+ }
+
+ podsUsingPVC = append(podsUsingPVC, podInfo{
+ name: item.GetName(),
+ vmName: vmName,
+ })
+
+ break
+ }
+ }
+
+ // If 0 or 1 pod uses the PVC, no conflict possible
+ if len(podsUsingPVC) <= 1 {
+ // Return VM name if there's exactly one pod
+ if len(podsUsingPVC) == 1 {
+ log.WithFields(logrus.Fields{
+ "volumeID": volumeID,
+ "vmName": podsUsingPVC[0].vmName,
+ "podCount": 1,
+ "pvcNamespace": pvcNamespace,
+ "pvcName": pvcName,
+ }).Info("validateRWXBlockAttachment: single pod found, returning VM name")
+
+ return podsUsingPVC[0].vmName, nil
+ }
+
+ log.WithFields(logrus.Fields{
+ "volumeID": volumeID,
+ "pvcNamespace": pvcNamespace,
+ "pvcName": pvcName,
+ }).Info("validateRWXBlockAttachment: no pods found using PVC")
+
+ return "", nil
+ }
+
+ // Check that all pods belong to the same VM
+ var vmName string
+ for _, pod := range podsUsingPVC {
+ if pod.vmName == "" {
+ // Strict mode: if any pod doesn't have the KubeVirt label and there are multiple pods,
+ // deny the attachment
+ return "", fmt.Errorf("RWX block volume %s/%s is used by multiple pods but pod %s does not have the %s label; "+
+ "RWX block volumes with allow-two-primaries are only supported for KubeVirt live migration",
+ pvcNamespace, pvcName, pod.name, KubeVirtVMLabel)
+ }
+
+ if vmName == "" {
+ vmName = pod.vmName
+ } else if vmName != pod.vmName {
+ // Different VMs are trying to use the same volume
+ return "", fmt.Errorf("RWX block volume %s/%s is being used by pods from different VMs (%s and %s); "+
+ "this is not supported - RWX block volumes with allow-two-primaries are only for live migration of a single VM",
+ pvcNamespace, pvcName, vmName, pod.vmName)
+ }
+ }
+
+ log.WithFields(logrus.Fields{
+ "pvcNamespace": pvcNamespace,
+ "pvcName": pvcName,
+ "vmName": vmName,
+ "podCount": len(podsUsingPVC),
+ }).Debug("RWX block attachment validated: all pods belong to the same VM (likely live migration)")
+
+ return vmName, nil
+}
+
+// GetVMNameFromPod extracts the VM name from a pod, handling both regular virt-launcher pods
+// and hotplug disk pods (which reference the virt-launcher pod via ownerReferences).
+func GetVMNameFromPod(ctx context.Context, kubeClient dynamic.Interface, log *logrus.Entry, pod *unstructured.Unstructured) (string, error) {
+ labels := pod.GetLabels()
+ if labels == nil {
+ return "", nil
+ }
+
+ // Direct case: pod has vm.kubevirt.io/name label (virt-launcher pod)
+ if vmName, ok := labels[KubeVirtVMLabel]; ok && vmName != "" {
+ return vmName, nil
+ }
+
+ // Hotplug disk case: pod has kubevirt.io: hotplug-disk label
+ // Follow ownerReferences to find the virt-launcher pod
+ if hotplugValue, ok := labels[KubeVirtHotplugDiskLabel]; ok && hotplugValue == "hotplug-disk" {
+ ownerRefs := pod.GetOwnerReferences()
+ for _, owner := range ownerRefs {
+ if owner.Kind != "Pod" || owner.Controller == nil || !*owner.Controller {
+ continue
+ }
+
+ // Get the owner pod (virt-launcher)
+ ownerPod, err := kubeClient.Resource(PodGVR).Namespace(pod.GetNamespace()).Get(ctx, owner.Name, metav1.GetOptions{})
+ if err != nil {
+ return "", fmt.Errorf("failed to get owner pod %s: %w", owner.Name, err)
+ }
+
+ // Extract VM name from owner pod
+ ownerLabels := ownerPod.GetLabels()
+ if ownerLabels != nil {
+ if vmName, ok := ownerLabels[KubeVirtVMLabel]; ok && vmName != "" {
+ log.WithFields(logrus.Fields{
+ "hotplugPod": pod.GetName(),
+ "virtLauncher": owner.Name,
+ "vmName": vmName,
+ }).Debug("resolved VM name from hotplug disk pod via owner reference")
+
+ return vmName, nil
+ }
+ }
+
+ return "", fmt.Errorf("owner pod %s does not have %s label", owner.Name, KubeVirtVMLabel)
+ }
+
+ return "", fmt.Errorf("hotplug disk pod %s has no controller owner reference", pod.GetName())
+ }
+
+ return "", nil
+}
diff --git a/pkg/utils/rwx_validation_test.go b/pkg/utils/rwx_validation_test.go
new file mode 100644
index 00000000..d75690f9
--- /dev/null
+++ b/pkg/utils/rwx_validation_test.go
@@ -0,0 +1,342 @@
+/*
+CSI Driver for Linstor
+Copyright © 2018 LINBIT USA, LLC
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+package utils
+
+import (
+ "context"
+ "testing"
+
+ "github.com/sirupsen/logrus"
+ "github.com/stretchr/testify/assert"
+ "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+ "k8s.io/apimachinery/pkg/runtime"
+ "k8s.io/apimachinery/pkg/runtime/schema"
+ dynamicfake "k8s.io/client-go/dynamic/fake"
+)
+
+func TestValidateRWXBlockAttachment(t *testing.T) {
+ testCases := []struct {
+ name string
+ pods []*unstructured.Unstructured
+ pvcName string
+ namespace string
+ expectError bool
+ errorMsg string
+ }{
+ {
+ name: "no pods using PVC",
+ pods: []*unstructured.Unstructured{},
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "single pod using PVC",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("pod1", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "two pods same VM (live migration)",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("virt-launcher-vm1-abc", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("virt-launcher-vm1-xyz", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "two pods different VMs (should fail)",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("virt-launcher-vm1-abc", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("virt-launcher-vm2-xyz", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm2"}, "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: true,
+ errorMsg: "different VMs",
+ },
+ {
+ name: "pod without KubeVirt label when multiple pods exist (strict mode)",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("pod1", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("pod2", "default", "test-pvc", map[string]string{}, "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: true,
+ errorMsg: "does not have the vm.kubevirt.io/name label",
+ },
+ {
+ name: "completed pods should be ignored",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("pod1", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("pod2", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm2"}, "Succeeded"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "failed pods should be ignored",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("pod1", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("pod2", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm2"}, "Failed"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "pods in different namespace should not conflict",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("pod1", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("pod2", "other", "test-pvc", map[string]string{KubeVirtVMLabel: "vm2"}, "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "pods using different PVCs should not conflict",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("pod1", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("pod2", "default", "other-pvc", map[string]string{KubeVirtVMLabel: "vm2"}, "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "three pods from same VM (multi-node live migration scenario)",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("virt-launcher-vm1-a", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("virt-launcher-vm1-b", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createUnstructuredPod("virt-launcher-vm1-c", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Pending"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "hotplug disk pod with virt-launcher (should succeed)",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("virt-launcher-vm1-abc", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createHotplugDiskPod("hp-volume-xyz", "default", "test-pvc", "virt-launcher-vm1-abc", "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: false,
+ },
+ {
+ name: "hotplug disks from different VMs (should fail)",
+ pods: []*unstructured.Unstructured{
+ createUnstructuredPod("virt-launcher-vm1-abc", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm1"}, "Running"),
+ createHotplugDiskPod("hp-volume-vm1", "default", "test-pvc", "virt-launcher-vm1-abc", "Running"),
+ createUnstructuredPod("virt-launcher-vm2-xyz", "default", "test-pvc", map[string]string{KubeVirtVMLabel: "vm2"}, "Running"),
+ createHotplugDiskPod("hp-volume-vm2", "default", "test-pvc", "virt-launcher-vm2-xyz", "Running"),
+ },
+ pvcName: "test-pvc",
+ namespace: "default",
+ expectError: true,
+ errorMsg: "different VMs",
+ },
+ }
+
+ for _, tc := range testCases {
+ t.Run(tc.name, func(t *testing.T) {
+ // Create fake dynamic client with test pods and PV
+ scheme := runtime.NewScheme()
+
+ // Create PV object that references the PVC
+ pv := createUnstructuredPV("test-volume-id", tc.namespace, tc.pvcName)
+
+ objects := make([]runtime.Object, 0, len(tc.pods)+1)
+ objects = append(objects, pv)
+
+ for _, pod := range tc.pods {
+ objects = append(objects, pod)
+ }
+
+ gvrToListKind := map[schema.GroupVersionResource]string{
+ PodGVR: "PodList",
+ PVGVR: "PersistentVolumeList",
+ }
+ client := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(scheme, gvrToListKind, objects...)
+
+ // Create logger
+ logger := logrus.NewEntry(logrus.New())
+ logger.Logger.SetLevel(logrus.DebugLevel)
+
+ // Run validation
+ vmName, err := ValidateRWXBlockAttachment(context.Background(), client, logger, "test-volume-id")
+
+ if tc.expectError {
+ assert.Error(t, err)
+
+ if tc.errorMsg != "" {
+ assert.Contains(t, err.Error(), tc.errorMsg)
+ }
+ } else {
+ assert.NoError(t, err)
+ // VM name is returned when there are pods using the volume
+ if len(tc.pods) > 0 {
+ assert.NotEmpty(t, vmName)
+ }
+ }
+ })
+ }
+}
+
+func TestValidateRWXBlockAttachmentNoKubeClient(t *testing.T) {
+ // When not running in Kubernetes (no client), validation should be skipped
+ logger := logrus.NewEntry(logrus.New())
+
+ vmName, err := ValidateRWXBlockAttachment(context.Background(), nil, logger, "test-volume-id")
+ assert.NoError(t, err)
+ assert.Empty(t, vmName)
+}
+
+func TestValidateRWXBlockAttachmentPVNotFound(t *testing.T) {
+ // When PV is not found, validation should be skipped with warning
+ scheme := runtime.NewScheme()
+
+ gvrToListKind := map[schema.GroupVersionResource]string{
+ PodGVR: "PodList",
+ PVGVR: "PersistentVolumeList",
+ }
+ client := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(scheme, gvrToListKind)
+
+ logger := logrus.NewEntry(logrus.New())
+ logger.Logger.SetLevel(logrus.DebugLevel)
+
+ vmName, err := ValidateRWXBlockAttachment(context.Background(), client, logger, "non-existent-pv")
+ assert.NoError(t, err)
+ assert.Empty(t, vmName)
+}
+
+// createUnstructuredPod creates an unstructured pod object for testing.
+func createUnstructuredPod(name, namespace, pvcName string, labels map[string]string, phase string) *unstructured.Unstructured {
+ pod := &unstructured.Unstructured{
+ Object: map[string]interface{}{
+ "apiVersion": "v1",
+ "kind": "Pod",
+ "metadata": map[string]interface{}{
+ "name": name,
+ "namespace": namespace,
+ "labels": toStringInterfaceMap(labels),
+ },
+ "spec": map[string]interface{}{
+ "volumes": []interface{}{
+ map[string]interface{}{
+ "name": "data",
+ "persistentVolumeClaim": map[string]interface{}{
+ "claimName": pvcName,
+ },
+ },
+ },
+ },
+ "status": map[string]interface{}{
+ "phase": phase,
+ },
+ },
+ }
+
+ return pod
+}
+
+// createUnstructuredPV creates an unstructured PersistentVolume object for testing.
+func createUnstructuredPV(name, pvcNamespace, pvcName string) *unstructured.Unstructured {
+ pv := &unstructured.Unstructured{
+ Object: map[string]interface{}{
+ "apiVersion": "v1",
+ "kind": "PersistentVolume",
+ "metadata": map[string]interface{}{
+ "name": name,
+ },
+ "spec": map[string]interface{}{
+ "claimRef": map[string]interface{}{
+ "name": pvcName,
+ "namespace": pvcNamespace,
+ },
+ "csi": map[string]interface{}{
+ "volumeHandle": name,
+ },
+ },
+ },
+ }
+
+ return pv
+}
+
+// toStringInterfaceMap converts map[string]string to map[string]interface{}.
+func toStringInterfaceMap(m map[string]string) map[string]interface{} {
+ result := make(map[string]interface{})
+
+ for k, v := range m {
+ result[k] = v
+ }
+
+ return result
+}
+
+// createHotplugDiskPod creates a hotplug disk pod that references a virt-launcher pod via ownerReferences.
+func createHotplugDiskPod(name, namespace, pvcName, ownerPodName, phase string) *unstructured.Unstructured {
+ pod := &unstructured.Unstructured{
+ Object: map[string]interface{}{
+ "apiVersion": "v1",
+ "kind": "Pod",
+ "metadata": map[string]interface{}{
+ "name": name,
+ "namespace": namespace,
+ "labels": map[string]interface{}{
+ "kubevirt.io": "hotplug-disk",
+ },
+ "ownerReferences": []interface{}{
+ map[string]interface{}{
+ "apiVersion": "v1",
+ "kind": "Pod",
+ "name": ownerPodName,
+ "controller": true,
+ "blockOwnerDeletion": true,
+ },
+ },
+ },
+ "spec": map[string]interface{}{
+ "volumes": []interface{}{
+ map[string]interface{}{
+ "name": "data",
+ "persistentVolumeClaim": map[string]interface{}{
+ "claimName": pvcName,
+ },
+ },
+ },
+ },
+ "status": map[string]interface{}{
+ "phase": phase,
+ },
+ },
+ }
+
+ return pod
+}

View File

@@ -21,6 +21,10 @@ spec:
scrapeInterval: 30s
selectAllByDefault: false
{{- with .Values.vmagent.inlineScrapeConfig }}
inlineScrapeConfig: |
{{- . | nindent 4 }}
{{- end }}
podScrapeNamespaceSelector:
matchLabels:
namespace.cozystack.io/monitoring: {{ .Release.Namespace }}

View File

@@ -79,4 +79,8 @@ vmagent:
remoteWrite:
urls:
- http://vminsert-shortterm:8480/insert/0/prometheus
- http://vminsert-longterm:8480/insert/0/prometheus
- http://vminsert-longterm:8480/insert/0/prometheus
## inlineScrapeConfig: |
## - job_name: "custom"
## static_configs:
## - targets: ["my-service:9090"]

View File

@@ -224,8 +224,8 @@ func buildPostProcessV3(kindSchemas map[string]string) func(*spec3.OpenAPI) (*sp
base, ok1 := doc.Components.Schemas[baseRef]
list, ok2 := doc.Components.Schemas[baseListRef]
stat, ok3 := doc.Components.Schemas[baseStatusRef]
if !(ok1 && ok2 && ok3) && len(kindSchemas) > 0 {
return doc, fmt.Errorf("base Application* schemas not found")
if !(ok1 && ok2 && ok3) {
return doc, nil // not the apps GV — nothing to patch
}
// Clone base schemas for each kind
@@ -339,8 +339,8 @@ func buildPostProcessV2(kindSchemas map[string]string) func(*spec.Swagger) (*spe
base, ok1 := defs[baseRef]
list, ok2 := defs[baseListRef]
stat, ok3 := defs[baseStatusRef]
if !(ok1 && ok2 && ok3) && len(kindSchemas) > 0 {
return sw, fmt.Errorf("base Application* schemas not found")
if !(ok1 && ok2 && ok3) {
return sw, nil // not the apps GV — nothing to patch
}
for kind, raw := range kindSchemas {