WorkloadMonitor resources are only needed when extra/monitoring is
deployed. Move them from system/monitoring to extra/monitoring package.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
- Replace hardcoded tenant-root with {{ .Release.Namespace }} in vmagent
and fluent-bit configs
- Add ExternalName services in cozystack-basics to redirect monitoring
traffic from cozy-monitoring to tenant-root when engine is deployed
- Add missing components to monitoring PackageSource
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add dashboards.list that defines which Grafana dashboards to create.
Without this file, no GrafanaDashboard resources were being generated.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Remove components that are already defined in separate PackageSources
to avoid race conditions and HelmRelease conflicts:
- vertical-pod-autoscaler
- postgres-operator
- grafana-operator
- victoria-metrics-operator
- prometheus-operator-crds
These components should be installed via their dedicated PackageSources.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The monitoring package depends on cozystack.cozystack-controller which
is only installed as part of cozystack-engine. This prevents monitoring
from becoming ready when cozystack-engine is not enabled.
Remove this dependency to allow monitoring to work independently.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## Summary
- Fix invalid PromQL expression in `CNPGClusterOffline` alert rule that
causes vmalert pods to crash
- Fix PromQL logic so the alert fires correctly when cluster is offline
## Problem
The `CNPGClusterOffline` alert in
`packages/system/postgres-operator/alerts/cnpg-default-alerts.yaml` had
two issues:
1. **Syntax error** - Extra closing parenthesis causing vmalert pods to
crash:
```promql
sum by (namespace, pod) (cnpg_collector_up)) OR on() vector(0) == 0
^^-- extra )
```
2. **Logic error** - After removing the extra parenthesis, the
expression still had incorrect logic:
```promql
sum by (namespace, pod) (cnpg_collector_up) OR on() vector(0) == 0
```
This evaluates as `cnpg_collector_up OR (vector(0) == 0)` which fires
when the cluster is **online** (wrong behavior).
## Solution
Fix both issues by properly wrapping the OR expression in parentheses:
```promql
(sum by (namespace, pod) (cnpg_collector_up) OR on() vector(0)) == 0
```
This correctly fires when `cnpg_collector_up` is 0 or absent (cluster
actually offline).
## Test plan
- [ ] Verify vmalert pods start successfully after applying the fix
- [ ] Verify the alert rule is properly loaded
- [ ] Verify alert only fires when CNPG cluster is actually offline
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## What this PR does
When OIDC is disabled, the dashboard's token-proxy now properly
validates bearer tokens against the k8s API's JWKS url.
### Release note
```release-note
[dashboard] Verify bearer tokens against the issuer's JWKS url.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Implemented JWKS-based JWT token verification for enhanced security.
* Added conditional role-based access control for non-OIDC deployments.
* **Chores**
* Updated authentication dependencies and modernized token validation.
* Removed legacy external token validation mechanism.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When OIDC is disabled, the dashboard's token-proxy now properly
validates bearer tokens against the k8s API's JWKS url.
### Release note
```release-note
[dashboard] Verify bearer tokens against the issuer's JWKS url.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Remove extra closing parenthesis in the CNPGClusterOffline alert expression
that causes vmalert pods to crash with "bad prometheus expr" error.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- Add monitoring as a system chart
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced comprehensive monitoring stack with Grafana, Victoria
Metrics operators, Prometheus operator, and vertical pod autoscaler for
enhanced system observability and performance metrics.
* **Chores**
* System upgraded to version 27 with automatic monitoring resource
reorganization for improved infrastructure management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add migration 26 that re-labels monitoring resources to be owned by
monitoring-system HelmRelease instead of monitoring. This allows
seamless transition from direct resource management in extra/monitoring
to delegated management via system/monitoring chart.
Migration steps:
- Find all monitoring HelmReleases in tenant namespaces
- Suspend HelmRelease to prevent reconciliation
- Delete helm secrets to orphan resources
- Relabel all resources to monitoring-system
- Delete suspended HelmRelease
Also updates:
- monitoring-application.yaml: add monitoring-system component
- helmrelease.yaml: reference monitoring-system ExternalArtifact
- values.yaml: bump targetVersion to 25
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Configure CoreDNS chart to create `kube-dns` ServiceAccount matching
Kubernetes bootstrap ClusterRoleBinding
- Fixes RBAC errors (`Failed to watch`) when CoreDNS pods restart
## Problem
Kubernetes bootstrap creates a `ClusterRoleBinding: system:kube-dns`
that references `ServiceAccount: kube-dns` in `kube-system`. However,
the coredns chart was using the `default` ServiceAccount because
`serviceAccount.create` was not enabled.
This caused CoreDNS pods to fail with `[ERROR] plugin/kubernetes: Failed
to watch` errors after restarts, as they lacked RBAC permissions to
watch the Kubernetes API. Old pods worked due to cached data, but new
pods failed after rollout.
## Solution
Add ServiceAccount configuration to
`packages/system/coredns/values.yaml`:
```yaml
serviceAccount:
create: true
name: kube-dns
```
## Test plan
- [x] Verify ServiceAccount `kube-dns` is created: `kubectl get sa -n
kube-system kube-dns`
- [x] Verify deployment uses correct ServiceAccount: `kubectl get
deployment -n kube-system coredns -o
jsonpath='{.spec.template.spec.serviceAccountName}'` → `kube-dns`
- [x] Restart CoreDNS pods and verify all pods are Ready
- [x] Check logs show no RBAC errors
- [x] Test DNS resolution works
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated DNS service configuration with service account settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add migration 25 for v1.0 upgrade cleanup:
- Remove legacy `cozystack` installer deployment
- Remove legacy `cozystack-assets` statefulset
- Remove `bootbox` and `bootbox-rd` HelmReleases (will be recreated by
new platform chart if enabled)
- Revert migration 23 to original state (only removes
cozystack-resource-definition-crd)
- Update targetVersion from 24 to 26
- Fix jq syntax error in `hack/migrate-to-version-1.0.sh`
Migration 23 was already executed on existing clusters at version 23+,
so adding cleanup there would never run. The new migration 25 ensures
proper cleanup during v1.0 upgrade.
## Test plan
- [ ] Upgrade from v0.4x to v1.0
- [ ] Verify old `cozystack` deployment is deleted
- [ ] Verify old `cozystack-assets` statefulset is deleted
- [ ] Verify old `bootbox` and `bootbox-rd` HelmReleases are deleted
- [ ] Verify platform installs successfully after migration
- [ ] Run `./hack/migrate-to-version-1.0.sh` without jq errors
Migration 23 was already executed on existing clusters, so the legacy
installer removal never ran. This moves the cleanup to a new migration 25
which will execute during upgrade.
Changes:
- Revert migration 23 to original state (only removes cozystack-resource-definition-crd)
- Add migration 25 for v1.0 upgrade cleanup:
- Remove legacy cozystack installer deployment
- Remove legacy cozystack-assets statefulset
- Remove bootbox and bootbox-rd HelmReleases
- Update targetVersion from 24 to 26
- Fix jq syntax error in migrate-to-version-1.0.sh
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Remove duplicate `bootbox-system` component from `bootbox-application`
PackageSource (was duplicating system/bootbox installation)
- Auto-create `cozystack.bootbox-application` Package when bootbox is
enabled in `bundles.enabledPackages`
- This ensures `bootbox-rd` (ApplicationDefinition) is properly
installed as a dependency before `cozystack.bootbox`
## Test plan
- [ ] Enable bootbox in `bundles.enabledPackages`
- [ ] Verify both `cozystack.bootbox-application` and
`cozystack.bootbox` Packages are created
- [ ] Verify `bootbox-rd` (ApplicationDefinition) is installed
- Fix docker build context for migrations image
- Fix yq flags for compatibility with Mike Farah's yq v4
- Add platform image to main build target
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add cleanup of old installer components during migration:
- Delete cozystack deployment
- Delete cozystack-assets statefulset
This ensures no conflict between old installer and new operator
when upgrading to v1.0 architecture.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Remove duplicate bootbox-system component from bootbox-application
PackageSource (was duplicating system/bootbox installation)
- Auto-create cozystack.bootbox-application Package when bootbox is
enabled in bundles.enabledPackages
- This ensures bootbox-rd (ApplicationDefinition) is properly installed
as a dependency before cozystack.bootbox
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Removes `hostNetwork` for Ingress NGINX Controller for Managed
kubernetes cluster
- Adds Service of type NodePort for load balancing
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] use NodePort service instead of hostNetwork for ingress-nginx
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Configuration**
* Updated ingress-nginx controller to use NodePort exposure method
(ports 30000 for HTTP, 30001 for HTTPS) when configured as "Proxied."
* Removed unnecessary hostNetwork configuration for the controller.
* Service port mappings updated to align with new NodePort settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds separate values to keycloak branding.
### Release note
```release-note
Added separate values to keycloak branding
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Enhanced branding configuration application with improved conditional
logic to ensure branding is only applied when appropriate configuration
values are present.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Aligns MongoDB configuration with postgres and mysql patterns by
introducing a separate `databases` section.
**Before:**
```yaml
users:
myuser:
db: mydb
roles:
- name: readWrite
db: mydb
- name: dbAdmin
db: mydb
```
**After:**
```yaml
users:
myuser: {}
databases:
mydb:
roles:
admin:
- myuser
readonly:
- reader
```
Changes:
- Add separate `databases` section with `roles.admin` and
`roles.readonly`
- Simplify `users` to only contain optional password field
- All users authenticate via `admin` database (MongoDB production best
practice)
- `admin` role maps to `readWrite` + `dbAdmin` permissions
- `readonly` role maps to `read` permission
## Migration
Includes migration script (`migrations/24`) that automatically converts
existing MongoDB HelmReleases:
- Detects old format users (with `db` and `roles` fields)
- Converts `readWrite`/`dbAdmin` roles → `databases.{db}.roles.admin`
- Converts `read` role → `databases.{db}.roles.readonly`
- Preserves user passwords in new `users` format
- Skips HelmReleases that don't have old format
### Release note
```release-note
[mongodb] Unified users and databases configuration to match postgres and mysql patterns. Users now defined separately from databases, with role assignments in the databases section. Includes automatic migration for existing deployments.
```
This PR prepares the release `v1.0.0-beta.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated multiple system component images from alpha to beta releases
(v1.0.0-beta.2)
* Updated container image digests across various services for
consistency and stability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Service will be recreated
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Allow changing field `external` after creation.
Service will be deleted and then created with correct type.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm] recreate service on `external` change to allow changing this field after creation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Detects Service type changes during virtual machine updates and
automatically recreates the Service as needed; update workflow now runs
a job to remove and recreate Services before reconciliation.
* **Chores**
* Updated RBAC to grant the update workflow permissions to manage
(delete) the targeted Service.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Kubernetes bootstrap creates a ClusterRoleBinding 'system:kube-dns'
that references ServiceAccount 'kube-dns' in 'kube-system'. However,
the coredns chart was using the 'default' ServiceAccount because
serviceAccount.create was not enabled.
This caused CoreDNS pods to fail with 'Failed to watch' errors after
restarts, as they lacked RBAC permissions to watch the Kubernetes API.
Configure the chart to create the 'kube-dns' ServiceAccount, which
matches the expected binding from Kubernetes bootstrap.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Add instance profile metadata to workload monitor
Currently getWorkloadMetadata only extracts the
kubevirt.io/cluster-instancetype-name annotation from VMI objects. This
misses the instance profile information
(kubevirt.io/cluster-preference-name), which is needed to track
workload sizing/configuration alongside the instance type.
This change adds extraction of the cluster instance profile annotation
and exposes it as the
workloads.cozystack.io/kubevirt-vmi-instance-profile label on collected
metrics.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Workload monitoring now detects kubevirt instance profile annotations
and adds a matching workload label
(workloads.cozystack.io/kubevirt-vmi-instance-profile) when present.
* Enriched workload metadata for virtualized workloads.
* No other behavior or control-flow changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Configured cluster for high availability with multiple replicas
* Implemented rolling update strategy to ensure safer deployments
* Enhanced health constraints to maintain cluster stability during
updates
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Enable Cozystack deployment on generic Kubernetes clusters (kubeadm,
k3s, RKE2, etc.) by making Talos-specific configurations conditional and
adding a new `isp-full-generic` bundle variant.
## Problem 1: Cilium uses hardcoded Talos API endpoint
**Issue**: Cilium is configured to connect to Kubernetes API at
`localhost:7445` (Talos KubePrism). On non-Talos clusters, the API
server runs on standard `<node-ip>:6443`.
**Solution**: Add `networking.apiServer.host` and
`networking.apiServer.port` to platform values, passed to Cilium via
bundle template.
**Why this approach**: Allows per-deployment configuration while keeping
Talos defaults for backwards compatibility.
## Problem 2: Cilium cgroup automount disabled
**Issue**: `values-talos.yaml` sets `cgroup.autoMount.enabled: false`
because Talos mounts cgroups. On Ubuntu/Debian, Cilium needs to mount
cgroups itself.
**Solution**: Add `networking.cilium.cgroup.autoMount` configuration and
create `cilium-generic` / `kubeovn-cilium-generic` networking variants
without `values-talos.yaml`.
**Why this approach**: Separate variants avoid complex conditional logic
in templates and make the difference explicit.
## Problem 3: KubeOVN helm lookup fails on fresh clusters
**Issue**: KubeOVN chart uses `lookup` to find control-plane nodes.
During initial deployment, lookup returns empty results causing
installation failure.
**Solution**: Add `networking.kubeovn.MASTER_NODES` parameter to
explicitly pass node IPs when needed.
**Why this approach**: Helm lookup is unreliable during initial
deployment; explicit configuration is more predictable.
## Problem 4: LinstorSatelliteConfiguration breaks non-Talos nodes
**Issue**: `cozystack-talos` LinstorSatelliteConfiguration removes DRBD
init containers (`drbd-module-loader`, `drbd-shutdown-guard`). These are
required on Ubuntu/Debian where DRBD modules aren't pre-loaded.
**Solution**: Add `talos.enabled` value (default: true) and wrap the
configuration in `{{- if .Values.talos.enabled }}`.
**Why this approach**: Conditional rendering is cleaner than node
selectors (Talos has no unique default label) and maintains backwards
compatibility.
## Problem 5: linstor-scheduler image tag invalid for k3s
**Issue**: k3s reports version as `v1.35.0+k3s1`. The `+` character is
invalid in Docker image tags, causing `InvalidImageName` error for
kube-scheduler.
**Solution**: Strip distribution suffix using `regexReplaceAll "\\+.*$"`
in the helper template.
**Why this approach**: k3s version `v1.35.0+k3s1` is based on upstream
`v1.35.0`, so stripping the suffix produces the correct upstream image
tag. This is automatic and doesn't require manual version specification.
## Problem 6: No bundle for non-Talos deployments
**Issue**: `isp-full` bundle always uses Talos-specific networking and
doesn't pass `talos.enabled=false` to linstor.
**Solution**: Add `isp-full-generic` bundle variant that uses
`kubeovn-cilium-generic` networking and passes `talos.enabled=false` to
linstor.
**Why this approach**: Dedicated bundle variant makes deployment
straightforward — users just set `bundle-name: isp-full-generic` without
needing post-install patches.
## Files Changed
| Path | Change |
|------|--------|
| `packages/core/platform/values.yaml` | Add apiServer, cilium.cgroup,
kubeovn.MASTER_NODES config |
| `packages/core/platform/sources/networking.yaml` | Add generic
variants |
| `packages/core/platform/templates/bundles/system.yaml` | Add
isp-full-generic bundle |
| `packages/system/linstor/values.yaml` | Add talos.enabled |
| `packages/system/linstor/templates/satellites-talos.yaml` |
Conditional rendering |
| `packages/system/linstor-scheduler/.../templates/_helpers.tpl` | Strip
version suffix |
| `packages/system/linstor-scheduler/.../templates/deployment.yaml` |
Use helper for image tag |
## Testing
Tested on k3s v1.35.0+k3s1 with Ubuntu 24.04 LTS. All HelmReleases
deploy successfully without post-install patches.
## Breaking Changes
None. Existing Talos deployments continue to work unchanged with default
values.
---
Closes#1933
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added support for generic (non-Talos) Kubernetes clusters with new
isp-full-generic platform bundle
* Introduced generic and hosted deployment variants for
cozystack-operator
* Added generic networking variants with Cilium and KubeOVN support
* Expanded IAAS, PAAS, and NAAS bundle support for generic deployments
* Improved backup retention configuration
* **Bug Fixes**
* Fixed Kubernetes version handling to work with distribution-specific
suffixes
* **Chores**
* Made LINSTOR Talos satellite support optional
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add external ips (LoadBalancer services with assigned IP address) count
to the Tenant details page
<img width="2560" height="1330" alt="external ip count on tenant details
page"
src="https://github.com/user-attachments/assets/f2256629-3815-4892-b80d-e150d0fd3908"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] add external ips count (count of services of type LoadBalancer with assigned ip) to the Tenant details page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* External IPs count now shown in the Tenant application Details tab — a
left-column item displays the number of external IPs for LoadBalancer
services.
* Application status now exposes an External IPs Count metric for Tenant
applications, so UI and APIs can surface the number of LoadBalancer
external IPs without changing other behaviors.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Fix `Used` column to show correct data
- Removes table from `Info` page
- Add patch to let flatMap work with jsonpath
<img width="1800" height="940" alt="resource quota table example"
src="https://github.com/user-attachments/assets/421d1d32-fdc8-4269-838f-b02db0ed6712"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] fix resource quota table on tenant details page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed resource quota data fetching for tenant resources to use correct
namespace references from status fields.
* **Refactor**
* Refined dashboard layout for resource quota displays and improved
custom field resolution with enhanced dynamic key mapping for complex
data structures.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Removed reference to .Values.networking.apiServer which was removed
from values.yaml. Use empty defaults and let ConfigMap or
apiServerEndpoint parsing override them.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Extract common system packages shared between isp-full and isp-full-generic
into a helper template `cozystack.platform.system.common-packages`.
Packages moved to helper:
- kubeovn-webhook, kubeovn-plunger, cozy-proxy
- multus, metallb, reloader
- linstor-scheduler, snapshot-controller
Packages NOT in helper (differ between variants):
- networking (variant differs: kubeovn-cilium vs kubeovn-cilium-generic)
- linstor (talos.enabled differs)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Per review feedback, create separate manifest files for different
deployment targets instead of using templated values:
- cozystack-operator.yaml (Talos): hardcoded localhost:7445 (KubePrism)
- cozystack-operator-generic.yaml: reads from cozystack-operator-config
ConfigMap (user must create before applying)
- cozystack-operator-hosted.yaml: no env override (uses in-cluster SA)
This keeps installation flow clean - users apply the manifest matching
their deployment target without needing to modify original files.
Build system updated to generate all three variants.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Remove apiServer and cilium.cgroup from user-facing values.yaml
- Hardcode Talos-specific values (localhost:7445, cgroup autoMount: false)
directly in isp-full bundle template
- isp-full-generic already hardcodes its own values
- Improve MASTER_NODES comment explaining helm lookup behavior
This hides implementation details from users as requested in review.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The CRD upgrade pre-install job has image compatibility issues.
CRDs are installed as part of the Helm chart install anyway.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
rancher/kubectl is a minimal image without shell.
alpine/k8s includes kubectl and shell for running upgrade jobs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bitnami images are forbidden and bitnamilegacy/kubectl:1.35 doesn't exist.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The enabledPackages list must use fully qualified package names
(e.g., cozystack.velero) to match the helper template check.
Also add backupstrategy-controller which is required by backup-controller.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add velero to enabledPackages list for isp-full-generic to enable
backup functionality by default.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Allow iaas, paas, naas bundles when using isp-full-generic variant
- Enable all bundles by default in values-isp-full-generic.yaml
- Update bundle templates to accept isp-full-generic alongside isp-full
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The cozystack-values secret was using the default publishing.host value
from values.yaml (example.org) instead of reading from the cozystack
ConfigMap where the actual root-host is configured.
Add ConfigMap lookup to apps.yaml to read root-host from cozystack
ConfigMap, falling back to chart values if not present.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Same issue as lineage-controller-webhook: DaemonSet uses hardcoded
nodeSelector with empty value, but k3s/kubeadm use value "true".
Changes:
- Add nodeSelector to cozystack-api values (default: empty for Talos)
- Update deployment template to use configurable nodeSelector
- Pass nodeSelector for both cozystack-api and lineage-controller-webhook
in isp-full-generic bundle variant
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On Talos, control-plane nodes have label node-role.kubernetes.io/control-plane
with empty value. On generic k8s (k3s, kubeadm), the same label has value "true".
The lineage-controller-webhook DaemonSet was hardcoded to use empty value,
causing 0 pods scheduled on k3s clusters.
Changes:
- Add nodeSelector to lineage-controller-webhook values (default: empty for Talos)
- Update DaemonSet template to use configurable nodeSelector
- Pass nodeSelector with value "true" for isp-full-generic bundle variant
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
k3s and kubeadm set node-role.kubernetes.io/control-plane=true,
while Talos uses an empty value. KubeOVN node selector needs
the exact label value to match.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
For isp-full-generic bundle variant, the template now looks up the
cozystack ConfigMap to read cluster-specific settings instead of only
relying on chart default values.
This fixes the issue where Cilium was trying to connect to localhost:7445
(Talos KubePrism default) instead of the actual API server endpoint.
Values read from ConfigMap:
- api-server-endpoint: parsed to extract host/port for Cilium
- ipv4-pod-cidr, ipv4-pod-gateway, ipv4-svc-cidr, ipv4-join-cidr: for KubeOVN
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Remove trailing dash from {{- end -}} constructs in _helpers.tpl to
preserve trailing newlines after each rendered package YAML document.
Without this fix, consecutive packages are rendered without proper
separators:
variant: default---
apiVersion: cozystack.io/v1alpha1
This causes YAML parse errors: "mapping key 'apiVersion' already
defined" because the YAML parser sees a single document with duplicate
keys instead of two separate documents.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The isp-full-generic variant should have iaas, paas, and naas bundles
disabled by default since these bundles require 'isp-full' or 'isp-hosted'
variants as per validation rules in their respective templates.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Create dedicated values file for isp-full-generic bundle variant with
bundles.system.variant set to "isp-full-generic" instead of "isp-full".
This fixes the ArtifactGenerator generating identical digests for both
isp-full and isp-full-generic variants, which caused the wrong bundle
template to be selected.
Also adds isp-full-generic variant to installer PackageSource definition.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The isp-full-generic bundle now automatically extracts host and port
from publishing.apiServerEndpoint URL for Cilium and KubeOVN
configuration. This eliminates the need for manual networking.apiServer
values when api-server-endpoint is set in the ConfigMap.
Changes:
- Parse apiServerEndpoint URL to extract host and port for Cilium
- Auto-detect MASTER_NODES for KubeOVN from apiServerEndpoint if not
explicitly set
- Hardcode cgroup.autoMount=true for generic k8s (non-Talos)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add isp-full-generic bundle variant for deploying Cozystack on non-Talos
Kubernetes clusters (kubeadm, k3s, RKE2, etc.).
This variant:
- Uses kubeovn-cilium-generic networking (no values-talos.yaml)
- Passes talos.enabled=false to linstor (keeps DRBD init containers)
- Supports all standard networking configuration via platform values
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add cilium-generic and kubeovn-cilium-generic variants that exclude
values-talos.yaml. These variants are suitable for kubeadm, k3s, RKE2
and other non-Talos Kubernetes distributions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
k3s and RKE2 include distribution suffixes in version string
(e.g., v1.35.0+k3s1) which are not valid container image tags.
Strip everything after '+' using regexReplaceAll to produce clean
version tags like v1.35.0.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add talos.enabled value (default: true) to control whether
LinstorSatelliteConfiguration cozystack-talos is created.
This allows deploying linstor on non-Talos clusters (Ubuntu, Debian, etc.)
where DRBD init containers are required.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Add networking.kubeovn.MASTER_NODES to platform values.yaml
- Conditionally include MASTER_NODES in kubeovn values when set
- Enables KubeOVN deployment on k3s/kubeadm without control-plane node labels
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add networking.apiServer.host and networking.apiServer.port to platform values
to allow overriding the default Talos KubePrism settings (localhost:7445).
Also add networking.cilium.cgroup.autoMount for non-Talos clusters that need
Cilium to mount cgroups automatically.
Changes:
- packages/core/platform/values.yaml: Add apiServer and cilium.cgroup settings
- packages/core/platform/templates/bundles/system.yaml: Pass cilium values
- packages/core/installer/values.yaml: Add kubernetesServiceHost/Port
- packages/core/installer/templates/cozystack-operator.yaml: Template env vars
Closes: #1933
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Fixes v1.0 migration script
### Release note
```release-note
V1.0 migration script fixed after an upgrade was performed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated bundle naming convention (paas → isp).
* Restructured configuration schema: flattened branding, renamed
scheduling field, and moved authentication redirect setting.
* Minor output text and prompt wording adjustments.
* **New Features**
* Added support for external IPs in generated configuration with safe
empty fallback.
* Consolidated branding and scheduling entries for simpler output.
* **Bug Fixes**
* Migration script now validates prerequisites and produces improved,
YAML-friendly output.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add migration script to convert existing MongoDB HelmReleases
from old users format to new users + databases format.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Add Tenant details page to dashboard
- Add Resource Quota usage to Tenant page in dashboard
<img width="1800" height="939" alt="resource quota in tenant"
src="https://github.com/user-attachments/assets/7f4f6ed6-e9b8-4258-a60b-15f203443a1f"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Add Tenant details page and resource quota usage to tenant page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Resource Quotas section to the Details tab to show allocation
limits and usage.
* Info and Tenant types now display quota data in an interactive table
with columns for resource, limits, and current usage.
* Enhanced quota table presentation with additional columns for
flattened resource keys and values for clearer reporting.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add "Edit" button to all resource pages in dashboard
fixes https://github.com/cozystack/cozystack/issues/1911
<img width="1974" height="1327" alt="edit button"
src="https://github.com/user-attachments/assets/8e28dd2d-b474-488b-a7ae-a6c60d23619f"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Add "Edit" button to all resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Edit action links to dashboard headers for quick access to edit
functionality.
* **UI/Layout**
* Improved dashboard header layout with enhanced spacing and center
alignment for better visual consistency.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes v1.0 packagesources.
### Release note
```release-note
V1.0 packagesources fixed after an upgrade was performed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated platform component configurations for cozy-proxy and
metrics-server deployments.
* Reorganized monitoring component namespace allocation.
* Added metrics server as a dependency in the monitoring pipeline.
* Integrated new system components (linstor-scheduler and
metrics-server) into the platform templates.
* Refined HelmRelease chart references for vertical pod autoscaler.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Align MongoDB configuration with postgres and mysql patterns:
- Add separate `databases` section with `roles.admin` and `roles.readonly`
- Simplify `users` to only contain optional password field
- All users now authenticate via `admin` database (MongoDB best practice)
- Admin role maps to readWrite + dbAdmin permissions
- Readonly role maps to read permission
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Add missing changelog documentation for recent patch releases that were
released without changelog entries.
### Release note
```release-note
[docs] Add changelog documentation for v0.40.3, v0.41.1, v0.41.2, v0.41.3
```
Add missing changelog documentation for recent patch releases.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fixes telemetry collection in cozystack-operator which was failing
because the manager's cache is filtered and doesn't include resources
needed for telemetry (kube-system namespace, nodes, services, PVs).
Switches to using `mgr.GetAPIReader()` which bypasses the cache.
### Release note
```release-note
[system] Fix telemetry collection in cozystack-operator
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated Docker image versions from 1.5.0 to 1.6.1 for platform
migrations and testing environments.
* Enhanced telemetry collection system with improved data access
patterns.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch enables the `offline_access` scope for the dashbord keycloak
client, so that users get a refresh token which gatekeeper can use to
automatically refresh an expiring access token. Also session timeouts
were increased.
### Release note
```release-note
[dashboard] Increase session timeouts, add the offline_access scope,
enable refresh tokens to improve the overall user experience when
working with the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated authentication configuration with additional OAuth2 scope
parameters to enhance security credentials handling.
* Enhanced Keycloak client configuration with optional client scopes and
session management attributes, including logout redirect settings and
session timeout controls for improved session lifecycle management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozystack-operator's manager cache is filtered to only include
specific secrets and namespaces with certain labels. This caused
telemetry collection to fail because resources like kube-system
namespace, nodes, services, and PVs were not in the cache.
Switch to using mgr.GetAPIReader() which bypasses the cache and
queries the API server directly. This is appropriate for telemetry
since it only runs every 15 minutes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Introduce a new "External IPs" sidebar and associated dashboard factory
to display services of type LoadBalancer.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add External IPs tab to sidebar with all `Service` resources of type
`LoadBalancer` in tenant
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] add External IPs tab with `Service` resources of type `LoadBalancer`
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added External IPs menu item in the Administration section for quick
access to load balancer services
* Introduced a new dashboard view displaying external IPs with tabbed
interface for enhanced visibility
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add labels and selectors for Service and Ingress resources to correctly
show them in corresponding tabs for Kubernetes app in dashboard
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] show Ingress and Service resources in dashboard
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Introduced standardized metadata labels across Kubernetes application
resources including cloud configuration, ingress, and service components
to improve resource organization, tracking, identification, and
multi-tenant environment management
* Extended the system's resource management framework to include and
manage additional Kubernetes infrastructure components, such as
ingress-nginx, enhancing overall cluster orchestration and service
networking capabilities
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Fixes filtering pods on Pods tab of Service details page
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Fix filtering pods on Pods tab for Service
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected label selector field name in pod details table configuration
to ensure proper label filtering functionality.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch enables the `offline_access` scope for the dashbord keycloak
client, so that users get a refresh token which gatekeeper can use to
automatically refresh an expiring access token. Also session timeouts
were increased.
### Release note
```release-note
[dashboard] Increase session timeouts, add the offline_access scope,
enable refresh tokens to improve the overall user experience when
working with the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Introduce a new "External IPs" sidebar and associated dashboard factory to display services of type LoadBalancer.
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Set minReplicas for VPA for VMAgent to work correctly in the default
case when replicas = 1
Fixes https://github.com/cozystack/cozystack/issues/1893
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[monitoring-agents] set minReplicas=1 in VPA for cluster-wide VMAgent to allow recrating single replica
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Changes
* **Chores**
* Updated system monitoring autoscaler configuration to enforce minimum
replica constraints, enhancing service resilience and availability
during scaling operations.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This reverts commit 73d6e3013e.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Revert wrong type change according review
[request](https://github.com/cozystack/cozystack/pull/1873#discussion_r2717088708)
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[fix strategyRef type in backups]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Breaking Changes**
* Backup strategy references have been updated from cluster-scoped to
namespace-scoped references. This change enables better namespace
isolation and control. Update your backup configurations to reference
strategies within the appropriate namespace.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes the ability for users to specify custom container images
(`images.pmm` and `images.backup`) in the MongoDB application values.
This is a security hardening measure - allowing users to specify
arbitrary container images could lead to:
- Running malicious or compromised images
- Supply chain attacks through untrusted image registries
- Privilege escalation if malicious images are designed to exploit the
cluster
The images are now hardcoded in the template:
- `percona/pmm-client:2.44.1`
- `percona/percona-backup-mongodb:2.11.0`
### Release note
```release-note
[apps] Remove user-configurable images from MongoDB chart for security hardening
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Removed container image customization options for MongoDB's monitoring
and backup components. The system now uses fixed, pre-determined image
versions (PMM Client 2.44.1 and Percona Backup MongoDB 2.11.0) to ensure
deployment consistency and reduce configuration complexity.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add documentation for the resourceQuotas parameter explaining
supported resource types and their usage.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
Closes#1880
```release-note
Added BackupClass into BackupJob and BackupPlan for simplify UX.
```
The following functions related StorageRef were removed as they are no
longer used after migrating to BackupClass API:
// - resolveBucketStorageRef: Previously resolved S3 credentials from
Bucket storageRef
// - createS3CredsForVelero: Previously created Velero S3 credentials
secrets
// - createBackupStorageLocation: Previously created Velero
BackupStorageLocation resources
// - createVolumeSnapshotLocation: Previously created Velero
VolumeSnapshotLocation resources
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added BackupClass CRD and backupClassName field; webhook enforces and
defaults/validates backupClassName.
* **Refactor**
* Removed inline storageRef/strategyRef from specs; controllers resolve
strategy and parameters from BackupClass. Application apiGroup defaults
to apps.cozystack.io. Velero flow now uses resolved parameters.
* **Tests**
* Added unit tests for BackupClass resolution, BackupJob/Plan factory,
webhook validation, and Velero template rendering.
* **Chores**
* Manager cache/indexing updated to track BackupClass and index
BackupJobs by backupClassName.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Removes Tenant `isolated` flag and enforces isolation using network
policies
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[tenant] Removed Tenant field `isolated` and enforced network isolation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Network policies for tenant namespace isolation are now always
enforced and cannot be disabled.
* Removed the `isolated` configuration parameter from all tenant
settings, schema definitions, and documentation.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove the ability for users to specify custom container images
(images.pmm and images.backup) in the MongoDB application values.
This is a security hardening measure - allowing users to specify
arbitrary container images could lead to running malicious or
compromised images, supply chain attacks, or privilege escalation.
The images are now hardcoded in the template:
- percona/pmm-client:2.44.1
- percona/percona-backup-mongodb:2.11.0
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add enum validation for addon `ingressNginx.exposeMethod` field
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] add enum validation for ingressNginx `exposeMethod` field
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Enhanced validation for Kubernetes ingress configuration to enforce
allowed exposure methods ("Proxied" or "LoadBalancer"). This ensures
configurations are properly restricted to supported options and prevents
invalid settings from being accepted.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[ci] Run e2e tests on shared runners
```
This PR prepares the release `v1.0.0-alpha.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container images and system components across all major
platform services. Includes infrastructure upgrades for networking,
storage systems, application APIs, data backup services, system
monitoring, and supporting components. All updates incorporate the
latest security patches, critical stability improvements, and
performance optimizations to enhance overall system reliability,
operational efficiency, and platform stability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add Feature Highlights section with MongoDB as the main feature of
v0.41.0
- Add Documentation section with website repository changes to both
v0.41.0 and v1.0.0-alpha.2
- Add @matthieu-robin to v1.0.0-alpha.2 contributors
Fixes#749
## Changes
- Add documentation explaining how to enable Hubble for network
observability in Grafana
- Add four Grafana dashboards for Hubble metrics visualization
(Overview, DNS, L7 HTTP, Network)
- Register Hubble dashboards in the monitoring package
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Hubble DNS Overview dashboard for namespace-level DNS
metrics visibility.
* Introduced Hubble L7 HTTP Metrics dashboard for HTTP request and
performance monitoring.
* Introduced Hubble Network Overview dashboard for network flows and
policy insights.
* **Documentation**
* Added comprehensive Hubble observability guide covering setup,
configuration, and troubleshooting.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds MongoDB as a managed application to Cozystack using Percona
Operator for MongoDB (Apache 2.0).
**New packages:**
- `packages/system/mongodb-operator/` - Percona psmdb-operator
deployment
- `packages/apps/mongodb/` - MongoDB application chart
- `packages/system/mongodb-rd/` - CozystackResourceDefinition for
dashboard
**Features:**
- Replica set mode with configurable replica count
- Sharded cluster mode (config servers, mongos routers, multiple shards)
- S3 backup integration with cron scheduling
- Point-in-time recovery from backups
- External access via LoadBalancer
- Multiple MongoDB versions (v6, v7, v8)
- Resource presets (nano to 2xlarge)
- User management with role-based access
### Release note
```release-note
[apps] Add MongoDB as a managed application with replica set, sharding, and S3 backup support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Managed MongoDB service and operator packaging; replica sets with
optional sharding, external access, per-user credentials, workload
monitoring, and version mappings for provider images
* Backup & restore with S3 support and point-in-time recovery
* **Documentation**
* Comprehensive README plus Helm values and schema for configuration
* **Tests**
* Extensive unit/template tests and end-to-end tests covering
credentials, services, backup, restore, CR rendering, and operator
install
* **Packaging**
* Charts and bundle entries added for operator and managed service
installation
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kilo] Introduce kilo
```
## What this PR does
This PR introduces a new flux-plunger controller that automatically
fixes HelmRelease resources experiencing "has no deployed releases"
errors by cleaning up corrupted Helm release secrets.
### Problem
When Helm releases fail with "has no deployed releases" error, FluxCD
cannot recover automatically. This typically happens when:
- Helm release secrets are corrupted
- Failed upgrades leave the release in an inconsistent state
- Manual intervention is required to delete the problematic secret
### Solution
The flux-plunger controller:
1. Watches HelmRelease resources for "has no deployed releases" errors
2. Suspends the HelmRelease (with `flux-client-side-apply` field
manager)
3. Identifies and deletes the latest (problematic) Helm release secret
4. Tracks processed versions via annotation to prevent recursive
deletion
5. Unsuspends the HelmRelease to allow FluxCD to retry
**Key features:**
- Only deletes one secret at a time (prevents cascading deletions)
- Uses version tracking: if `latest+1 == processed`, skips (already
handled)
- Handles suspended HelmReleases: only unsuspends if it was suspended by
flux-plunger itself
- Gracefully handles invalid annotations by treating them as unprocessed
### Components
- **Controller:** `internal/controller/fluxplunger/flux_plunger.go`
- **Main:** `cmd/flux-plunger/main.go`
- **Helm Chart:** `packages/system/flux-plunger/`
- **Platform Integration:** Added to `paas-full.yaml` bundle
### Release note
```release-note
[platform] Add flux-plunger controller to automatically fix HelmRelease resources with "has no deployed releases" error by cleaning up corrupted Helm secrets
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced a new controller that automatically resolves HelmRelease
resource failures with built-in health probes and graceful recovery
procedures.
* Enabled by default in system installations.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds a new package `local-ccm` to the Cozystack platform.
The local-ccm package provides a Cloud Controller Manager for local
development and testing environments. It integrates the upstream Helm
chart from https://github.com/cozystack/local-ccm (version 0.2.0).
**Changes:**
- Created package structure in `packages/system/local-ccm/`
- Added Makefile with automated upstream chart synchronization from
GitHub
- Added PackageSource definition in
`packages/core/platform/sources/local-ccm.yaml`
- Configured dependency on networking components
- Set privileged mode as required for cloud controller operations
**Package details:**
- Namespace: `cozy-local-ccm`
- Release name: `local-ccm`
- Upstream: https://github.com/cozystack/local-ccm
### Release note
```release-note
[local-ccm] Add local-ccm package for cloud controller manager in local environments
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Local Cloud Controller Manager package providing automatic node
IP detection, internal and external IP support, initialization taint
removal, and continuous node reconciliation with configurable intervals.
* **Chores**
* Updated system component tolerations and scheduling policies to
improve compatibility with cloud infrastructure, including better
handling of node readiness, network initialization, and cloud provider
states.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes the redundant `cozystack.io/ui=true` label from HelmReleases.
The existing `apps.cozystack.io/application.kind`,
`apps.cozystack.io/application.group`, and
`apps.cozystack.io/application.name` labels already provide more
specific identification and are sufficient for filtering.
Updates all label selectors to use `apps.cozystack.io/application.kind`
instead.
### Release note
```release-note
[platform] Remove cozystack.io/ui label - use apps.cozystack.io/application.kind for filtering instead
```
## What this PR does
Splits telemetry collection between cozystack-operator and
cozystack-controller for better separation of concerns.
### Release note
```release-note
[platform] Split telemetry collection between cozystack-operator and cozystack-controller
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added telemetry support to the operator with configurable options
(disable, endpoint, interval).
* Enhanced telemetry to track application kinds and UI-managed releases.
* **Bug Fixes**
* Removed the runtime cozystack-version flag; version is now managed at
build time.
* **Chores**
* Removed unused YAML dependency.
* Simplified build process with improved Makefile commands.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- Fix view of loadbalancer ip in services window
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed LoadBalancer IP column display to correctly retrieve IP
addresses from the updated data source in the dashboard details view.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases the `startupProbe.failureThreshold` for kube-apiserver
container from 3 to 30.
The default 30 seconds (3 failures × 10s period) is often insufficient
for kube-apiserver to complete RBAC bootstrap, especially:
- Under high load conditions
- With slower etcd connections
- During initial cluster provisioning
This change gives kube-apiserver up to 300 seconds to initialize,
preventing unnecessary CrashLoopBackOff cycles.
### Release note
```release-note
[kubernetes] Increase kube-apiserver startup probe threshold to prevent CrashLoopBackOff during slow RBAC initialization
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated startup probe threshold configuration to enhance system
initialization stability across core components.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Changes the default `resourcesPreset` for kube-apiserver from "medium"
to "large" to prevent OOM kills during normal operation. The "medium"
preset provides insufficient memory for clusters with moderate
workloads, leading to repeated crashes and service disruption.
### Release note
```release-note
[kubernetes] Change default apiServer resourcesPreset from "medium" to "large" to prevent OOM kills
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Configuration Changes**
* Updated default API Server resource preset from medium to large for
enhanced resource allocation
* Removed version property from Kubernetes chart configuration schema
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add new package local-ccm from https://github.com/cozystack/local-ccm
- Create package structure in packages/system/local-ccm
- Add Makefile with upstream chart sync from GitHub
- Add PackageSource definition in packages/core/platform/sources/
- Package provides cloud controller manager for local environments
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The apps.cozystack.io/application.kind, apps.cozystack.io/application.group,
and apps.cozystack.io/application.name labels already exist on all HelmReleases
and provide more specific identification. Remove the redundant cozystack.io/ui
label and update all selectors to use apps.cozystack.io/application.kind instead.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add MongoDB managed service based on Percona Operator for MongoDB with:
- Replica set mode (default) and sharded cluster mode
- Configurable replicas, storage, and resource presets
- Custom users with role-based access control
- S3-compatible backup with PITR support
- Bootstrap/restore from backup
- External access support
- WorkloadMonitor integration for dashboard
- Comprehensive helm-unittest test coverage (91 tests)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Increase startupProbe.failureThreshold from 3 to 30 for kube-apiserver
container. The default 30 seconds (3 failures * 10s period) is often
insufficient for apiserver to complete RBAC bootstrap, especially under
load or with slower etcd connections.
This change gives kube-apiserver up to 300 seconds to initialize,
preventing unnecessary CrashLoopBackOff cycles.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add flux-plunger controller to automatically fix HelmRelease resources
with "has no deployed releases" error.
The controller watches HelmRelease resources and performs the following:
- Detects HelmRelease with "has no deployed releases" error
- Suspends the HelmRelease with flux-client-side-apply field manager
- Deletes the latest Helm release secret
- Updates annotation with processed version to prevent recursive deletion
- Unsuspends the HelmRelease to allow Flux to retry
Special handling for suspended HelmRelease:
- If suspend=true and latest+1==processed: removes suspend
- Otherwise: skips processing as suspended by external process
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR updates Talos Linux which includes DRBD fixes:
- https://github.com/LINBIT/drbd/releases/tag/drbd-9.2.15
- https://github.com/LINBIT/drbd/releases/tag/drbd-9.2.16
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update Talos Linux v1.11.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Bumped Talos profile version from v1.11.3 to v1.12.1 across all
configuration profiles
* Updated baseInstaller and system extensions with latest firmware and
driver images
* Added output format settings to profile configurations
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases etcd probe thresholds to allow more time for cluster
synchronization after pod restarts. This addresses issues where etcd
members were being killed by startup probes before they could fully sync
with the cluster, especially when VPA assigns minimal resources.
Changes:
- `startupProbe.failureThreshold`: 3 → 300 (allows 25 minutes for
initial sync)
- `livenessProbe.failureThreshold`: default → 10 (reduces unnecessary
restarts)
- `readinessProbe.failureThreshold`: default → 3
### Release note
```release-note
[etcd] Increase probe thresholds to prevent premature pod termination during cluster synchronization
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved etcd service stability with automatic health monitoring and
failover detection capabilities in Kubernetes environments.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Change the default resourcesPreset for kube-apiserver from "medium" to
"large" to prevent OOM kills during normal operation. The "medium"
preset provides insufficient memory for clusters with moderate workloads.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Increase startup probe failureThreshold to 300 (25 minutes) to allow
etcd members more time to sync with the cluster after restart or
recovery. This prevents pods from being killed during initial
synchronization when VPA assigns minimal resources.
Also increase liveness probe failureThreshold to 10 to reduce
unnecessary restarts during temporary network issues.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Updates cilium component to version 1.18.6
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cilium] update to v1.18.6
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded Cilium to v1.18.6 and updated container images and checksums
across components (Hubble Relay, Operator, Preflight, Cluster Mesh API
Server, SPIRE, etc.).
* Updated Envoy to v1.35.9 with new image digest.
* **Bug Fixes / Behavior**
* Adjusted preflight/Envoy gating: preflight-enabled now suppresses
certain Envoy bootstrap config and removed the preflight Envoy bootstrap
config mount.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes aggregated API server Watch implementation to properly work with
controller-runtime informers. External controllers watching Tenant
resources via the aggregated API were experiencing issues with cache
sync timeouts and missing reconciliations on startup.
### Changes
**ResourceVersion handling in List:**
- Compute ResourceVersion from items when the cached client doesn't set
it on the list itself
**Bookmark handling:**
- Pass through bookmark events with converted types for proper informer
sync
**ADDED event filtering (main fix):**
- Simplified the filtering logic that was incorrectly skipping initial
events
- Only skip ADDED events when `startingRV > 0` AND `objRV <= startingRV`
(client already has from List)
- When `startingRV == 0`, always send ADDED events (client wants full
state)
- Removed the complex `initialSyncComplete` tracking that had inverted
logic
### Problem
When a controller starts watching resources, controller-runtime may call
Watch with `resourceVersion=""`. The server should send all existing
objects as ADDED events. The previous `initialSyncComplete` logic was
inverted and could skip these events, causing objects (like Tenants with
lock annotations) to not be reconciled on controller startup.
### Release note
```release-note
[apiserver] Fix Watch resourceVersion and bookmark handling for controller-runtime compatibility
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Enhanced resource watching and synchronization with Kubernetes 1.27+
compatibility, including proper handling of initial events and
bookmarks.
* Optimized event filtering and resource version tracking for
applications, tenant modules, tenant namespaces, and tenant secrets to
reduce unnecessary event noise.
* Improved list metadata consistency by deriving accurate resource
versions when unavailable in responses.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Add resourceVersion handling for Watch requests by filtering ADDED events
based on the resourceVersion provided by the client
- Forward bookmark events from underlying HelmRelease watchers to clients
for proper resourceVersion synchronization
- Extract MaxResourceVersion helper using meta.EachListItem for cleaner code
- This ensures clients don't receive duplicate objects they already have
from List+Watch patterns
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Update changelog for v1.0.0-alpha.1 to include missing features:
- **Cozystack Operator**: New operator for Package and PackageSource
management (#1740, #1741, #1755, #1756, #1760, #1761)
- **Backup System**: Comprehensive backup functionality with Velero
integration (#1640, #1685, #1687, #1708, #1719, #1720, #1737, #1762)
- Add @androndo to contributors
- Update Full Changelog link to v0.38.0...v1.0.0-alpha.1
### Release note
```release-note
[docs] Update changelog for v1.0.0-alpha.1: add cozystack-operator and backup system
```
This PR prepares the release `v1.0.0-alpha.1`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container images for core system components to v1.0.0-alpha.1
* Upgraded Cilium networking from 1.17.8 to 1.18.5
* Updated infrastructure component images including Metallb, Linstor,
and storage services with latest digests for enhanced security and
stability
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add file existence checks to talos asset targets to prevent
rebuilding kernel, initramfs, installer, iso, nocloud, and metal
assets when they already exist from a previous build step.
This fixes duplicate talos builds in the release pipeline where
`make build` and `make assets` both triggered talos image generation.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Renames `CozystackResourceDefinition` CRD to `ApplicationDefinition` for
better clarity and consistency.
Changes include:
- Renamed all Go types (ApplicationDefinition,
ApplicationDefinitionList, ApplicationDefinitionSpec, etc.)
- Renamed controller files and reconciler structs
- Updated all packages that use these types (dashboard,
lineagecontrollerwebhook, crdmem, etc.)
- Renamed CRD Helm chart from `cozystack-resource-definition-crd` to
`application-definition-crd`
- Updated all 25 cozyrds YAML manifests to use `kind:
ApplicationDefinition`
- Added migration 23 to remove old `cozystack-resource-definition-crd`
HelmRelease
### Release note
```release-note
[api] Rename CozystackResourceDefinition CRD to ApplicationDefinition
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Renamed the primary custom resource from CozystackResourceDefinition
to ApplicationDefinition across the platform, dashboards, controllers,
and manifests.
* Updated packaging and Helm charts to use ApplicationDefinition
consistently.
* **Chores**
* Bumped platform migration target to v24 and added a migration to apply
the new resource CRD.
* **Bug Fix**
* Restored deep-copy support for Component-related types.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Rename the CRD and all related types for better clarity:
- CozystackResourceDefinition -> ApplicationDefinition
- CozystackResourceDefinitionList -> ApplicationDefinitionList
- CozystackResourceDefinitionSpec -> ApplicationDefinitionSpec
- All related nested types updated accordingly
Updated components:
- API types and generated deepcopy code
- Controllers and reconcilers
- Dashboard, lineagecontrollerwebhook, crdmem packages
- CRD YAML definition and Helm chart
- All 25 cozyrds YAML manifests
- Migration scripts and documentation
Added migration 23 to remove old cozystack-resource-definition-crd HelmRelease.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
- Add pattern rule for building cozypkg binaries per platform
(`assets-cozypkg-<os>-<arch>`) with checksums generation
- Add Version variable to cozypkg CLI injected via ldflags
- Split manifests into separate `cozystack-crds.yaml` and
`cozystack-operator.yaml` files
- Update CI workflow to handle CRDs and operator as separate artifacts
- Update e2e tests and upload script for new manifest structure
### Release note
```release-note
[ci] Add cross-platform cozypkg build targets with version injection and split installer manifests into CRDs and operator
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Multi-platform binaries (Linux, macOS, Windows; amd64 & arm64)
* Checksum file added for release artifact verification
* **Refactor**
* Installation flow reworked to validate and apply CRDs and Operator
manifests separately
* CLI exposes a configurable build-time version for reporting
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Add pattern rule for building cozypkg binaries per platform
(assets-cozypkg-<os>-<arch>) with checksums generation
- Add Version variable to cozypkg CLI injected via ldflags
- Split manifests into separate cozystack-crds.yaml and
cozystack-operator.yaml files
- Update CI workflow to handle CRDs and operator as separate artifacts
- Update e2e tests and upload script for new manifest structure
- Suppress git describe stderr when no tags exist
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
- Move `common-envs.mk` and `package.mk` from `scripts/` to `hack/`
directory
- Update all Makefile includes to use new paths
- Remove unused `issue-flux-certificates.sh` script
### Release note
```release-note
[refactor] Move build scripts from scripts/ to hack/ directory
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Reorganized build infrastructure by consolidating helper scripts and
configuration paths.
* Removed deprecated script from the build tooling.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Move common-envs.mk and package.mk from scripts/ to hack/ directory.
Update all Makefile includes to use new paths. Remove unused
issue-flux-certificates.sh script.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Replace HelmRelease-based bundle system with Package resources managed
by cozystack-operator. Restructure values.yaml with full configuration
support.
## What this PR does
- Restructure values.yaml with full configuration (networking,
publishing, authentication, scheduling, branding, resources)
- Add values-isp-full.yaml and values-isp-hosted.yaml for bundle
variants
- Create templates/packages/isp-full.yaml with Package resources
- Move PackageSources from sources/ to templates/sources/
- Remove old bundle files and HelmRelease templates
- Add hack/migrate-to-version-1.0.sh migration script for converting
ConfigMaps to Package resource
### Release note
```release-note
[platform] Migrate from HelmRelease bundles to Package-based deployment
```
Add dedicated flux-tenants controller with label selector
--watch-label-selector=sharding.fluxcd.io/key=tenants to handle
tenant workloads separately from platform components.
Update all kubernetes app HelmReleases to:
- Use chartRef with ExternalArtifact instead of OCIRepository sourceRef
- Add sharding.fluxcd.io/key=tenants label
- Add cozystack.io/target-cluster-name label
Update fluxinstall to parse multiple YAML manifest files and
use flux service for storage-adv-addr.
Add cozystack-basics package for core tenant/namespace setup.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Restructure platform bundles from monolithic files to modular
directory structure with separate applicationdefinitions.
Add PackageSources for better dependency management and
migrate from legacy HelmRepositories to new repository format.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update all CozystackResourceDefinition files to use chartRef
with ExternalArtifact instead of OCIRepository sourceRef.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Remove legacy installer components (cozystack-assets-server,
installer.sh script, cozystack container image) in favor of
cozystack-operator based deployment.
Move migration scripts from scripts/migrations/ to
packages/core/platform/images/migrations/ for containerized execution.
Add grafana-dashboards image for centralized dashboard management.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Replace the chart field with chartRef for referencing Helm charts via
ExternalArtifact resources. This enables the Package controller to
manage chart sources centrally.
Changes:
- Add chartRef field to CozystackResourceDefinition spec
- Remove chart field (deprecated)
- Remove validation (moved to controller)
- Update lineage mapper for new field structure
- Regenerate openapi specs
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Fix getVersion to parse "0.1.4+abcdef" format (with "+" separator)
instead of incorrectly looking for "sha256:" prefix.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
upstream PR https://github.com/piraeusdatastore/linstor-csi/pull/403
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[linstor] Refactor node-level RWX validation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a command-line option to disable strict RWX block volume
validation for specialized deployments.
* Centralized RWX validation into a shared utility used by the driver.
* **Improvements**
* Improved VM identification and pod/PV handling for RWX block
attachments, including hotplug scenarios.
* **Tests**
* Added comprehensive unit tests covering RWX validation and related
pod/PV scenarios.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes node-level RWX block validation from linstor-csi as
controller-level check is sufficient. The controller already validates
that all pods attached to RWX block volume belong to the same VM by
extracting vmName from pod owner references (VirtualMachineInstance).
This simplifies the validation logic and fixes VM live migration issues.
### Release note
```release-note
[linstor] Remove node-level RWX block validation to fix VM live migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Enhanced RWX (read-write-many) block validation with VM-aware checks
across node and controller flows, including support for hotplug-disk
pods and stricter prevention of cross-VM block sharing.
* Improved propagation and resolution of VM identity for attachments to
ensure consistent validation.
* **Tests**
* Added comprehensive unit tests covering single/multiple pod scenarios,
VM ownership, hotplug disks, upgrade paths, and legacy volumes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR updates piraeus-server patches to address several critical
production issues with DRBD resources and LUKS encryption:
1. **Add fix-duplicate-tcp-ports.diff** - Prevents duplicate TCP ports
after toggle-disk operations (upstream PR #476)
2. **Update skip-adjust-when-device-inaccessible.diff** - Comprehensive
fix for multiple issues:
- Resources stuck in StandAlone state after node reboot
- Unknown state race condition during satellite restart
- Encrypted LUKS resource deletion failures
- Network reconnect blocked by unavailable child device checks
These patches resolve scenarios where DRBD resources fail to
automatically reconnect after node reboots and improve LUKS resource
lifecycle management.
Upstream PRs:
- https://github.com/LINBIT/linstor-server/pull/476
- https://github.com/LINBIT/linstor-server/pull/477
### Release note
```release-note
[linstor] Fix DRBD resources stuck in StandAlone state after reboot and encrypted resource deletion issues
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Prevents duplicate TCP port conflicts after disk toggle operations
* Fixes resources stuck in StandAlone or Unknown state after reboot
* Resolves issues with encrypted resource deletion
* Improves handling of temporarily inaccessible storage devices
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove node-level RWX block validation from linstor-csi as
controller-level check is sufficient. The controller already validates
that all pods attached to RWX block volume belong to the same VM by
extracting vmName from pod owner references (VirtualMachineInstance).
This simplifies the validation logic and fixes VM live migration issues.
Update linstor-csi image tag with rebuilt image containing the fix.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update piraeus-server patches to address critical production issues:
- Add fix-duplicate-tcp-ports.diff to prevent duplicate TCP ports
after toggle-disk operations (upstream PR #476)
- Update skip-adjust-when-device-inaccessible.diff with comprehensive
fixes for resources stuck in StandAlone after reboot, Unknown state
race condition, and encrypted LUKS resource deletion (upstream PR #477)
```release-note
[linstor] Fix DRBD resources stuck in StandAlone state after reboot and encrypted resource deletion issues
```
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This PR generates missing changelogs for releases v0.37.10,
v0.38.5-v0.38.8, v0.39.2-v0.39.4, and v0.40.0 following the instructions
from `docs/agents/changelog.md`.
### Release note
```release-note
[docs] Add missing changelogs for releases v0.37.10, v0.38.5-v0.38.8, v0.39.2-v0.39.4, and v0.40.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added release notes for versions v0.37.10 through v0.40.0, documenting
improvements to VM storage resize, dashboard form schema, Windows VM
scheduling, SeaweedFS updates, and various platform enhancements and bug
fixes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Generated changelogs for missing releases following changelog.md instructions:
- v0.37.10: Backports for dashboard schema fix and VM resize improvements
- v0.38.5-v0.38.8: Backports for SeaweedFS, Cilium, VM scheduling, and Multus fixes
- v0.39.2-v0.39.4: Backports for VM services, tenant policies, LINSTOR fixes, and Multus dependencies
- v0.40.0: Minor release with LINSTOR scheduler, SeaweedFS traffic locality, valuesFrom mechanism, auto-diskful, and version management systems
All changelogs include proper PR author attribution using GitHub CLI and user impact descriptions.
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fixes field selector filtering for registry resources (Applications,
TenantModules, TenantSecrets) when using kubectl with field selectors
like `--field-selector=metadata.namespace=tenant-kvaps` or
`metadata.name=test`.
Controller-runtime cache doesn't support field selectors natively, which
caused incorrect filtering behavior. This PR implements manual filtering
for `metadata.name` and `metadata.namespace` field selectors in List()
and Watch() methods.
Changes:
- Created `pkg/registry/fields` package with `ParseFieldSelector`
utility for common field selector parsing
- Refactored field selector logic in application, tenantmodule, and
tenantsecret registries to use the common implementation
- Implemented manual post-processing filtering after label-based queries
- Removed `Raw` field usage and field selectors from
`client.ListOptions`
### Release note
```release-note
[registry] Fix field selector filtering for kubectl queries with metadata.name and metadata.namespace selectors
```
Controller-runtime cache doesn't support field selectors, causing
incorrect filtering when using kubectl with field selectors like
--field-selector=metadata.namespace=tenant-kvaps or metadata.name=test.
Changes:
- Created pkg/registry/fields package with ParseFieldSelector utility
- Refactored field selector parsing logic in application, tenantmodule,
and tenantsecret registries to use common implementation
- Implemented manual filtering for metadata.name and metadata.namespace
in List() and Watch() methods
- Removed Raw field usage and field selectors from client.ListOptions
- Label selectors passed directly via LabelSelector field
Field selectors now properly filter resources by name and namespace
through manual post-processing after label-based filtering.
See: https://github.com/kubernetes-sigs/controller-runtime/issues/612
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds multus as a dependency as other CNIs
### Release note
```release-note
Adds multus as a dependency as other CNIs
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated component initialization ordering to ensure more consistent
and reliable platform startup sequencing.
* Streamlined dependency requirements for several core system components
to improve deployment reliability and reduce startup complexity.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds custom linstor-csi image build to the linstor package with
an important upstream patch for RWX block volume validation.
**Changes:**
- Add linstor-csi Dockerfile based on upstream linstor-csi master
- Import patch from upstream PR piraeusdatastore/linstor-csi#403 for RWX
block volume validation
- Update Makefile to build both piraeus-server and linstor-csi images
- Configure LinstorCluster CR to use custom linstor-csi image in CSI
controller and node pods
**RWX Validation Patch:**
The imported patch prevents misuse of DRBD allow-two-primaries by
ensuring RWX block volumes are only used by pods belonging to the same
KubeVirt VM during live migration. This prevents data corruption when
multiple non-related pods attempt to use the same RWX block volume.
**Upstream PR:**
https://github.com/piraeusdatastore/linstor-csi/pull/403
### Release note
```release-note
[linstor] Add custom linstor-csi image build with RWX block volume validation patch that prevents data corruption during KubeVirt live migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Integrated LINSTOR CSI with deployable CSI components and node init
helpers; packaged as a production-ready image.
* **Updates**
* Added build/publish flow for Piraeus server and LINSTOR CSI images;
chart values updated with new image references and tags.
* **Behavior changes**
* Node-side RWX block-volume pre-validation to prevent conflicting
multi-node/multi-writer access.
* **Tests**
* Comprehensive RWX validation tests covering multi-pod and VM
scenarios.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add custom linstor-csi image build to packages/system/linstor:
- Add Dockerfile based on upstream linstor-csi
- Import patch from upstream PR #403 for RWX block volume validation
(prevents misuse of allow-two-primaries in KubeVirt live migration)
- Update Makefile to build both piraeus-server and linstor-csi images
- Configure LinstorCluster CR to use custom linstor-csi image in
CSI controller and node pods
The RWX validation patch ensures that RWX block volumes with
allow-two-primaries are only used by pods belonging to the same
KubeVirt VM during live migration.
Upstream PR: https://github.com/piraeusdatastore/linstor-csi/pull/403
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Removes multus memory limit due to short but unpredictable and large
memory consumption in some cases, such as starting up after a node
reboot (reported up to 3Gi).
The root cause will be fixed in future releases.
### Release note
```release-note
Multus memory limit removed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated Multus system namespace configuration and DaemonSet naming for
improved environment organization.
* Adjusted container resource allocation: increased memory requests and
refined memory limits for optimized performance.
* Updated deployment template specifications to reflect infrastructure
changes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Update kube-ovn from v1.14.11 to v1.14.25.
Changes synced from upstream include:
- Updated chart templates
- New configuration options in values
### Release note
```release-note
[kube-ovn] Update to v1.14.25
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added tolerations support for VpcNatGateway resources
* Added automatic VLAN subinterface creation capability for provider
networks
* Enhanced installation documentation with OCI Registry and Talos Linux
deployment examples
* **Security Improvements**
* Applied runtime-default seccomp profiles across deployments
* Configured service account token mounting behavior for improved pod
security
* **Chores**
* Updated Kube-OVN to v1.14.25
* Updated Helm chart metadata and dependencies
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
After satellite restart with patched linstor-server, some DRBD resources
ended up in **Unknown** state. Investigation revealed a race condition
between `drbdadm adjust` completion and `updateDiscGran()` lsblk check:
1. `drbdadm adjust` completes successfully (21:53:49.791) - brings
devices up
2. `updateDiscGran()` immediately tries to check discard granularity
(21:53:49.799)
3. `/dev/drbd*` device node doesn't exist yet - kernel hasn't created it
4. `lsblk` fails with exit code 32: "not a block device"
5. `StorageException` interrupts DeviceManager cycle (21:53:50.811)
6. DRBD device remains in incomplete state → **Unknown**
### Timeline from logs
```
21:53:49.791 - [DeviceManager] Resource 'pvc-aafbd92a' [DRBD] adjusted ✓
21:53:49.799 - [lsblk_parser] ERROR: /dev/drbd1169 not a block device
21:53:49.804-50.807 - [lsblk_parser] 10+ retry errors (every ~100ms)
21:53:50.811 - [DeviceManager] ERROR: Error executing lsblk
21:53:50.878 - [DeviceManager] End cycle 9 (WITH ERROR!)
```
## Solution
Update `skip-adjust-when-device-inaccessible.diff` patch to add physical
device path existence check before calling `lsblk`:
- Check `Files.exists(devicePath)` in `updateDiscGran()` before lsblk
- If device doesn't exist yet → skip check, set `discGran =
UNINITIALIZED_SIZE`
- Next DeviceManager cycle (few seconds later) → device node available →
lsblk succeeds
This complements the existing patch which checks **child layer** devices
(LUKS/Storage) for deletion scenarios, while this fix addresses the
**DRBD device itself** during adjust operations.
## Testing
Manual `drbdadm up` on affected devices confirmed they were in down
state and brought them back to UpToDate, proving the issue was
incomplete device initialization.
## Related
- Upstream linstor-server PR:
https://github.com/LINBIT/linstor-server/pull/471#issuecomment-3723392917
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced robustness for storage operations by adding device
accessibility validation. Operations now gracefully skip when device
paths are unavailable or invalid, preventing unnecessary failures and
improving system resilience during device access issues.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
By its nature, the kubernetes client provisioned in keycloak is public:
anyone can read the client secret. Therefore the secret is not necessary
and the client should be explicitly provisioned as a public client, not
as a confidential client.
### Release note
```release-note
[keycloak] Change the kubernetes client to be public, without a client
secret.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Add OIDC enablement flag and optional host derivation from tenant
release; allow explicit management kubeconfig endpoint override.
* **Refactor**
* Convert Keycloak client to public mode and remove reliance on a
Kubernetes client secret, simplifying authentication configuration.
* Remove secret-based credential plumbing from deployment templates,
streamlining kubeconfig generation.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Enable DRBD auto-diskful options to automatically convert diskless nodes
to diskful when they remain in Primary state for an extended period.
### How it works
When LINSTOR is integrated with Kubernetes, the platform may schedule
workloads on nodes that don't have local storage replicas. In such
cases, DRBD operates in "diskless" mode, accessing data over the network
from nodes that have actual disk replicas.
The `auto-diskful` feature addresses this by:
1. **Monitoring Primary state duration**: When a diskless node holds a
DRBD resource in Primary state (actively using the volume) for more than
the configured time (30 minutes), LINSTOR automatically creates a local
disk replica on that node.
2. **Automatic cleanup**: With `auto-diskful-allow-cleanup` enabled,
when the resource is no longer in Primary state on that node, LINSTOR
can automatically remove the disk replica that was created, freeing up
storage space.
### Configuration
- `DrbdOptions/auto-diskful: 30` — Convert diskless to diskful after 30
minutes in Primary state
- `DrbdOptions/auto-diskful-allow-cleanup: true` — Allow automatic
removal of auto-created replicas when no longer needed
### Benefits
- Improves I/O performance for long-running workloads by creating local
replicas
- Reduces network traffic for frequently accessed data
- Automatic cleanup prevents storage waste from temporary replicas
### Release note
```release-note
[linstor] Enable auto-diskful to automatically create local replicas on diskless nodes that hold volumes in Primary state for more than 30 minutes, improving I/O performance for long-running workloads.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced an "auto diskful" option to automatically manage diskful
resources with configurable interval and optional cleanup.
* **Chores**
* Added default configuration values to enable and time the auto-diskful
behavior and to control automatic cleanup.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Add linstor-scheduler-extender for optimal pod placement on nodes with
LINSTOR storage.
### How it works
The linstor-scheduler consists of two components:
#### 1. Scheduler Extender
A custom Kubernetes scheduler that works alongside the default
kube-scheduler. When a pod requests LINSTOR-backed storage, the
scheduler extender communicates with the LINSTOR controller to find
nodes that have local replicas of the requested volumes, prioritizing
placement on nodes with existing data to minimize network traffic.
#### 2. Admission Webhook
A MutatingWebhookConfiguration that automatically sets `schedulerName:
linstor-scheduler` on pods that use LINSTOR CSI driver volumes. This
ensures that only pods with LINSTOR PVCs are routed through the custom
scheduler, while all other pods continue using the default scheduler.
The webhook inspects pod creation requests and checks if any of the
pod's PVCs use a StorageClass with the LINSTOR CSI provisioner
(`linstor.csi.linbit.com`). If so, it mutates the pod spec to use the
linstor-scheduler.
### Components
- `linstor-scheduler` package in `packages/system/linstor-scheduler/`
- PackageSource definition in
`packages/core/platform/sources/linstor-scheduler.yaml`
- Integration into `paas-full` and `distro-full` bundles
### Upstream PRs
- https://github.com/piraeusdatastore/helm-charts/pull/67 — fix:
KubeSchedulerConfiguration v1 support for Kubernetes 1.25+
- https://github.com/piraeusdatastore/helm-charts/pull/68 — feat:
admission webhook support with cert-manager
- https://github.com/piraeusdatastore/helm-charts/pull/69 — fix chart
template issues
### Release note
```release-note
[linstor] Add linstor-scheduler for optimal pod placement on nodes with local LINSTOR replicas. Includes admission webhook that automatically routes pods using LINSTOR CSI volumes to the custom scheduler.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added LINSTOR scheduler component to the platform, packaged as a Helm
chart and installable package.
* Includes admission webhook, scheduler extender, autoscaling support,
pod disruption budget, RBAC and service account integration, and Helm
test hooks.
* **Documentation**
* Added chart README and default configuration values for easy
deployment and customization.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Addes traffic locality capabilities to Seaweedfs.
### Release note
```release-note
Added traffic locality capabilities to Seaweedfs
Add physical device path existence check to prevent race condition where
lsblk is called before kernel creates DRBD device node after drbdadm adjust.
This fixes issue where resources ended up in Unknown state after satellite
restart because:
- drbdadm adjust completes successfully (brings devices up)
- updateDiscGran() immediately tries lsblk
- /dev/drbd* doesn't exist yet (kernel hasn't created node)
- lsblk fails with "not a block device" exit code 32
- StorageException interrupts DeviceManager cycle
- Device remains in incomplete state
The patch now checks Files.exists(devicePath) before lsblk, allowing
the check to be retried in next cycle when device node is available.
Upstream: https://github.com/LINBIT/linstor-server/pull/471
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
By its nature, the kubernetes client provisioned in keycloak is public:
anyone can read the client secret. Therefore the secret is not
necessary and the client should be explicitly provisioned as a public
client, not as a confidential client.
### Release note
```release-note
[keycloak] Change the kubernetes client to be public, without a client
secret.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Add representation for backup Plans, BackupJobs and Backups in the dashboard and a readme
describing, how to add further resources to the dashboard.
### Release note
```release-note
[backups,dashboard] Add backup Plans, BackupJobs, and Backups to the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Backups section in the dashboard navigation with Plans,
BackupJobs, and Backups.
* Added detailed views for Plan, BackupJob, and Backup showing metadata,
references, status, timings, and artifacts.
* Enhanced list views with custom columns (application refs,
phase/status, created/taken timestamps).
* **Documentation**
* Added a Dashboard integration guide documenting how to add resources,
list/detail wiring, and verification steps.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Cozystack-api and lineage-webhook tolerate all taints, so they are able
to run on masters no matter what.
Needed for unschedulable control-plane setup, quorum nodes, etc.
### Release note
```release-note
Cozystack-api and lineage-webhook tolerate all taints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated pod scheduling tolerations to improve deployment flexibility
and compatibility across diverse cluster configurations.
* Enhanced workload distribution by making pods more resilient to
cluster taints, enabling better resource utilization in various
environments.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add DRBD options to automatically convert diskless nodes to diskful
when they remain in Primary state for more than 30 minutes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add linstor-scheduler-extender for optimal pod placement on nodes
with LINSTOR storage.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
Fix bug in toggle-disk retry logic that left orphaned DRBD devices in
kernel.
## Problem
When toggle-disk retry was triggered (e.g., user retries after a failed
operation), the code called `removeLayerData()` to clean up and recreate
the layer stack. However, `removeLayerData()` only removes data from the
controller's database — it does NOT call `drbdadm down` on the
satellite.
This caused DRBD devices to remain in the kernel (visible in `drbdsetup`
but not managed by LINSTOR), occupying ports and blocking subsequent
operations.
## Solution
Changed retry logic to simply repeat the operation with existing layer
data intact. The satellite handles this idempotently without creating
orphaned resources.
## Upstream
- https://github.com/LINBIT/linstor-server/pull/475 (updated)
```release-note
[linstor] Fix orphaned DRBD devices during toggle-disk retry
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added cancel and retry capabilities for disk addition operations
* Added cancel and retry capabilities for disk removal operations
* Improved cleanup handling for diskless resources with orphaned storage
layers
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The previous retry logic in toggle-disk removed layer data from controller DB
and recreated it. However, removeLayerData() only deletes from the database
without calling drbdadm down on the satellite, leaving orphaned DRBD devices
in the kernel that occupy ports and block new operations.
This fix changes retry to simply repeat the operation with existing layer data,
allowing the satellite to handle it idempotently.
Upstream: https://github.com/LINBIT/linstor-server/pull/475
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Moved README describing how to add new resources to the Cozystack
dashboard to a separate commit to directly reference relevant code
changes from the document.
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Add representation for backup Plans in the dashboard and a readme
describing, how to add further resources to the dashboard.
### Release note
```release-note
[backups,dashboard] Add backup Plans to the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Update kube-ovn from v1.14.11 to v1.14.25.
Sync chart templates with upstream changes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Add automatic `valuesFrom` injection with `cozystack-values` secret into
HelmReleases created by Package reconciler. This enables charts to
access cluster and namespace configuration via `.Values._cluster` and
`.Values._namespace`.
**Changes:**
- **Package Reconciler:** Inject `valuesFrom` referencing
`cozystack-values` secret, add `Owns(&HelmRelease{})` to reconcile on
changes
- **SecretReplicatorReconciler (new):** Replicate secret to namespaces
with active HelmReleases, cleanup orphaned secrets
- **PackageSource Reconciler:** Replace Watches with
`Owns(&ArtifactGenerator{})`
- **Platform:** Add `operator.cozystack.io/skip-cozystack-values`
annotation to platform PackageSource
### Release note
```release-note
[operator] Add valuesFrom injection to HelmReleases created by Package reconciler with automatic secret replication
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Configurable secret replication that copies designated values secrets
into namespaces selected by labels.
* Annotation to opt out of injecting cozystack-values into specific
PackageSource resources.
* New runtime flags to customize secret name, namespace, and namespace
selector.
* **Refactor**
* Simplified ownership/watcher setup to improve event handling and
reconciliation efficiency.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Build piraeus-server (linstor-server) from source with custom patches:
- **adjust-on-resfile-change.diff** — Use actual device path in res file
during toggle-disk; fix LUKS data offset
- Upstream: [#473](https://github.com/LINBIT/linstor-server/pull/473),
[#472](https://github.com/LINBIT/linstor-server/pull/472)
- **allow-toggle-disk-retry.diff** — Allow retry and cancellation of
failed toggle-disk operations
- Upstream: [#475](https://github.com/LINBIT/linstor-server/pull/475)
- **force-metadata-check-on-disk-add.diff** — Create metadata during
toggle-disk from diskless to diskful
- Upstream: [#474](https://github.com/LINBIT/linstor-server/pull/474)
- **skip-adjust-when-device-inaccessible.diff** — Skip DRBD adjust/res
file regeneration when child layer device is inaccessible
- Upstream: [#471](https://github.com/LINBIT/linstor-server/pull/471)
Also updates plunger-satellite script and values.yaml for the new build.
### Release note
```release-note
[linstor] Build linstor-server with custom patches for improved disk handling
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automatic DRBD stall detection and recovery, improving storage
resync resilience without manual intervention.
* Introduced configurable container image references via Helm values for
streamlined deployment.
* **Bug Fixes**
* Enhanced disk toggle operations with retry and cancellation support
for better error handling.
* Improved metadata creation during disk state transitions.
* Added device accessibility checks to prevent errors when underlying
storage devices are unavailable.
* Fixed LUKS encryption header sizing for consistent deployment across
nodes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add patches for piraeus-server:
- adjust-on-resfile-change: handle resource file changes
- allow-toggle-disk-retry: enable disk operation retries
- force-metadata-check-on-disk-add: verify metadata on disk addition
Update plunger-satellite script and values.yaml for new build.
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds a `k=kubectl` alias and bash completion for kubectl to the
e2e-testing sandbox container to maintainers have an easier time
exec'ing into the CI container when something needs to be debugged.
### Release note
```release-note
[testing] Add k=kubectl alias and enable kubectl completion in the CI
container.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced the e2e sandbox image to enable shell bash-completion and
kubectl command completion.
* Added an alias (k) and completion wiring for kubectl to improve
interactive command use.
* These changes augment the test environment shell during image build to
provide a smoother developer/testing experience.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add automatic injection of cozystack-values secret reference into
HelmReleases created by Package reconciler. This enables charts to
access cluster and namespace configuration via .Values._cluster and
.Values._namespace.
Add annotation operator.cozystack.io/skip-cozystack-values to disable
injection for specific PackageSources (used for platform PackageSource
to avoid circular dependency).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Integrate fix
- https://github.com/kubevirt/cloud-provider-kubevirt/pull/379
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Fix endpoints for cilium-gateway
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of incomplete endpoint data by introducing fallback
detection mechanisms.
* Enhanced service discovery to gather endpoints from all available
resources when standard detection fails.
* Updated logging to provide better visibility into fallback operations
and current resource status.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Replaces Helm lookup functions with FluxCD valuesFrom mechanism for
passing configuration to HelmReleases. This provides cleaner config
propagation and eliminates the need for force reconcile controllers.
### Changes:
**Platform/Tenant charts:**
- Add Secret `cozystack-values` creation in platform chart (for
tenant-root and system namespaces)
- Add Secret `cozystack-values` creation in tenant chart (for child
namespaces)
**cozystack-api:**
- Add `valuesFrom` references to HelmRelease when creating applications
- Filter keys starting with `_` when returning Application specs
- Validate that user values don't contain `_` prefixed keys
**cozystack-controller:**
- Add validation that HelmRelease contains correct valuesFrom
configuration
- Remove `CozystackConfigReconciler` (no longer needed)
- Remove `TenantHelmReconciler` (no longer needed)
**Helm charts (40+ files):**
- Add helper templates in cozy-lib for `_cluster`/`_namespace` access
- Replace ConfigMap lookups with `.Values._cluster.*`
- Replace Namespace annotation lookups with `.Values._namespace.*`
### Architecture:
```
Secret cozystack-values (in each namespace)
├── _cluster: YAML with data from ConfigMaps (cozystack, cozystack-branding, cozystack-scheduling)
└── _namespace: YAML with namespace service references (etcd, host, ingress, monitoring, seaweedfs)
HelmRelease
└── spec.valuesFrom:
├── Secret/cozystack-values → _namespace → .Values._namespace
└── Secret/cozystack-values → _cluster → .Values._cluster
```
### Release note
```release-note
[platform] Replace Helm lookup functions with FluxCD valuesFrom mechanism for configuration propagation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Helm releases and namespaces now source centralized cluster/namespace
configuration via a new secret (cozystack-values), and many templates
read values from chart-provided _cluster/_namespace entries.
* **Bug Fixes**
* API now rejects application specs containing reserved keys prefixed
with "_" to prevent invalid configurations.
* **Refactor**
* Two background reconciler controllers were removed from startup,
simplifying controller initialization.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes an issue where backport releases incorrectly became marked as
"Latest" despite passing `make_latest: 'false'` to the GitHub API.
**Root cause:** The `getLatestRelease()` API returns the release with
the "Latest" flag, not the highest semver version. Combined with race
conditions during parallel release publishing and GitHub API potentially
ignoring `make_latest: 'false'`, backport releases were incorrectly
marked as latest.
**Solution:**
- Replace `getLatestRelease()` with semver-based max version detection
across all published releases
- After publishing a backport release, explicitly restore the latest
flag on the highest semver release
- Remove unused dead code from `tags.yaml` workflow
### Release note
```release-note
[ci] Fix latest release detection to use semver comparison instead of GitHub's "Latest" flag
```
## What this PR does
Reverts PR #1770 which incorrectly changed status/phase checks from
lowercase to uppercase.
The actual KubeVirt metrics use **lowercase**:
- `kubevirt_vm_info` uses `status="running"` (not `"Running"`)
- `kubevirt_vmi_info` uses `phase="running"` (not `"Running"`)
### Verification
Queried virt-controller metrics directly in the instories cluster:
```
kubevirt_vm_info{...,status="running",status_group="running",...} 1
kubevirt_vmi_info{...,phase="running",...} 1
```
Note: `kubectl get vm` shows `STATUS: Running` with capital R, but this
is display formatting — the actual metric labels use lowercase.
### Release note
```release-note
[kubevirt-operator] Fix VM alert rules to use correct lowercase status values
```
Replace Helm lookup functions with FluxCD valuesFrom mechanism for
reading cluster and namespace configuration.
Changes:
- Create Secret cozystack-values in each namespace with values.yaml key
containing _cluster and _namespace configuration as nested YAML
- Configure HelmReleases to read from this Secret via valuesFrom
(valuesKey defaults to values.yaml, so it can be omitted)
- Update cozy-lib helpers to access config via .Values._cluster
- Add default values for required _cluster keys to ensure all fields exist
- Update Go code (cozystack-api and helm reconciler) to use new format
This eliminates the need for Helm lookup functions while maintaining
the same configuration interface for charts.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Revert PR #1770 which incorrectly changed status check from lowercase
to uppercase. The actual metrics use lowercase:
- kubevirt_vm_info uses status="running" (not "Running")
- kubevirt_vmi_info uses phase="running" (not "Running")
Verified by querying virt-controller metrics in instories cluster.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds a `k=kubectl` alias and bash completion for kubectl to the
e2e-testing sandbox container to maintainers have an easier time
exec'ing into the CI container when something needs to be debugged.
### Release note
```release-note
[testing] Add k=kubectl alias and enable kubectl completion in the CI
container.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Implement a controller for BackupJobs referencing a Velero strategy. Creates a Backup.velero.io according to the template in the `Velero.strategy.backups.cozystack.io`.
### Release note
```release-note
[backups] Implement a backup strategy controller for Velero strategies.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Co-authored-by: Andrey Kolkov <androndo@gmail.com>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch narrows the scope of the Velero backup strategy controller to
simply template Velero Backups according to the application being backed
up and the template in the strategy. Creating storage locations is now
out of scope.
### Release note
```release-note
[backups] Implement templating for Velero backups and remove creation of
backup storage locations and volume snapshot locations.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch implements the Reconcile function for BackupJobs with a
Velero strategy ref.
### Release note
```release-note
[backups] Implement the Velero backup strategy controller.
```
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Replace unreliable getLatestRelease() API with semver-based max version
detection. After publishing a backport release, explicitly restore the
latest flag on the highest semver release to handle cases where GitHub
API ignores make_latest: 'false'.
Also remove dead code (unused steps) from tags.yaml workflow.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This PR splits the monolithic `cozystack-resource-definitions` package
into 25 individual resource definition packages (`*-rd`) for better
modularity and independent versioning.
**Changes:**
- Created 25 separate `*-rd` packages (bootbox-rd, bucket-rd,
clickhouse-rd, etcd-rd, ferretdb-rd, foundationdb-rd, http-cache-rd,
info-rd, ingress-rd, kafka-rd, kubernetes-rd, monitoring-rd, mysql-rd,
nats-rd, postgres-rd, rabbitmq-rd, redis-rd, seaweedfs-rd,
tcp-balancer-rd, tenant-rd, virtual-machine-rd, virtualprivatecloud-rd,
vm-disk-rd, vm-instance-rd, vpn-rd)
- Removed `packages/system/cozystack-resource-definitions`
- Updated platform bundles (paas-hosted, paas-full, distro-full) to
reference individual -rd packages
- Updated `hack/update-crd.sh` to use package-specific directories
Each `*-rd` package contains:
- `Chart.yaml` - package metadata
- `values.yaml` - default values
- `Makefile` - build instructions
- `cozyrds/<name>.yaml` - CRD definition
- `templates/cozyrd.yaml` - Helm template
**Benefits:**
- **Modularity**: Each resource definition is now a standalone package
- **Independent versioning**: Resources can be versioned independently
- **Maintainability**: Easier to update individual resources
- **Build efficiency**: Parallel building of resource packages
### Release note
```release-note
[platform] Split cozystack-resource-definitions into 25 separate *-rd packages for better modularity and independent versioning. Each resource definition is now a standalone package.
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **Refactor**
* Split the monolithic resource-definitions into many independent resource-definition packages (e.g., bootbox-rd, bucket-rd, clickhouse-rd, etc.) for modular deployment and finer-grained management
* Updated deployment bundles to reference the new per-resource releases with uniform cozy-system namespace and CRD dependency
* **Chores**
* Added packaging/Helm stubs (Chart.yaml, Makefile, values, templates) for each new resource-definition
* **Bug Fixes**
* Made CRD path resolution dynamic (NAME validated before assignment)
<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Add lb tests for tenant k8s
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Increased readiness and port-forward timeouts to improve stability.
* Added full end-to-end provisioning and validation: automated namespace
and backend deployment, load balancer provisioning, health checks with
retries, reachability validation, and cleanup.
* Provisioning sequence now runs earlier and is duplicated within the
test flow, altering execution order and adding extra validation/cleanup
steps.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Service selector was using app: "${test_name}-backend" but the
Deployment pod template has app: backend. Fixed selector to match
the actual pod labels so endpoints are created correctly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Allows maintainers to trigger test rerun by commenting /retest on a PR.
The workflow finds the latest run for the PR and reruns starting from
the "Prepare environment" job, skipping the build step.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
actions/checkout configures http.extraheader with GITHUB_TOKEN which
takes priority over URL credentials. This caused tag pushes to
authenticate as github-actions instead of cozystack-bot, preventing
the Versioned Tag workflow from being triggered.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Run curl directly from the testing container instead of creating
a separate pod with kubectl run. This avoids PodSecurity policy
violations and simplifies the test execution.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Added resource requests and limits for etcd-defrag container.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[system] Add resource requests and limits to etcd-defrag
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced resource configurations for the etcd defragmentation process
to ensure optimal performance and system stability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add documentation explaining how to enable Hubble for network
observability in Grafana. Include four pre-built Hubble dashboards
(overview, dns-namespace, l7-http-metrics, network-overview) and
register them in the monitoring hub's dashboard list.
Closes#749
Signed-off-by: majiayu000 <1835304752@qq.com>
## What this PR does
The Helm hook that creates a job deleting all applications in a tenant
before deleting the tenant itself now runs this job from the cozy-system
namespace. This prevents conflicts with resource quotas (not enough
resources to run cleanup job) without temporary increases of the quota
or similar vulnerability-introducing hacks.
### Release note
```release-note
[tenant] Run cleanup job in system namespace to avoid conflicts on
resource quotas.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Infrastructure Updates**
* Updated cleanup job namespace targeting to use a fixed system
configuration
* Adjusted cleanup job execution priority level
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes egress policy for nested Kubernetes clusters using `exposeMethod:
Proxied`.
The clusterwide egress policy blocks traffic from tenant pods to ingress
pods in parent namespaces. This breaks:
- cert-manager HTTP-01 self-check
- Any scenario where pods need to access services exposed through parent
ingress
Adds egress rule allowing traffic to ingress pods
(`cozystack.io/service: ingress`) in parent namespaces, following the
same pattern as existing vminsert and etcd rules.
### Release note
```release-note
[tenant] Fixed tenant egress policy to allow traffic to parent ingress pods, enabling cert-manager HTTP-01 challenges and external domain access for nested clusters with exposeMethod: Proxied
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced network policies for multi-tenant environments with improved
traffic routing based on namespace hierarchies, enabling more granular
egress control.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fix case sensitivity in status check for VMNotRunningFor10Minutes alert
rule.
The metric `kubevirt_vm_info` uses `status="Running"` (capital R), but
the alert rule was checking for `status!="running"` (lowercase), causing
false alerts for running VMs.
Fixes https://github.com/cozystack/cozystack/issues/1552
### Release note
```release-note
[kubevirt-operator] Fix VMNotRunningFor10Minutes alert false positives
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed alert rule condition for virtual machine status detection to
ensure proper matching of running VMs.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Add deployment files
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Deployable CozyStack operator with configurable enablement, image, and
platform-source settings
* Operator-managed package source and variant-based platform
installation options
* **Chores**
* Release/packaging targets added to automate image and package
publishing and update manifests
* Configuration schema extended to include operator-related fields and
versioning controls
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR renames cozypkg to cozyhr
https://github.com/cozystack/cozyhr
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[installer,dx] Rename cozypkg to cozyhr
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does

### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Add PackageSources for cozystack-operator
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for 76 new package sources, including operators,
applications, and system components for the CozyStack platform ecosystem
(e.g., backup controller, database operators, monitoring tools,
networking, storage, and virtualization packages).
* Introduced a template aggregation mechanism to manage package source
definitions.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozypkg] Add tool for managing Package and PackageSources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* New cozy CLI with entrypoint and root command.
* Interactive "add" command to install packages from args or files, with
dependency resolution, topological install order, variant selection, and
progress feedback.
* "del" command to remove packages safely, including dependent analysis,
confirmation prompts, and safe delete ordering.
* "list" command to show available or installed packages, with optional
per-component output and concise status lines.
* "dependencies" command to emit Graphviz DOT graphs of
package/component relationships.
* Common flags: kubeconfig, file inputs, --installed and --components
where applicable.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
depends on https://github.com/cozystack/cozystack/pull/1741
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Add Package and PackageSource reconcilers
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Launches CozyStack operator with CLI-configurable runtime
(metrics/health/leader election/HTTP2), optional pre-reconcile steps,
optional automatic Flux install, and platform-source URL support
(OCI/Git).
* Adds PackageSource and Package controllers for artifact generation,
variant resolution, dependency tracking, HelmRelease orchestration,
namespace reconciliation, orphan cleanup, and status propagation.
* Package status now exposes per-dependency readiness.
* **Chores**
* Embeds Flux manifests and adds a local Flux install workflow; updates
manifest generation path and Makefile targets.
* **Other**
* PackageSource CRD shortName simplified to "pks".
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases multus memory limit.
Based on multiple community reports stating that multus tend to consume
a lot of memory during startup after a node reboot.
### Release note
```release-note
Multus memory limit increased.
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Introduce Cozystack-operator core logic
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Operator entrypoint with flag-driven configuration (metrics, probes,
leader election, Flux pre-install, version, platform-source options).
* Optional pre-start Flux installation and automatic provisioning of
platform sources from OCI or Git/HTTP(S)/SSH.
* **Chores**
* Embedded Flux manifests packaged for simplified installation; build
tooling updated to produce new manifest outputs.
* **Behavior**
* Pre-start Flux and platform-source installation failures are treated
as fatal.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The Helm hook that creates a job deleting all applications in a tenant
before deleting the tenant itself now runs this job from the cozy-system
namespace. This prevents conflicts with resource quotas (not enough
resources to run cleanup job) without temporary increases of the quota
or similar vulnerability-introducing hacks.
### Release note
```release-note
[tenant] Run cleanup job in system namespace to avoid conflicts on
resource quotas.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch adds boilerplate code for a Velero strategy for the backups
API. Implementation can be added into the Reconcile function in a
subsequent PR.
###
```release-note
[backups] Add the boilerplate for a new Velero strategy for backups.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Velero as a supported backup strategy, including a
cluster-scoped Velero resource type and CRD for creating Velero-backed
strategies.
* Backup job controller updated to recognize and dispatch Velero-backed
jobs, with status conditions exposed for monitoring progress and health.
* Introduced basic reconciliation flow for Velero strategy to enable
future behavior and observability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fix case sensitivity in status check: "running" → "Running".
The metric uses "Running" (capital R), so the lowercase check was
always triggering alerts even for running VMs.
Fixes: https://github.com/cozystack/cozystack/issues/1552
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
- Update Cilium from v1.17.8 to v1.18.5
- Add `SED_INPLACE` variable to `scripts/common-envs.mk` for macOS
compatibility
- Remove deprecated `enableRuntimeDeviceDetection` option (now default
behavior in 1.18)
Cilium 1.18 requires Linux kernel 5.10+ (compatible with Talos).
References:
- [Cilium 1.18 Upgrade
Guide](https://docs.cilium.io/en/stable/operations/upgrade/)
- [Cilium 1.18 Release
Blog](https://isovalent.com/blog/post/cilium-1-18/)
### Release note
```release-note
[cilium] Update Cilium to v1.18.5
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added configurable Kubernetes service discovery with ConfigMap-based
endpoint source
* Introduced exponential backoff settings for Kubernetes API client
connections
* Added metrics sampling interval configuration for internal agent
metrics
* Implemented identity management mode options for endpoint slices and
DNS proxy pre-allocation
* Enhanced Prometheus scrape timeout configuration across monitoring
components
* **Improvements**
* Upgraded core components to v1.18.5
* Strengthened security defaults by disabling privilege escalation
across pods
* Added startup and liveness probe configurations for improved health
monitoring
* Extended network policy correlation capabilities in Hubble
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-controller] Fix: move crds to definitions
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the Custom Resource Definition generation template to source
files from a new location, ensuring consistent configuration management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Update Cilium from v1.17.8 to v1.18.5.
Breaking changes addressed:
- Remove deprecated enableRuntimeDeviceDetection option from
values-kubeovn.yaml (now default behavior in 1.18)
- Use SED_INPLACE variable for macOS compatibility in Makefile
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add SED_INPLACE variable to common-envs.mk that detects the OS and uses
the appropriate sed in-place syntax: `sed -i ''` on macOS (Darwin) and
`sed -i` on Linux.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds deterministic alphabetical sorting to all registry resource lists:
- TenantNamespace (sorted by name)
- TenantSecret (sorted by namespace/name)
- TenantModule (sorted by namespace/name)
- Application (sorted by namespace/name)
Introduces a new `pkg/registry/sorting` package with generic helper
functions
to avoid code duplication.
Also fixes pre-existing linter errors:
- Unused `helmReleaseGVR` variables
- Non-constant format strings in `klog.Errorf` calls
- Redundant embedded field selectors
### Release note
```release-note
[platform] Registry resource lists are now returned in alphabetical order
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a reusable alphabetical sorting utility and applied consistent
sorting across list endpoints.
* **Bug Fixes**
* Ensured ResourceVersion/UID are sourced correctly for accurate
list/table responses.
* Simplified and improved error logging and table ResourceVersion
handling.
* Made schema defaulting behavior more consistent.
* **Tests**
* Added unit tests validating alphabetical sorting for multiple resource
types.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Nested Kubernetes clusters with exposeMethod: Proxied cannot reach their
own external domains because the clusterwide egress policy blocks traffic
to ingress pods in parent namespaces.
This breaks cert-manager HTTP-01 self-check and any scenario where pods
need to access services exposed through parent ingress.
Add egress rule allowing traffic to ingress pods (cozystack.io/service:
ingress) in parent namespaces, following the same pattern as existing
vminsert and etcd rules.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Simplify ByName with strings.Compare
- Avoid string allocation in ByNamespacedName by comparing fields separately
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add sorting helper package with generic functions for sorting
resources by name (cluster-scoped) or namespace/name (namespace-scoped).
Apply sorting to:
- TenantNamespace (by name)
- TenantSecret (refactored to use helper)
- TenantModule (new)
- Application (new)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
This patch adds boilerplate code for a Velero strategy for the backups
API. Implementation can be added into the Reconcile function in a
subsequent PR.
###
```release-note
[backups] Add the boilerplate for a new Velero strategy for backups.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[agents] Add instructions for working with unresolved code review comments
```
This PR adds a new GitHub workflow that automatically creates patch
releases for maintenance branches.
## What it does
- Runs daily at 2:00 AM CET (1:00 UTC)
- Checks all `release-X.Y` branches
- For each branch, finds the latest published release with tag `vX.Y.Z`
(without suffixes like -rc, -alpha, -beta)
- If new commits exist after the release, creates a new tag `vX.Y.Z+1`
and force-pushes it
- This triggers the existing tag workflow to build and create a release
PR
## Why releases instead of tags
The workflow checks published releases (not just tags) because if a
release PR hasn't been merged yet, the workflow should run again the
next day and move the tag to newer commits if they exist.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Automated patch release workflow configured for release branches,
enabling automatic version tagging when new commits are detected.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds a GitHub Actions workflow that automatically updates
release notes from changelog files when changes are merged to main.
The workflow:
- Triggers on push to main branch
- Processes up to 30 releases from the first page
- For each release, checks if a corresponding changelog file exists in
`docs/changelogs/`
- Updates the release notes with the changelog content if it exists and
differs from the current content
- Skips releases that are already up to date to avoid unnecessary API
calls
This automates the process of keeping release notes synchronized with
changelog files.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Automated workflow added to synchronize GitHub releases with local
changelog files, updating release notes only when content differs.
* **Documentation**
* Changelogs and release notes reorganized and expanded with improved
sectioning and new tooling/documentation entries.
* **New Features**
* Added VM disk SVG icon entry under Features/Improvements.
* **Bug Fixes**
* Pinned CoreDNS image tag for reproducible deployments.
* Fixed a documentation typo that prevented a component from displaying
in the web UI.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Extract Talos-related functionality from `packages/core/installer` into
a separate `packages/core/talos` package.
This refactoring separates concerns:
- The `installer` package now focuses solely on the Cozystack installer
- The `talos` package handles all Talos Linux image building and assets
generation
Changes:
- Created new `packages/core/talos` package with Chart.yaml, Makefile,
and values.yaml
- Moved Talos profiles (initramfs, kernel, iso, installer, metal,
nocloud) to talos package
- Moved matchbox configuration and Dockerfile to talos package
- Moved `hack/gen-profiles.sh` and `hack/gen-versions.sh` scripts to
talos package
- Updated installer Makefile to remove all Talos-related targets
- Updated root Makefile to build talos package separately
- Updated matchbox Dockerfile paths to reference talos package
### Release note
```release-note
[core] Extract Talos package from installer into separate packages/core/talos package
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Reorganized build system to add a dedicated Talos package with its own
image and asset build workflow
* Switched asset generation to the new Talos packaging path and
simplified build dependency chain
* Added Helm chart manifest and streamlined image build/publishing steps
for Talos-related artifacts
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Add changelogs for v.0.39.0 and v.0.39.1
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Topology-aware routing for services
* Automatic pod rollouts on configuration changes
* Windows VM scheduling support
* SLACK_SEVERITY_FILTER for notification filtering
* VMAgent resource for metrics scraping
* **Improvements**
* Enhanced networking and storage capabilities
* Configuration management enhancements
* Updated dependencies
* **Bug Fixes**
* Schema nesting and resizing validation
* Namespace deletion regression
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
See the examples (not part of this PR)
PackageSource:
-
43cef5d50e/packages/core/platform/templates/sources/backup-controller.yaml
Package:
-
43cef5d50e/packages/core/platform/templates/packages/isp-full.yaml
Platform Package:
-
43cef5d50e/packages/core/installer/example/platform.yaml
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Introduce API objects: packages and packagesources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Package and PackageSource custom resources to manage
software packages and their configurations, with support for variants,
components, library management, and dependency handling.
* **Bug Fixes**
* Made the Chart field optional in CozystackResourceDefinitionRelease.
* **Chores**
* Updated code generation tooling and CRD artifact management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This change is extracted from
- https://github.com/cozystack/cozystack/pull/1641
and reworked to work standalone
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack] Update go modules
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Go toolchain to 1.25.0 and upgraded core Kubernetes libraries,
OpenTelemetry, Prometheus, gRPC/protobuf and many indirect dependencies.
Bumped builder base images to golang:1.25-alpine across multiple
components.
* **Refactor**
* Removed legacy component versioning/emulation and simplified server
startup and configuration paths.
* **Tests**
* Removed tests related to the legacy versioning/emulation behavior.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When VMs are created without a public IP, no service is created for them
and they have no in-cluster DNS name. This PR fixes this and resolves
#1731.
### Release note
```release-note
[vm] Always expose VMs with at least a ClusterIP service.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Service templates now conditionally enable external-specific
annotations and settings when external mode is enabled.
* External LoadBalancer deployments support richer port configuration
(including whole-IP fallback), while internal services retain a single
default port (65535).
* External traffic policy and node port allocation are applied only in
external mode to preserve internal behavior.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This change is extracted from
- https://github.com/cozystack/cozystack/pull/1641
and reworked to work standalone
## What this PR does
This PR extracts changes from the
https://github.com/cozystack/cozystack/pull/1641. It adds application
metadata labels to HelmReleases and updates the filtering mechanism to
use labels instead of chart/sourceRef matching.
Changes:
- Add three application metadata labels
(`apps.cozystack.io/application.kind`,
`apps.cozystack.io/application.group`,
`apps.cozystack.io/application.name`) when creating/updating HelmRelease
via Cozystack-API
- Replace `shouldIncludeHelmRelease` filtering with label-based
filtering in Get, List, and Update methods
- Always add kind and group label requirements in List for precise
filtering
- Update CozystackResourceDefinitionController to watch only
HelmReleases with `cozystack.io/ui=true` label
- Update LineageControllerWebhook to extract metadata directly from
HelmRelease labels instead of using chart mapping configuration
- Add functionality to update HelmRelease chart from
CozystackResourceDefinition using label selectors
### Release note
```release-note
[registry] Add application labels and update filtering mechanism to use label-based filtering instead of chart/sourceRef matching
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added application metadata labels (kind, group, name) and exported
label keys for HelmRelease identification.
* New reconciler keeps HelmRelease charts in sync with application CRDs.
* **Refactor**
* Mapping, listing and selection moved to label-driven logic;
reconciliation responsibilities split so core reconciler focuses on
restart/debounce while a separate reconciler updates HelmReleases.
* **Chores**
* Migration script to backfill application labels on existing
HelmReleases.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Enable automatic restart of cilium and cilium-operator pods when
cilium-config ConfigMap is updated.
This change adds:
- `rollOutCiliumPods: true` - enables automatic rollout of cilium-agent
pods
- `operator.rollOutPods: true` - enables automatic rollout of
cilium-operator pods
When the ConfigMap is updated, pods will automatically restart due to
the `cilium.io/cilium-configmap-checksum` annotation that contains the
SHA256 hash of the configmap.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Cilium configuration settings to enable pod rollout behavior
for Cilium and operator pods by default.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds nodeAffinity configuration to virtual-machine and
vm-instance charts to support dedicated nodes for Windows VMs.
## Changes
- Added logic to check `cozystack-scheduling` ConfigMap in `cozy-system`
namespace
- If `dedicatedNodesForWindowsVMs` is enabled, adds appropriate
nodeAffinity:
- **Windows VMs**: Strong affinity (required) to nodes with label
`scheduling.cozystack.io/vm-windows=true`
- **Non-Windows VMs**: Soft affinity (preferred) to nodes without the
Windows label
## Implementation
- Windows detection based on `instanceProfile` value starting with
"windows"
- Strong affinity uses `requiredDuringSchedulingIgnoredDuringExecution`
- Soft affinity uses `preferredDuringSchedulingIgnoredDuringExecution`
with weight 100
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable node-affinity for VM scheduling: when the cluster
scheduling flag is enabled, Windows VMs are placed on dedicated Windows
nodes (required rule), while non-Windows VMs are preferred to avoid
those nodes (soft preference).
* Change is gated by the cluster scheduling configuration and only
affects placement rules; no other VM specs were altered.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When VMs are created without a public IP, no service is created for them
and they have no in-cluster DNS name. This PR fixes this and resolves
#1731.
### Release note
```release-note
[vm] Always expose VMs with at least a ClusterIP service.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
- Add three application metadata labels to HelmRelease:
- apps.cozystack.io/application.kind
- apps.cozystack.io/application.group
- apps.cozystack.io/application.name
- Replace shouldIncludeHelmRelease filtering with label-based filtering
in Get, List, and Update methods
- Always add kind and group label requirements in List for precise filtering
- Update CozystackResourceDefinitionController to watch only HelmReleases
with cozystack.io/ui=true label
- Update LineageControllerWebhook to extract metadata directly from
HelmRelease labels instead of using mapping configuration
- Add functionality to update HelmRelease chart from CozystackResourceDefinition
using label selectors
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR adds Cloupard as new adopter to the list.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Add Cloupard to ADOPTERS.md
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated the list of project adopters with recent entries.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR includes new seaweedfs with improved perfomance for S3 daemon
and fixes issue https://github.com/seaweedfs/seaweedfs/issues/7757
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update SeaweedFS v4.02
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added all-in-one deployment mode with configurable replicas and update
strategy
* Expanded storage configuration supporting PersistentVolumeClaims with
customizable access modes and size
* Introduced configurable certificate duration and renewal periods
* Enhanced monitoring configuration with gateway host/port and
additional labels
* **Bug Fixes**
* Fixed probe endpoint scheme references across components
* **Chores**
* Updated to SeaweedFS 4.02
* Updated default ingress class configuration
* S3 disabled by default
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.
```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable severity filtering for Telegram alerts.
* Extended Slack severity filtering to accept lists of severities.
* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.
* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the Alerta deployment to enable
filtering of alert severities for Slack notifications based on the disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics within tenant namespaces, improving
monitoring granularity and control.
```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
This PR introduces a bare-bones Job backup strategy API type and stubs
out all the boilerplate for a new controller that will handle this
strategy, as well as any others in the `strategy.backups.cozystack.io`
API group.
### Release note
```release-note
[backups] Create stubs and minimal implmentations for controllers for
the strategy.backups.cozystack.io API group.
```
## What this PR does
This adds custom indexable fields on core backup resources to enable
filtering by backed-up application. This will later be useful for
displaying backup resources in the dashboard: it will be possible to
filter them with a field selector to display only those backup resources
that are relevant to a given application.
### Release note
```release-note
[backups] Enable filtering backup resources by backed up application for
per-app views of backups.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced new Backup, BackupJob, Plan, and RestoreJob API resources
for enhanced backup management.
* Added selectable fields to backup resources, enabling efficient
filtering and querying by application reference.
* Added status subresource with conditions to Plan resources for
improved status tracking and observability.
* **Chores**
* Updated repository metadata configuration.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch adds the boilerplate for the `strategy.backups.cozystack.io`
API group that will contain reference backup/restore strategy
implementations that will live in Cozystack core.
### Release note
```release-note
[backups] Scaffold the `strategy.backups.cozystack.io` API group to
provide a well-defined point for adding reference strategy
implementations.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced v1alpha1 backup strategy API group registration, enabling
support for backup strategy resources within the system.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR introduces a bare-bones Job backup strategy API type and stubs
out all the boilerplate for a new controller that will handle this
strategy, as well as any others in the `strategy.backups.cozystack.io`
API group.
### Release note
```release-note
[backups] Create stubs and minimal implmentations for controllers for
the strategy.backups.cozystack.io API group.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This adds custom indexable fields on core backup resources to enable
filtering by backed-up application. This will later be useful for
displaying backup resources in the dashboard: it will be possible to
filter them with a field selector to display only those backup resources
that are relevant to a given application.
### Release note
```release-note
[backups] Enable filtering backup resources by backed up application for
per-app views of backups.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch adds the boilerplate for the `strategy.backups.cozystack.io`
API group that will contain reference backup/restore strategy
implementations that will live in Cozystack core.
### Release note
```release-note
[backups] Scaffold the `strategy.backups.cozystack.io` API group to
provide a well-defined point for adding reference strategy
implementations.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This change is extracted from
- https://github.com/cozystack/cozystack/pull/1641
and reworked to work standalone
requires:
- https://github.com/cozystack/cozystack/pull/1705
## What this PR does
Adds a new `flux-aio` module and migration script to upgrade FluxCD to
version 22. This introduces a new modular approach to FluxCD
installation using the flux-aio OCI module.
Changes:
- Created new `flux-aio` package with Chart.yaml, Makefile, and CUE
configuration
- Added flux-aio module configuration using OCI module from
`ghcr.io/stefanprodan/modules/flux-aio`
- Generated large fluxcd.yaml template (11956+ lines) for FluxCD
resources
- Added migration script (migrations/21) to handle upgrade from version
21 to 22
- Updated installer to include flux-aio module
- Added script `issue-flux-certificates.sh` for managing TLS
certificates for cozystack-assets
- Updated platform templates to support flux-aio module
- Updated cozystack-assets service references
### Release note
```release-note
[fluxcd] Add flux-aio module and migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added TLS certificate support for Helm package repositories with
automatic certificate provisioning.
* **Chores**
* Refactored FluxCD integration using Helm chart-based deployment.
* Updated system to version 22 with automatic migration support.
* Enhanced security dependencies (OpenSSL).
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This change is extracted from
- https://github.com/cozystack/cozystack/pull/1641
and reworked to work standalone
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[fluxcd] Enable source-watcher
```
## What this PR does
Separates the assets server from the main cozystack installer into a
dedicated StatefulSet deployment. This improves separation of concerns
and allows the assets server to run independently from the installer.
Changes:
- Created new `cozystack-assets` StatefulSet in the platform package
- Added dedicated Dockerfile for assets server image
(`packages/core/platform/images/cozystack-assets/Dockerfile`)
- Removed assets server container and Service from installer deployment
- Updated HelmRepository URLs to point to new `cozystack-assets` service
- Updated dashboard URLs in monitoring package to use new service
- Added image build target to platform Makefile
- Configured assets server with hostNetwork and proper RBAC permissions
### Release note
```release-note
[platform] Separate assets server into dedicated StatefulSet deployment
```
## What this PR does
This PR refactors the apiserver REST handlers to use typed objects
(`appsv1alpha1.Application`) instead of `unstructured.Unstructured`,
eliminating
the need for runtime conversions and simplifying the codebase.
Additionally, it fixes an issue where `UnstructuredList` objects were
using
the first registered kind from `typeToGVK` instead of the kind from the
object's field when multiple kinds are registered with the same Go type.
This is a more comprehensive fix for the problem addressed in
https://github.com/cozystack/cozystack/pull/1630, which was reverted in
https://github.com/cozystack/cozystack/pull/1677.
The fix includes the upstream fix from kubernetes/kubernetes#135537,
which enables short-circuit path for `UnstructuredList` similar to
regular
`Unstructured` objects, using GVK from the object field instead of
`typeToGVK`.
### Changes
- Refactored `rest.go` handlers to use typed `Application` objects
- Removed `unstructured.Unstructured` conversions
- Fixed `UnstructuredList` GVK handling
- Updated dependencies in `go.mod`/`go.sum`
- Added e2e test for OpenAPI validation
- Updated Dockerfile
### Release note
```release-note
[apps] Refactor apiserver to use typed objects and fix UnstructuredList GVK handling
This change refactors the apiserver REST handlers to use typed Application objects instead of unstructured.Unstructured, eliminating runtime conversions and simplifying the codebase. Additionally, it fixes an issue where UnstructuredList objects were incorrectly using the first registered kind from typeToGVK instead of the kind from the object's field when multiple kinds are registered with the same Go type. This fix includes the upstream fix from kubernetes/kubernetes#135537.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a module replacement in dependency configuration to ensure
reproducible builds.
* **Refactor**
* REST internals now use strongly-typed Application and TenantModule
resources and return typed API objects with consistent metadata.
* **Tests**
* Strengthened end-to-end checks for resource kinds and added a
create/delete namespace scenario.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
A malformed glob ("*" instead of "*.yaml") captured the .gitattributes
file and sent it to the templating engine. Using `split` instead of
`splitList` on a string returned a map instead of a list, so templating
broke on `mustLast`. This patch corrects the errors.
### Release note
```release-note
[backups] Fix template-breaking errors in the backup-controller Helm
chart.
```
## What this PR does
A malformed glob ("*" instead of "*.yaml") captured the .gitattributes
file and sent it to the templating engine. Using `split` instead of
`splitList` on a string returned a map instead of a list, so templating
broke on `mustLast`. This patch corrects the errors.
### Release note
```release-note
[backups] Fix template-breaking errors in the backup-controller Helm
chart.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
PVC resizing now only occurs when storage is being increased, preventing
unintended storage reduction operations
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[virtual-machine] Improve check for resizing job
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* PVC resizing now only triggers when the requested storage increases,
preventing unintended shrink attempts.
* **Enhancements**
* More robust storage-size comparison and validation to accurately
detect growth.
* Resize operations now run only when needed, reducing unnecessary jobs.
* **New Features**
* Added conditional pre-install/pre-upgrade hook workflow to perform
controlled PVC resize jobs.
* **Chores**
* Minor template formatting cleanup.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes the logic for generating CustomFormsOverride schema to properly
nest properties under `spec.properties` instead of directly under
`properties`.
Changes:
- Updated `buildMultilineStringSchema` to check for `spec` property in
OpenAPI schema
- Process `spec.properties` instead of root `properties`
- Create schema structure as `schema.properties.spec.properties.*`
- Updated tests to reflect the new nested structure
### Release note
```release-note
[dashboard] Fix CustomFormsOverride schema generation to nest properties under spec.properties
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Updated custom form schema generation to organize form fields within a
nested structure for improved organization.
* **Tests**
* Expanded test coverage to validate the new schema organization and
field type handling across nested levels.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This release updates LINSTOR CSI to fix issues with the new fsck
behaviour.
```
MountVolume.SetUp failed for volume "pvc-37245cc3-bf28-4da4-a127-e5c9c03cc088" : rpc error: code = Internal desc = NodePublishVolume failed for pvc-37245cc3-bf28-4da4-a127-e5c9c03cc088: failed to run fsck on device '/dev/zvol/data/pvc-37245cc3-bf28-4da4-a127-e5c9c03cc088_00000': failed to run fsck: output: "fsck from util-linux 2.41
/dev/zd832 is mounted.
e2fsck: Cannot continue, aborting.
```
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[linstor] Update piraeus-operator v2.10.2
```
This PR improves the backport workflow by:
- Adding `merge_commits: skip` to skip merge commits during backport
- Adding `conflict_resolution: draft_commit_conflicts` to create draft
PRs when conflicts occur instead of failing
- Removing the 'Report if backport failed' step as it's no longer needed
with the new conflict resolution strategy
These changes ensure that backports handle merge commits and conflicts
more gracefully.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced the backport workflow to support simultaneous backporting to
current and previous release branches based on PR labels
* Added validation to ensure target branches exist before attempting
backports
* Improved workflow reliability by separating preparation and execution
stages
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The `distro-full` bundle was missing critical components that exist in
paas-full, causing multiple pod failures during installation. This PR
adds the missing packages and fixes dependency issues.
When installing cozystack with bundle-name: "distro-full", several pods
failed to start:
1. CozystackResourceDefinition CRD missing
- cozystack-controller pod: CrashLoopBackOff
- lineage-controller-webhook pods: CrashLoopBackOff
- Error: no matches for kind "CozystackResourceDefinition" in version
"cozystack.io/v1alpha1"
a861814c24/packages/system/cozystack-resource-definition-crd/definition/cozystack.io_cozystackresourcedefinitions.yaml (L1-L14)
2. selfsigned-cluster-issuer ClusterIssuer missing
- snapshot-validation-webhook pods: ContainerCreating (waiting for TLS
secret)
- snapshot-controller HelmRelease: Failed to install (timeout)
- Error: clusterissuer.cert-manager.io "selfsigned-cluster-issuer" not
found
a861814c24/packages/system/cert-manager-issuers/templates/cluster-issuers.yaml (L52-L57)a861814c24/packages/system/snapshot-controller/template/clusterissuer.yaml (L1-L8)
3. Cascading failures
- linstor HelmRelease: Blocked (depends on snapshot-controller)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added two new resource-definition components to the platform
distribution for enhanced configuration management.
* **Improvements**
* Made certificate issuer components required during deployment (no
longer optional).
* Adjusted snapshot controller startup order to wait for certificate
issuers, improving startup reliability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch adds compilation and docker build steps for the backup
controller as well as adding a Helm chart to deploy it as part of the
PaaS bundles.
### Release note
```release-note
[backups] Build and deploy backup controller
```
## What this PR does
This patch adds compilation and docker build steps for the backup
controller as well as adding a Helm chart to deploy it as part of the
PaaS bundles.
### Release note
```release-note
[backups] Build and deploy backup controller
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch creates a new backups.cozystack.io API group which includes
backup-related resources owned exclusively by Cozystack core. A
cronjob-like controller is implemented to create backup jobs that will
be handled by appropriate third-party or external controllers.
### Release note
```release-note
[backups] Implement the core backup API and an accompanying controller.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Adds a full backups system: create scheduled Plans (cron), run
BackupJobs with lifecycle phases (Pending→Running→Succeeded/Failed), and
produce Backup artifacts with metadata.
* Adds RestoreJobs to restore from stored Backups and track restore
progress/status.
* Exposes new API resources (Plan, BackupJob, Backup, RestoreJob) so
backups and restores can be declared and observed via the platform.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Previously, all system charts have had their versions bumped with every
update of Cozystack, even if the underlying images and templates did not
change. Additionally, some components would have their docker images
rebuilt and get updated digests because of a changed docker context,
even though no relevant code had been changed. This would cause an
upgrade of several HelmReleases on every upgrade of Cozystack, even when
in practice nothing had changed. This is especially noticeable for
charts, such as kubeovn, which is a dependency for almost all other
system HelmReleases. By moving kubeovn out of the monorepo and
decoupling its release cycle from the release cycle of Cozystack, this
issue can be resolved and is another step towards decomposing Cozystack.
### Release note
```release-note
[kubeovn] Decouple the release cycle of kubeovn from Cozystack's release
cycle, preventing unnecessary redeployments of all Helm releases, when
upgrading Cozystack.
```
## What this PR does
More granular k8s app extensions dependencies.
As a part of the change, packages were updated:
- monitoring-agents: metrics-server moved to a separate package
- monitoring-agents: prometheus-operator-crds moved to a separate
package
Main Cozystack bundles were updated to reflect packages changes.
### Release note
```release-note
Granular k8s extensions dependencies.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a standalone metrics-server release and a
prometheus-operator-crds release.
* **Improvements**
* Expanded Prometheus-related CRDs with new configuration options and
status subresources.
* Adjusted monitoring orchestration so components (cilium, metrics,
CRDs) install in a clearer order.
* **Refactor**
* Split metrics-server out of the monitoring-agents bundle and moved
Prometheus CRDs into a dedicated package.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
https://github.com/fluxcd/flux2/discussions/5572
There are some upgrade instructions for Flux 2.7, but I don't think that
they apply to Cozystack.
They might be relevant for some users though, depending on how they are
using Flux in their clusters - Cozystack only uses HelmReleases AFAIK
(and now gitrepository and helmrelease for external apps) so there is no
problem with the Flux 2.7 upgrade, which brings Image APIs to GA (v1)
That change has no impact on Cozystack because we don't depend on the
Flux Image APIs yet.
I haven't tested the upgrade to Flux 2.7, but I will include the bump in
the FluxInstance definition since Flux 2.7 has been out for a while
already, has had several patch releases, and is probably stable for the
next CozyStack release now!
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added distribution variant option for FluxInstance (upstream-alpine,
enterprise-alpine, enterprise-distroless)
* Added status fields to improve tracking of force and reconcile
annotation handling across resources
* **Chores**
* Bumped Flux Operator and Flux Instance charts to 0.33.0 (appVersion
v0.33.0)
* Upgraded Flux distribution to 2.7.x
* Added/cleaned up network policy templates and README badges
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch introduces vmagent support inside tenant-level monitoring
stacks, ensuring that all tenant workloads have consistent and reliable
metrics scraping. By deploying a vmagent per tenant and wiring it to the
configured VictoriaMetrics short- and long-term storage, metrics are now
properly collected, labeled, and remote-written. This resolves gaps
where tenant environments previously missed metrics due to the absence
of dedicated agents.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
```release-note
[monitoring,platform] Add vmagent integration into tenant monitoring
stacks to ensure complete and correct metrics collection for all tenants.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added VictoriaMetrics Agent (Vmagent) support in monitoring:
configurable external labels and remote-write endpoints for metric
collection.
* **Documentation**
* Added README documentation for the new Vmagent configuration options
and example defaults.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds a version management system for PostgreSQL that
automatically fetches and updates supported versions from the
CloudNativePG PostgreSQL container registry.
## Changes
- Added `hack/update-versions.sh` script to fetch and update PostgreSQL
versions
- Added `files/versions.yaml` to store version mappings
- Added `templates/_versions.tpl` template for version handling
- Updated `values.yaml` with version enum and parameter
- Updated `templates/db.yaml` and `templates/init-job.yaml` to use
version management
- Updated `Makefile` to include version update workflow
## Features
- Automatically fetches supported major versions from GitHub
- Queries container registry for latest minor versions
- Updates version enum in values.yaml
- Maintains version mappings in versions.yaml
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added PostgreSQL version selection (v13–v18) with default v18; images
now follow the selected version via a resolved version map.
* **Chores**
* Added an automated update workflow to refresh supported PostgreSQL
versions and regenerate mappings.
* Packaging ignore updated to exclude hack tooling.
* **Schema**
* Exposed a version property in chart and resource schemas for
validation and ordering.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds a version management system for Kubernetes that
automatically fetches and updates supported versions based on kamaji.
## Changes
- Added `hack/update-versions.sh` script to fetch and update Kubernetes
versions
- Updated `files/versions.yaml` with version mappings
- Updated `values.yaml` with version enum and parameter
- Updated `Makefile` to include version update workflow
- Added `hack` directory to `.helmignore`
## Features
- Automatically fetches Kubernetes version from kamaji repository
- Gets latest patch version for the supported minor version
- Queries container registry for available versions
- Updates version enum in values.yaml using reliable awk-based approach
- Updates KUBERNETES_VERSION in Makefile
- Maintains version mappings in versions.yaml
## Workflow
- Reads kamaji version from Dockerfile
- Fetches Kubernetes version from kamaji's kubeadm_version.go
- Finds latest patch version for that minor version
- Includes supported version + 5 previous minor versions
- Updates all configuration files automatically
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Kubernetes version parameter now uses an enum constraint to limit
selection to supported versions (v1.33, v1.32, v1.31, v1.30, v1.29,
v1.28).
* **Documentation**
* Simplified Kubernetes version field description for clarity.
* **Chores**
* Updated Kubernetes patch version mappings (v1.32→v1.32.10,
v1.31→v1.31.14).
* Relaxed add-on schema validation requirements.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[redis] Add version management system with automated version updates
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[mariadb] Add version management system with automated version updates
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable MariaDB version selection for deployments.
Supported versions: v11.8, v11.4, v10.11, and v10.6 (default: v11.8).
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch creates a new backups.cozystack.io API group which includes
backup-related resources owned exclusively by Cozystack core. A
cronjob-like controller is implemented to create backup jobs that will
be handled by appropriate third-party or external controllers.
### Release note
```release-note
[backups] Implement the core backup API and an accompanying controller.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch introduces vmagent support inside tenant-level monitoring
stacks, ensuring that all tenant workloads have consistent and reliable
metrics scraping. By deploying a vmagent per tenant and wiring it to the
configured VictoriaMetrics short- and long-term storage, metrics are now
properly collected, labeled, and remote-written. This resolves gaps
where tenant environments previously missed metrics due to the absence
of dedicated agents.
```release-note
[monitoring,platform] Add vmagent integration into tenant monitoring
stacks to ensure complete and correct metrics collection for all tenants.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Buildx is worried about Dockerfile syntax, this pr fixes it.
```
- FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 1)
- FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 16)
```
### Release note
```release-note
Buildx warnings addressed.
```
## What this PR does
Updates coredns app labels to match Talos coredns labels
### Release note
```release-note
Coredns app labels updated to match Talos coredns labels.
```
## What this PR does
Renames coredns metrics service
### Release note
```release-note
Renamed coredns metrics service not to interfere with coredns service used for name resolution in tenant k8s clusters.
```
This commit refactors the apiserver REST handlers to use typed objects
(appsv1alpha1.Application) instead of unstructured.Unstructured, eliminating
the need for runtime conversions and simplifying the codebase.
Additionally, it fixes an issue where UnstructuredList objects were using
the first registered kind from typeToGVK instead of the kind from the
object's field when multiple kinds are registered with the same Go type.
This is a more comprehensive fix for the problem addressed in
https://github.com/cozystack/cozystack/pull/1630, which was reverted in
https://github.com/cozystack/cozystack/pull/1677.
The fix includes the upstream fix from kubernetes/kubernetes#135537,
which enables short-circuit path for UnstructuredList similar to regular
Unstructured objects, using GVK from the object field instead of
typeToGVK.
Changes:
- Refactored rest.go handlers to use typed Application objects
- Removed unstructured.Unstructured conversions
- Fixed UnstructuredList GVK handling
- Updated dependencies in go.mod/go.sum
- Added e2e test for OpenAPI validation
- Updated Dockerfile
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
This PR prepares the release `v0.38.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container image versions from v0.38.1 to v0.38.2 across system
packages with corresponding digest updates. All image references have
been bumped to the latest version with new SHA256 hashes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
This PR reverts the changes introduced in #1630 that caused a regression
affecting namespace deletion and upgrades from previous versions,
particularly when running migration 20.
### Symptoms
Users are experiencing issues with namespace deletion and upgrades. The
namespace controller reports:
```
NamespaceDeletionContentFailure: True
ContentDeletionFailed: Failed to delete all resource types, 25 remaining:
content is not a list: []unstructured.Unstructured
```
The error indicates that Kubernetes API server expects a proper list
type, but receives `[]unstructured.Unstructured` instead, which breaks
namespace finalization.
### Root Cause
The changes in #1630 modified how dynamic list kinds are represented,
switching from `unstructured.UnstructuredList` to typed
`ApplicationList`. While this was intended to fix kind representation
issues, it introduced a compatibility problem:
1. The namespace deletion controller expects resources to implement the
standard Kubernetes list interface
2. The typed lists may not be properly recognized during namespace
finalization
3. This breaks upgrades from previous versions where resources were
stored as unstructured
### Solution
This PR reverts the problematic changes to restore compatibility with
namespace deletion and upgrade scenarios. We are planning a more
comprehensive fix that will address both the original kind
representation issue and namespace deletion compatibility.
### Related Issues
- Fixes namespace deletion failures
- Fixes upgrade issues from previous versions (migration 20)
- Reverts regression introduced in #1630
### Next Steps
A more comprehensive fix is being prepared that will:
- Properly handle kind representation for dynamic types
- Ensure compatibility with namespace deletion controller
- Support upgrades from previous versions
**Chores**
* Updated SeaweedFS Helm chart templates with configurable certificate
duration and renewal settings.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- Extended CA certificate duration to reduce disruptive CA rotations.
- Added mechanism to sync SeaweedFS client certificates across clusters.
- Enhanced certificate management to ensure Hikube provisioner always receives up-to-date client certificates.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced CA certificate configuration for SeaweedFS by extending the
validity duration and adjusting the renewal window to improve long-term
certificate management and system stability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[tenant][kubernetes] Introduce better cleanup logic
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an automated pre-delete cleanup job for tenant namespaces to
remove tenant-related releases during uninstall.
* **Improvements**
* Cleanup now runs early in the uninstall sequence with a clearer,
stepwise orchestration for resource removal.
* Expanded permissions and execution allowances to enable the cleanup
workflow.
* Deployment annotations now use a content checksum to better detect
config changes.
* **Removals**
* Previous teardown sequence for certain release types was removed and
replaced by the new workflow.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds config hash annotations to the dashboard deployment
templates to ensure pods are automatically restarted when their
configuration changes.
Changes:
- Added `checksum/config` annotation to `nginx.yaml` deployment that
computes a hash of `nginx-config.yaml`
- Added `checksum/config` annotation to `web.yaml` deployment that
computes a hash of `configmap.yaml`
This replaces the hardcoded checksum in nginx.yaml with a dynamically
computed one, and adds a new checksum for the web deployment.
### Release note
```release-note
[dashboard] Add config hash annotations to automatically restart pods when configuration changes
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Add changelogs for v0.37.* and v0.38.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VPC with Multus CNI, dashboard VNC, configurable worker versions,
dedicated lineage webhook daemonset, tenant workload listing
* **Bug Fixes**
* RBAC and namespace listing fixes (including non‑OIDC/system:masters),
dashboard logout, migration/readiness checks, load balancer cleanup,
various chart/config fixes
* **Security**
* Redis image update, stricter ingress HTTPS enforcement, flux operator
hardening
* **Documentation**
* Kubernetes troubleshooting, backup/recovery, migration guidance,
website and mobile updates
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Add AI-agent for changelogs generation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated agent documentation to include changelog generation guidance
* Added comprehensive changelog generation guide with workflow
procedures and validation steps
* **Chores**
* New scripts added for optional repository management and release notes
publishing
* Project configuration updated to exclude additional directories
* Removed outdated changelog entry
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds instructions for AI agents on how to handle comments from
AI bot reviewers (like Qodo, Copilot, etc.).
Changes:
- Added new section "Addressing AI Bot Reviewer Comments" to
`docs/agents/contributing.md`
- Instructions on how to get PR comments using `gh pr view --comments`
- Guidelines on evaluating each comment carefully (don't blindly apply
all suggestions)
- Clear criteria for when to apply and when to skip AI bot suggestions
- Emphasis on leaving changes uncommitted for user review
Key principles:
- Evaluate each suggestion based on context, project conventions, and
impact
- Apply legitimate fixes but skip over-engineering or style mismatches
- Leave changes uncommitted so the user can review and decide
### Release note
```release-note
[docs] Add instructions for AI agents on handling AI bot reviewer comments
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added guidance for addressing AI bot reviewer comments in the
contributor guide.
* Introduced comprehensive changelog generation procedures.
* **Chores**
* Finalized v0.37.0 release.
* Enhanced release automation infrastructure with new tooling scripts
for repository checks and release note uploads.
* Updated project documentation index.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds AI agent documentation structure to help AI coding
assistants work more effectively with the Cozystack codebase.
Changes:
- Added `AGENTS.md` in the root with an overview and agent documentation
table
- Created `docs/agents/` directory with specialized agent instructions:
- `overview.md` - Project structure and conventions
- `contributing.md` - Git workflow, commits, and pull requests
- `releasing.md` - Release process (references `docs/release.md`)
- Organized documentation to make it easy for AI agents to find relevant
instructions
### Release note
```release-note
[docs] Add AI agent documentation structure with instructions for contributing, releasing, and project overview
```
This PR prepares the release `v0.38.0`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Chores
* Updated image references and digests across multiple packages from
alpha prerelease versions to stable v0.38.0 releases, including updates
to kubevirt-csi-driver, cozystack-api, cozystack-controller, dashboard
components, kamaji, kubeovn modules, s3manager, and other system
packages.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Remove spaces while processing exposed-external-ips list in cozystack
configmap as they 1) are user-specified and 2) lead to an incorrect
resource being created from it.
### Release note
```release-note
Remove spaces while processing exposed-external-ips list in cozystack configmap
```
This patch breaks introduces a helper function in cozy-lib to correctly
handle special case resources when transforming a nested map of limits
and requests to a flat map suitable for use in resourceQuotas. As a
result, admins can now specify any types of resources as resource quotas
for tenants, and they will be correctly transformed to the correct
format for the underlying kubernetes ResourceQuota. In addition to the
previously supported compute resources, such as CPU, memory, and custom
resources, like GPUs, special quota strings such as
"services.loadbalancers" are now correctly handled.
```release-note
[cozy-lib,platform] Support resource quotas for special kubernetes
quotas, such as service.loadbalncer count and others.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
Update MariaDB operator to a new version.
```release-note
[ci,dx] Update MariaDB operator version
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Support for external MariaDB instances and physical backups management
* Validation webhook added for physical backups
* New config option to specify MariaDB image registry/name
* **Updates**
* MariaDB operator bumped to 25.10.2
* Default MariaDB version updated from 11.4 to 11.8
* Expanded Kubernetes permissions for endpoint slices and volume
snapshots
* Pod metadata label added to MariaDB pods
* **Documentation**
* Updated docs links and version badges in charts README
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch adds special handling for raw Kubernetes ResourceQuota
fields, such as `services.loadbalancers`, preventing them from being
wrapped as `limits.*` or `requests.*` keys by the flatten helper. This
ensures that LoadBalancer quotas render correctly in tenant
specifications.
```release-note
[cozy-lib] Correctly render services.loadbalancers in ResourceQuota without limits.* or requests.* prefixes.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Resource flattening now handles compute and quota keys separately:
compute values are sanitized/flattened, quota-like inputs are emitted
directly as plain YAML.
* **Documentation**
* Added in-template comments and clarified examples for resource
processing behavior.
* **New Features**
* CI now runs unit tests; new test targets and test harnesses added
along with a test chart and test cases for quota handling.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR updates Piraeus Operator to enable RWX support released in
https://github.com/piraeusdatastore/piraeus-operator/releases/tag/v2.10.0
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[linstor] Update Piraeus Operator to v2.10.1 to enable RWX support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added NFS server component and affinity controller support for
enhanced storage configuration
* Expanded kubectl output with additional status columns for improved
cluster visibility
* Added support for pod labels, annotations, and security context
customization
* **Chores**
* Updated Helm chart to version 2.10.1
* Updated multiple component versions including CSI provisioner,
snapshotter, DRBD reactor, and other dependencies
* **Documentation**
* Updated README to reference external Helm-based deployment guidelines
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The cozy-lib library package got complicated enough to warrant its own
unit tests. Since unit tests are a "good thing" (tm), a somewhat generic
framework for running all kinds of unit tests was introduced into the CI
pipeline and Makefile targets. For now all it runs is `make test`
against the `packages/{library,apps,system,extra}/*` directories,
wherever a `test` target is present in the Makefile, and for now this is
only for the `cozy-lib` Helm library chart.
### Release note
```release-note
[ci,dx] Introduce a scaffold for running unit tests locally and in CI
and add the first unit tests for the cozy-lib helper Helm chart.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Integrated Helm chart unit tests into the CI/CD pipeline for automated
validation.
* Established test infrastructure for cozy-lib package with test cases
for quota configuration and resource validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch breaks introduces a helper function in cozy-lib to correctly
handle special case resources when transforming a nested map of limits
and requests to a flat map suitable for use in resourceQuotas. As a
result, admins can now specify any types of resources as resource quotas
for tenants, and they will be correctly transformed to the correct
format for the underlying kubernetes ResourceQuota. In addition to the
previously supported compute resources, such as CPU, memory, and custom
resources, like GPUs, special quota strings such as
"services.loadbalancers" are now correctly handled.
```release-note
[cozy-lib,platform] Support resource quotas for special kubernetes
quotas, such as service.loadbalncer count and others.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Update MariaDB operator to a new version.
```release-note
[ci,dx] Update MariaDB operator version
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
This patch adds special handling for raw Kubernetes ResourceQuota fields,
such as `services.loadbalancers`, preventing them from being wrapped as
`limits.*` or `requests.*` keys by the flatten helper. This ensures that
LoadBalancer quotas render correctly in tenant specifications.
```release-note
[cozy-lib] Correctly render services.loadbalancers in ResourceQuota without limits.* or requests.* prefixes.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
The cozy-lib library package got complicated enough to warrant its own
unit tests. Since unit tests are a "good thing" (tm), a somewhat generic
framework for running all kinds of unit tests was introduced into the CI
pipeline and Makefile targets. For now all it runs is `make test`
against the `packages/{library,apps,system,extra}/*` directories,
wherever a `test` target is present in the Makefile, and for now this is
only for the `cozy-lib` Helm library chart.
### Release note
```release-note
[ci,dx] Introduce a scaffold for running unit tests locally and in CI
and add the first unit tests for the cozy-lib helper Helm chart.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This PR prepares the release `v0.38.0-alpha.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container image references across system components from
v0.38.0-alpha.1 to v0.38.0-alpha.2, including kubevirt-csi-driver,
cozystack-api, cozystack-controller, dashboard, kamaji, kubeovn, and
related services.
* Updated corresponding image digests to reflect new component builds.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Introduce VNC console
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added VNC tab for accessing VirtualMachine and VMInstance resources
directly from the dashboard
* **Chores**
* Updated base images and builder references
* Enhanced proxy configuration with improved header handling and
extended connection timeouts
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The VPC chart incorrectly used the wrong template for the subjects that
should have access to the configmap info resource. This patch grants
this access to all subjects at or above a certain access level, rather
than just at a specific level.
### Release note
```release-note
[vpc] Grant read access to the subnets configmap to all users inside a
tenant.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Enhanced role binding generation to better incorporate access level
information in subject configuration and access control setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The VPC chart incorrectly used the wrong template for the subjects that
should have access to the configmap info resource. This patch grants
this access to all subjects at or above a certain access level, rather
than just at a specific level.
### Release note
```release-note
[vpc] Grant read access to the subnets configmap to all users inside a
tenant.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VPC subnets are now displayed in the dashboard details view with
dedicated information blocks
* Subnet data is presented with improved formatting and structured
columns for better visibility
* Access controls updated to support proper subnet data permissions
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
To use the new VPC feature, users need to pass the subnet ID to the VMs
they wish to launch in a given VPC/subnet. As the dashboard cannot
compute the subnet ID in the same manner as the Helm template, a helper
configmap is created, containing the details of the subnets attached to
a given VPC. This configmap is queried by the dashboard frontend to
render those details to the user.
### Release note
```release-note
[vpc,dashboard] Expose subnet details to end-user in the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch fixes an issue with the Cozystack API server that causes it
to respond with the first registered dynamic list kind when listing
dynamic resources. E.g., when running `k get postgreses`, the raw JSON
response from the cozystack API would be
```json
{
"apiVersion":"apps.cozystack.io/v1alpha1",
"kind":"BootBoxList",
"items": [
{
"apiVersion":"apps.cozystack.io/v1alpha1",
"kind":"Postgres",
...
},
...
],
...
}
```
The root cause is the way the `Typer` interface is implemented for the
`runtime.Scheme` where the dynamic types are registered. Since the base
type for all dynamic types is a `&cozyv1alpha1.Application{}`, all types
are registered under the same key and the `Typer` defaults to the first
`GroupVersionKind` that was registered. Only when a correctly formed
`&unstructured.Unstructured{}` is returned by the API, is this resolving
logic circumvented and the `GroupVersionKind` is instead inferred from
the fields of the returned object. Even an `UnstructuredList` is not
acceptable as a return type, instead the `items` key should be directly
set on the underlying `map[string]interface{}`.
This patch implements the changes detailed above. Additionally, the
following features, fixes, and improvements are added:
* Makefile targets to build and run the Cozystack API locally, against a
Kubernetes server in the environment's KUBECONFIG. Debugging with Delve
is also supported.
* CI tests are added to verify the new changes.
* A bug in the registration of the corev1alpha1 types is fixed.
* Updated the `ConvertToTable` function to properly handle list kinds
which might be of the `&unstructured.Unstructured{}` concrete type (not
an `UnstructuredList`).
* The scheme used by the API server's Client and WatchClient is
separated from the scheme used to serve dynamic types.
* The client config for reading the runtime configuration now uses the
controller-runtime, which handles flags and environment variables
properly, unlike `clientcmd`.
### Release note
```release-note
[api] Fix incorrect list kind for list requests to the Cozystack API for
dynamic resources. Add Makefile targets for local testing. Minor schema
building improvements.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added an end-to-end test validating Kubernetes API kinds for tenants
and ingresses.
* **Chores**
* Improved local development: ignore local API server config, added
run/debug targets with local TLS tooling, and added an OpenSSL config
for CSR generation.
* **Refactor**
* Internal API server and registry storage reworked; may affect
integrations expecting concrete resource types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch fixes an issue with the Cozystack API server that causes it
to respond with the first registered dynamic list kind when listing
dynamic resources. E.g., when running `k get postgreses`, the raw JSON
response from the cozystack API would be
```json
{
"apiVersion":"apps.cozystack.io/v1alpha1",
"kind":"BootBoxList",
"items": [
{
"apiVersion":"apps.cozystack.io/v1alpha1",
"kind":"Postgres",
...
},
...
],
...
}
```
The root cause is the way the `Typer` interface is implemented for the
`runtime.Scheme` where the dynamic types are registered. Since the base
type for all dynamic types is a `&cozyv1alpha1.Application{}`, all types
are registered under the same key and the `Typer` defaults to the first
`GroupVersionKind` that was registered. Only when a correctly formed
`&unstructured.Unstructured{}` is returned by the API, is this resolving
logic circumvented and the `GroupVersionKind` is instead inferred from
the fields of the returned object. Even an `UnstructuredList` is not
acceptable as a return type, instead the `items` key should be directly
set on the underlying `map[string]interface{}`.
This patch implements the changes detailed above. Additionally, the
following features, fixes, and improvements are added:
* Makefile targets to build and run the Cozystack API locally, against a
Kubernetes server in the environment's KUBECONFIG. Debugging with
Delve is also supported.
* CI tests are added to verify the new changes.
* A bug in the registration of the corev1alpha1 types is fixed.
* Updated the `ConvertToTable` function to properly handle list kinds
which might be of the `&unstructured.Unstructured{}` concrete type
(not an `UnstructuredList`).
* The scheme used by the API server's Client and WatchClient is
separated from the scheme used to serve dynamic types.
* The client config for reading the runtime configuration now uses the
controller-runtime, which handles flags and environment variables
properly, unlike `clientcmd`.
### Release note
```release-note
[api] Fix incorrect list kind for list requests to the Cozystack API for
dynamic resources. Add Makefile targets for local testing. Minor schema
building improvements.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Similar to an earlier issue with DataVolumes remaining after deleting
the tenant k8s cluster using them, a similar problem is observed with
LoadBalancer services consuming external IPs. This patch adds another
step to the cleanup Helm hook to delete any such services.
### Release note
```release-note
[kubernetes] Add a cleanup hook to delete LoadBalancer services after
deleting the tenant Kubernetes cluster that they were servicing.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automatic cleanup of LoadBalancer services during resource
deletion workflows.
* **Chores**
* Updated resource naming conventions for consistency.
* Extended service management permissions in access control
configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Similar to an earlier issue with DataVolumes remaining after deleting
the tenant k8s cluster using them, a similar problem is observed with
LoadBalancer services consuming external IPs. This patch adds another
step to the cleanup Helm hook to delete any such services.
### Release note
```release-note
[kubernetes] Add a cleanup hook to delete LoadBalancer services after
deleting the tenant Kubernetes cluster that they were servicing.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Fixes#1623
### Release note
```release-note
Multus updated to the latest version to address race condition during startup.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated kube-multus container to version v4.2.3-thick.
* Increased memory resource allocation from 100Mi to 300Mi for improved
stability and performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases kube-ovn-cni limits
### Release note
```release-note
Increased kube-ovn-cni limits so that it is not oomkilled during startup on busy nodes.
```
## What this PR does
This patch grants "admin" permissions to super-admins, "use" permissions
to admins and super-admins, "view" permissions to "use"-privileged
users, admins, and super-admins. Previously lower-privileged roles were
not assigned to higher-privileged users, so a viewer could excercise
their basic read-only permissions which were not available to
high-privilege users. This patch corrects the template function used to
generate subjects in rolebindings, fixing the issue.
### Release note
```release-note
[rbac] Fix issue of privileged users not having low-privilege read-only
permissions.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Updated access level handling in role-based authorization to ensure
proper permission evaluation across tenant environments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch grants "admin" permissions to super-admins, "use" permissions
to admins and super-admins, "view" permissions to "use"-privileged
users, admins, and super-admins. Previously lower-privileged roles were
not assigned to higher-privileged users, so a viewer could excercise
their basic read-only permissions which were not available to
high-privilege users. This patch corrects the template function used to
generate subjects in rolebindings, fixing the issue.
### Release note
```release-note
[rbac] Fix issue of privileged users not having low-privilege read-only
permissions.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
The kubelet version of tenant k8s clusters is baked into the worker VM
image. Previously, selecting any version of tenant k8s had an impact
only on the controlplane, the workers were fixed at v1.33. This patch
modifies the KubeadmConfigTemplate to attempt to download the
user-selected versions of kubelet and kubeadm and replace the baked-in
versions with those. If failing, the bootstrap continues with the
baked-in versions.
### Release note
```release-note
[kubernetes] Make kubelet versions on tenant k8s clusters' worker nodes
user-configurable.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automated Kubernetes component update during bootstrap with x86_64 and
ARM64 support; invoked as part of pre-bootstrap steps to ensure
kubelet/kubeadm versions.
* **Tests**
* Per-test isolated kubeconfig filenames to avoid conflicts.
* Simplified, stricter per-node version validation and alignment of
readiness checks; increased machine deployment readiness timeout to 10
minutes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The kubelet version of tenant k8s clusters is baked into the worker VM
image. Previously, selecting any version of tenant k8s had an impact
only on the controlplane, the workers were fixed at v1.33. This patch
modifies the KubeadmConfigTemplate to attempt to download the
user-selected versions of kubelet and kubeadm and replace the baked-in
versions with those. If failing, the bootstrap continues with the
baked-in versions.
### Release note
```release-note
[kubernetes] Make kubelet versions on tenant k8s clusters' worker nodes
user-configurable.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Adds check also for core.cozystack.io group
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-controller] improve API tests
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
# Release Notes
* **Tests**
* Enhanced validation during installation to verify multiple API
services
* Expanded OpenAPI endpoint verification to include additional services
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard-controller] Fix static resources reconciliation and showing secrets
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Revert per-vm network policies functionality introduced by
https://github.com/cozystack/cozystack/pull/1611
As it is not working as expected any way.
This is temporary solution before implementing full-fledged security
groups in Cozystack
fixes https://github.com/cozystack/cozystack/issues/1601
alternative solution: https://github.com/cozystack/cozystack/pull/1602
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[virtual-machine] Revert per-vm network policies
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- **[dashboard-controller] Fix labelSelectors**
- **[cozystack-api] Enhance TenantSecrets filtering**
- **[cozystack-api] Fix sorting for TenantSecrets**
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-api][dashboard] Fix filtering for application services/ingresses/secrets
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Standardized internal configuration naming conventions across
dashboard components.
* Enhanced tenant secret validation and filtering logic with improved
label-based operations for consistency and correctness.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Fixes issue:
```
error: template: tcp-balancer/templates/deployment.yaml:37:23: executing "tcp-balancer/templates/deployment.yaml" at <include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesP
reset .Values.resources $)>: error calling include: template: tcp-balancer/charts/cozy-lib/templates/_resources.tpl:157:20: executing "cozy-lib.resources.defaultingSanitize" at <deepCopy $re
sources>: error calling deepCopy: reflect: call of reflect.Value.Type on zero Value
```
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozy-lib] Fix: handling resources=nil
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved resource handling to ensure proper behavior when resources
are not provided, enhancing system reliability and consistency in
resource merging operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When deleting a Kubernetes, some resources may linger post deletion
because of a race to remove HelmReleases deployed inside the tenant
cluster and the removal of the cluster and its controlplane itself. This
patch modifies the existing pre-delete hook to remove those helmreleases
instead of simply suspending them. Similarly, datavolumes may also
remain. These are now delete with a post-delete hook.
### Release note
```release-note
[kubernetes] Use Helm hooks to clean up HelmReleases deployed in tenant
clusters and DataVolumes backing the tenant clusters' PVCs when deleting
a tenant Kubernetes.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added an automated post-delete cleanup job to remove persistent data
volumes scoped to the release namespace when a release is deleted.
* Updated Helm release teardown to actively delete lingering release
resources (rather than only suspending them) for cleaner uninstall
behavior.
* Broadened lifecycle hooks to run on successful completions and
expanded teardown permissions to list and delete related release
artifacts, including gateway CRDs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When deleting a Kubernetes, some resources may linger post deletion
because of a race to remove HelmReleases deployed inside the tenant
cluster and the removal of the cluster and its controlplane itself. This
patch modifies the existing pre-delete hook to remove those helmreleases
instead of simply suspending them. Similarly, datavolumes may also
remain. These are now delete with a post-delete hook.
### Release note
```release-note
[kubernetes] Use Helm hooks to clean up HelmReleases deployed in tenant
clusters and DataVolumes backing the tenant clusters' PVCs when deleting
a tenant Kubernetes.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Move patches to upstream: `namespaces` and `hide inside`
- Introduce flatMap logic
- Remove `tenantsecretstables` resource
- Extend dashboard-controller to specify `multilineString` for any
string without enum in spec (previusly it was for all strings)
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] sync with upstream & enhancements
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced OpenAPI form handling: string fields now better support
multiline input.
* **Improvements**
* Secrets UI and API alignment: secrets display and data keys updated
for consistency.
* Form generation improved for nested objects and arrays.
* Deployment defaults adjusted (logger flags normalized; inside feature
hidden via env).
* **Removed**
* Removed the "Inside" header menu item and the legacy secrets-table
API/resource.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When deleting a Kubernetes, some resources may linger post deletion
because of a race to remove HelmReleases deployed inside the tenant
cluster and the removal of the cluster and its controlplane itself. This
patch modifies the existing pre-delete hook to remove those helmreleases
instead of simply suspending them. Similarly, datavolumes may also
remain. These are now delete with a post-delete hook.
### Release note
```release-note
[kubernetes] Use Helm hooks to clean up HelmReleases deployed in tenant
clusters and DataVolumes backing the tenant clusters' PVCs when deleting
a tenant Kubernetes.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR fixes an issue where NATS user credentials were being
regenerated on every helm release update, rather than reusing existing
secrets. The fix implements the same secret reuse pattern that is
already used in the postgres app.
### Changes:
- Added `lookup` call to fetch existing credentials secret before
generating passwords
- Pre-populate passwords from existing secret data (base64 decoded)
- Only generate new random passwords for users that don't have existing
credentials
### Behavior:
- **Before**: Every helm upgrade would regenerate credentials for users
without explicit passwords, breaking existing connections
- **After**: Existing credentials are preserved across helm upgrades,
matching postgres app behavior
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[nats] Fix credential regeneration on helm release updates by implementing existing secret lookup pattern
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* NATS deployments can now read and reuse existing release credentials,
reducing unnecessary credential rotation and keeping logins consistent
across updates.
* When credentials are missing, the system still auto-generates
passwords; when users are defined it emits the computed credentials for
use by the deployment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
### Release note
```release-note
[vpc] Change the subnets configmap structure from
.data.subnets==[]Subnet to .data==map[SubnetName]Subnet for simpler
representation in the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Restructured VPC subnet data organization in configuration from a
static list format to a dynamic map structure, where each subnet is now
stored with its own key containing subnet name, ID, and CIDR
information.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Adds kubevirt settings:
`vmRolloutStrategy`: how changes to a manifest are propagated to a vm:
changes will be applied on-the-fly if possible (such as guest memory)
`workloadUpdateStrategy`: how vms will react to an eviction, less
disruptive method will be used.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Kubevirt rollout and eviction settings tuned
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Enhanced VM rollout strategy with Live Update support
* Introduced configurable workload update strategy with Live Migration
and Eviction options
* Added batch eviction controls for optimized resource management
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
### Release note
```release-note
[vpc] Change the subnets configmap structure from
.data.subnets==[]Subnet to .data==map[SubnetName]Subnet for simpler
representation in the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard-controller] Move bages generation logic to internal dashboard component
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactors**
* Dashboard badges now render with a unified ResourceBadge component
across columns and headers.
* Explicit badge size options removed; badges use simplified default
sizing.
* Badge payload simplified to a single value-driven field; abbreviation
and rendering are handled by the badge component.
* Stable per-kind color hashing removed; badges use streamlined,
consistent styling with optional color overrides.
* Column and header labels expanded to full descriptive names for
clearer navigation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
```release-note
[adopters] Add QOSI to ADOPTERS.md
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added a new adopter entry for QOSI (dated 2025-10-04) with a
descriptive use-case.
* Removed an erroneous stray dash that preceded the new entry.
* Cleaned up formatting (trailing newline added); existing adopter list
otherwise unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The new VPC feature creates a configmap, containing the list of subnets
under the VPC. However, the configmap has the VPC ID in its name, rather
than the name of the VPC, making it harder to target inside the
dashboard, as the helper functions from Helm (e.g. sha256sum) may not be
available in the dashboard's custom resources. This patch renames the
configmap to use the VPC's name.
### Release note
```release-note
[vpc] Change the subnet configmap name to a human-friendly value
(matching the VPC name), instead of being derived via a sha256sum,
making it easier to reference in the dashboard.
```
This patch updates the Flux Operator Deployment to remove hostPort and
hostNetwork, ensuring that ports 8080 and 8081 are only accessible
within the cluster. This prevents external exposure and improves
security.
```release-note
[flux] Close Flux Operator ports (8080/8081) to external access for improved security.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Security**
* Added a cluster-wide network policy for the Flux Operator to block
external access to internal service ports (notably TCP 8080 and 8081)
while preserving intra-cluster communication.
* **Chores**
* Update process now applies the new network policy as part of Flux
Operator deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Migrate patches to upstream project
[dashboard] Fix nested lists in addtiionalProperties
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced form generation with improved type inference and nested
property resolution for dynamic form fields.
* **Bug Fixes**
* Fixed stream data handling issues and improved form field value
normalization.
* Better support for array item initialization with sensible defaults
based on field types.
* **Chores**
* Updated API endpoints for namespace resource management.
* Updated container images and configurations.
* Improved tenant branding configuration structure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
[nats] Fixes for NATS App Helm chart, fix template issues with
config.merge value
## What this PR does
This PR fixes two critical bugs in the NATS application chart
(`packages/apps/nats`) that prevented successful deployment when using
user authentication with custom configuration:
**Bug #1: YAML Parse Error with Empty config.merge**
- When users were defined but `config.merge` was empty (`{}`), the Helm
template would fail with YAML parse errors
- Fixed by adding safe existence checks before accessing
`.Values.config.merge` and preventing rendering of empty merge blocks
**Bug #2: Incorrect Config Nesting**
- When both `users` and `config.merge` were defined, custom NATS config
values (e.g., `max_payload`, `max_connections`) were incorrectly nested
as children of the `accounts` object instead of being placed at the root
NATS configuration level
- This caused NATS pods to crash with "Expected map entries for
accounts" errors
- Fixed by correcting the indentation from `nindent 12` to `nindent 10`
in the template
The related issue is #1354
### Changes Made
**File**: `packages/apps/nats/templates/nats.yaml`
1. Line 55: Added safe existence checks for `.Values.config.merge`
2. Line 66: Added length validation to prevent rendering empty merge
objects
3. Line 67: Fixed indentation to ensure config values merge at root
level
### Impact
- Enables NATS deployments with user authentication and custom
configuration
- Fixes Stalwart mail server clustering scenarios that require NATS with
authentication
- Resolves HelmRelease failures in multi-tenant environments
### Testing
Can be tested with:
```yaml
apiVersion: apps.cozystack.io/v1alpha1
kind: NATS
metadata:
name: test-nats
spec:
replicas: 2
users:
testuser: {}
config:
merge:
max_payload: 2097152
max_connections: 500
```
Expected result: HelmRelease succeeds, NATS pods start successfully, and
configuration is valid.
---
### Release note
```release-note
[nats] Fix NATS application chart template bugs that prevented deployments with user authentication and custom configuration.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced configuration validation with improved guard conditions to
ensure proper handling of merge configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The new VPC feature creates a configmap, containing the list of subnets
under the VPC. However, the configmap has the VPC ID in its name, rather
than the name of the VPC, making it harder to target inside the
dashboard, as the helper functions from Helm (e.g. sha256sum) may not be
available in the dashboard's custom resources. This patch renames the
configmap to use the VPC's name.
### Release note
```release-note
[vpc] Change the subnet configmap name to a human-friendly value
(matching the VPC name), instead of being derived via a sha256sum,
making it easier to reference in the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch makes the fixes from `b1ebc9cc` by @insignia96 terser by
making use of Helm's `with` blocks.
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
The recent patch introducing VPCs in Cozystack did not include enabling
Multus, which is a dependency for this feature. This patch enables
Multus by default in the paas-full bundle.
### Release-note
```release-note
[vpc] Enable Multus by default as a necessary dependency for VPCs.
```
## What this PR does
The recent patch introducing VPCs in Cozystack did not include enabling
Multus, which is a dependency for this feature. This patch enables
Multus by default in the paas-full bundles.
### Release-note
```release-note
[vpc] Enable Multus by default as a necessary dependency for VPCs.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubevirt] Fix: kubevirt metrics rule
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed Prometheus alert rule expressions for virtual machine
monitoring. Corrected status and phase condition comparisons to
accurately identify when virtual machines are not running, ensuring
alerts trigger reliably in such scenarios. These improvements enhance
the accuracy of monitoring notifications.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Removes Talos lldp extension. Please build a custom talos image with factory.talos.dev if you need it.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Talos lldp extension removed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Removed LLDPD (Link Layer Discovery Protocol Daemon) system extension
from cluster configuration. This eliminates the LLDPD kernel module from
cluster setups, removes LLDPD references from build processes, and
updates installation profiles across all supported deployment methods
including bare metal, cloud environments, and ISO installations,
resulting in a reduced system footprint.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch updates the default API Ingress to add the
nginx.ingress.kubernetes.io/force-ssl-redirect annotation, ensuring all
HTTP traffic (port 80) is redirected to HTTPS (port 443). This prevents
unencrypted external access and improves security.
```release-note
[ingress] Force HTTPS access for api.dev3.infra.aenix.org and block direct HTTP.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enforced SSL/TLS redirect for API ingress connections to enhance
security.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch updates the RedisFailover Helm template to use a newer,
secure Redis version (8.2.0). This addresses known security issues in
the previous Redis version and ensures safer deployments.
```release-note
[redis] Upgrade Redis to a secure version (8.2.0) to fix security vulnerabilities.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Redis deployments can now specify the container image via a new
configurable value (default: redis:8.2.0), allowing easy override of the
Redis image used.
* **Schema**
* Values schema and resource definition schemas updated to include and
validate the new image setting.
* **Documentation**
* README updated to document the new image parameter.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch updates the default API Ingress to add the
nginx.ingress.kubernetes.io/force-ssl-redirect annotation,
ensuring all HTTP traffic (port 80) is redirected to HTTPS (port 443).
This prevents unencrypted external access and improves security.
```release-note
[ingress] Force HTTPS access for api.dev3.infra.aenix.org and block direct HTTP.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
This patch updates the Flux Operator Deployment to remove hostPort and hostNetwork,
ensuring that ports 8080 and 8081 are only accessible within the cluster.
This prevents external exposure and improves security.
```release-note
[flux] Close Flux Operator ports (8080/8081) to external access for improved security.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
This patch updates the RedisFailover Helm template to use a newer,
secure Redis version (8.2.0). This addresses known security issues
in the previous Redis version and ensures safer deployments.
```release-note
[redis] Upgrade Redis to a secure version (8.2.0) to fix security vulnerabilities.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
It was observed during upgrades to the `cozystack-api` Helm release that
when enabling the local endpoint for the traffic locality feature, hence
switching from a deployment to a daemonset, the deployment may remain
unpruned and the pods of the deployment will continue to run
indefinitely. This patch adds a post-upgrade hook that explicitly
deletes the deployment in case it exists and was not pruned.
### Release-note
```release-note
[api] Delete the cozystack-api deployment in a post-upgrade hook when
migrating to a daemonset and vice-versa.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automatic post-upgrade cleanup that removes outdated cluster resources
when the local Kubernetes API endpoint is disabled.
* Cleanup runs in the release namespace during upgrades and includes
necessary permissions for the cleanup job to complete.
* **Configuration**
* New release value toggles the local Kubernetes API endpoint to enable
or skip the cleanup behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
It was observed during upgrades to the `cozystack-api` Helm release that
when enabling the local endpoint for the traffic locality feature, hence
switching from a deployment to a daemonset, the deployment may remain
unpruned and the pods of the deployment will continue to run
indefinitely. This patch adds a post-upgrade hook that explicitly deletes
the deployment in case it exists and was not pruned.
### Release-note
```release-note
[api] Delete the cozystack-api deployment in a post-upgrade hook when
migrating to a daemonset and vice-versa.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[tenant] Allow listing workload
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[seaweedfs] Fix migration to v3.99
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded seaweedfs configuration to version 3.
* Updated pre-upgrade hook execution conditions to ensure proper upgrade
procedures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The NATS chart incorrectly used podTemplate+merge instead of
container+merge to add resource requests and limits to the NATS
container in the statefulset, but as a result it just completely wiped
out the default container spec. By moving the overrides under the
container key, the upstream chart now correctly merges the resource
requests, instead of overwriting the container spec.
### Release note
```release-note
[nats] Fix incorrect path to container resources in the NATS chart.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Restructured NATS deployment configuration while maintaining existing
functionality and resource settings; templates were reorganized to
streamline how container and resource definitions are represented. No
functional or behavioral changes are expected for deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The NATS chart incorrectly used podTemplate+merge instead of
container+merge to add resource requests and limits to the NATS
container in the statefulset, but as a result it just completely wiped
out the default container spec. By moving the overrides under the
container key, the upstream chart now correctly merges the resource
requests, instead of overwriting the container spec.
### Release note
```release-note
[nats] Fix incorrect path to container resources in the NATS chart.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch increases the connection and request timeouts used in the E2E
tests when communicating with the Kubernetes API. The change improves
test stability under high load and slow cluster response conditions.
```release-note
[e2e] Increase connection and request timeouts for Kubernetes API calls in E2E tests to improve stability.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Optimized infrastructure timeout and polling configurations to improve
deployment reliability and test execution efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch increases the connection and request timeouts used in the
E2E tests when communicating with the Kubernetes API. The change improves
test stability under high load and slow cluster response conditions.
```release-note
[e2e] Increase connection and request timeouts for Kubernetes API calls in E2E tests to improve stability.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update LINSTOR v1.32.3
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Update openapi-ui to v1.0.3
- Show YAML editor as readonly in YAML tab
- Remove inside link from user menu
- fix editing for tenantmodules, fixes
https://github.com/cozystack/cozystack/issues/1550
- fix editing valuesOverride, fixes
https://github.com/cozystack/cozystack/issues/1560
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Update openapi-ui v1.0.3 + fixes
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Made YAML editor read-only to prevent accidental modifications in
details view
* Fixed API request header handling to prevent stream abort issues
* Updated resource API endpoint paths for correct data retrieval
* Removed menu navigation item from user interface
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add VPC support
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Added VPC support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
- Added Virtual Private Cloud (VPC) support with configurable subnets,
per-subnet network attachments, and generated subnet resources.
- Enabled subnet support for Virtual Machine and VM Instance: additional
interfaces, Multus networks, and conditional cloud-init/network secret
wiring for supported images.
* **Documentation**
- Added/updated docs and examples for VPC, Virtual Machine, and VM
Instance showing subnet parameters.
* **Chores**
- Expanded admin role permissions to manage Virtual Private Cloud
resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Helm values now control ingress paths; computed cluster endpoint env
vars are injected.
* Optional container securityContext for volume init containers added.
* Node architecture-specific targeting disabled by default.
* **Refactor**
* Image configuration reorganized with separate registry field;
container image build simplified.
* **Bug Fixes / Behavior**
* S3-related authorization and signature handling changed; S3 gateway
toggled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR enables building of `seaweedfs` image.
Also backports patch from upstream
https://github.com/seaweedfs/seaweedfs/pull/7335
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[seaweedfs] Allow users to discover their buckets
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* S3 signature handling adjusted so signature verification focuses on
authentication; permission checks are evaluated afterward.
* **Chores**
* Build process now discovers and uses remote release versions
dynamically.
* Introduced an optimized multi-stage container build with improved
tagging and registry caching.
* Added configurable image settings (global image name and image tag)
for deployment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
In a previous patch (#1555) the reconciliation loop for the OpenAPI UI
resources was accidentally removed. This patch reintroduces a separate
controller, which handles updates to CozystackResourceDefinitions and
creates, updates, or deletes the dashboard's custom resources.
### Release note
```release-note
[dashboard] Reintroduce the accidentally removed reconciler that
autoconfigures custom dashboard resources for the OpenAPI UI.
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
[mariadb-operator] Add post-delete job to remove PVCs
This patch adds a Helm post-delete hook job that removes
PersistentVolumeClaims
left behind after Helm release deletion. The MariaDB Operator currently
does not
handle PVC cleanup, so this job ensures proper resource removal.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[mariadb-operator] Add a post-delete hook job to clean up PVCs left after Helm release deletion.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Persistent storage volumes are now automatically cleaned up when the
MySQL application is deleted, preventing orphaned storage resources from
accumulating in your cluster.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
In a previous patch (#1555) the reconciliation loop for the OpenAPI UI
resources was accidentally removed. This patch reintroduces a separate
controller, which handles updates to CozystackResourceDefinitions and
creates, updates, or deletes the dashboard's custom resources.
### Release note
```release-note
[dashboard] Reintroduce the accidentally removed reconciler that
autoconfigures custom dashboard resources for the OpenAPI UI.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch adds a Helm post-delete hook job that removes PersistentVolumeClaims
left behind after Helm release deletion. The MariaDB Operator currently does not
handle PVC cleanup, so this job ensures proper resource removal.
```release-note
[mariadb-operator] Add a post-delete hook job to clean up PVCs left after Helm release deletion.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Since 0.37, many requests to the k8s API now go through a mutating
webhook (lineage-controller-webhook). Since the lineage webhook makes
multiple requests to the k8s API and, indirectly, to the Cozystack API
server, each request for, e.g., creating a secret now causes a lot of
chatter between the webhook, the k8s API, and the Cozystack API. When
this happens cross-node or, worse yet, cross-zone, this can blow up the
latency for simple requests.
### BREAKING CHANGES
This patch changes the Cozystack API to a DaemonSet targetting
controlplane nodes, configures its service for an `Local` internal
traffic policy and adds environment variables indicating that the k8s
API server is to be found at \<hostIP\>:6443, **not only for the
Cozystack API, but also for the lineage-controller-webhook.** This is a
valid configuration in most scenarios, including the default
installation method on top of Talos Linux in Cozystack, however, if this
is not valid in your environment, you must now set the values
`.lineageControllerWebhook.localK8sAPIEndpoint.enabled` and
`.cozystackAPI.localK8sAPIEndpoint.enabled` to `false` in the respective
system Helm releases.
### Release note
```release-note
[api,lineage] Configure all chatter between the Lineage webhook, the
Cozystack API server and the Kubernetes API server to be confined to a
single controlplane node, improving k8s API latency.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional local Kubernetes API endpoint mode with configurable topology
(DaemonSet vs Deployment), replica setting, service behavior, and node
scheduling.
* Certificate lifecycle managed via cert-manager with namespace-scoped
issuers and certificates; secret-backed TLS assets with restricted
permissions.
* Controller runtime flag to select API workload kind; webhook can
optionally target local API host/port.
* **Security**
* Enforced TLS verification using cert-manager CA injection; removed
insecure TLS-skip behavior.
* **Permissions**
* Controller role expanded to allow daemonset management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The managed Kubernetes app accepts a .controPlane.replicas field, but
this value was never used, instead being hardcoded in the
KamajiControlPlane template to 2. This patch fixes this.
### Release note
```release-note
[kubernetes] Pass the .controlPlane.replicas field into the
KamajiControlPlane template, making the replica count of the
controlplane pods user-configurable.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Control plane replica count is now configurable via Helm values,
allowing flexible deployment scaling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The managed Kubernetes app accepts a .controPlane.replicas field, but
this value was never used, instead being hardcoded in the
KamajiControlPlane template to 2. This patch fixes this.
### Release note
```release-note
[kubernetes] Pass the .controlPlane.replicas field into the
KamajiControlPlane template, making the replica count of the
controlplane pods user-configurable.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch drops the custom caching of the Cozystack resource
definitions in favor of the informer cache and adds a flag to the
Cozystack controller to select, whether it restarts the cozystack-api
deployment or the cozystack-api daemonset.
### BREAKING CHANGES
As with the new default behavior of using a local endpoint for the k8s
API by the lineage webhook and the Cozystack API, the Cozystack
controller now also defaults to restarting a Cozystack API DaemonSet
instead of a Deployment. To revert to the old behavior, disable the
local k8s API endpoint on the webhook and cozystack API and set the
`cozystackController.cozystackAPIKind` value in the Cozystack controller
system Helm chart to "Deployment".
### Release note
```release-note
[controller] Use informer cache instead of the older bespoke
implementation and add support for running the Cozystack API as a
DaemonSet.
```
This patch drops the custom caching of the Cozystack resource
definitions in favor of the informer cache and adds a flag to the
Cozystack controller to select, whether it restarts the cozystack-api
deployment or the cozystack-api daemonset.
As with the new default behavior of using a local endpoint for the k8s
API by the lineage webhook and the Cozystack API, the Cozystack
controller now also defaults to restarting a Cozystack API DaemonSet
instead of a Deployment. To revert to the old behavior, disable the
local k8s API endpoint on the webhook and cozystack API and set the
`cozystackController.cozystackAPIKind` value in the Cozystack controller
system Helm chart to "Deployment".
```release-note
[controller] Use informer cache instead of the older bespoke
implementation and add support for running the Cozystack API as a
DaemonSet.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Since 0.37, many requests to the k8s API now go through a mutating
webhook (lineage-controller-webhook). Since the lineage webhook makes
multiple requests to the k8s API and, indirectly, to the Cozystack API
server, each request for, e.g., creating a secret now causes a lot of
chatter between the webhook, the k8s API, and the Cozystack API. When
this happens cross-node or, worse yet, cross-zone, this can blow up the
latency for simple requests.
This patch changes the Cozystack API to a DaemonSet targetting
controlplane nodes, configures its service for an `Local` internal
traffic policy and adds environment variables indicating that the k8s
API server is to be found at <hostIP>:6443, **not only for the Cozystack
API, but also for the lineage-controller-webhook.** This is a valid
configuration in most scenarios, including the default installation
method on top of Talos Linux in Cozystack, however, if this is not valid
in your environment, you must now set the values
`.lineageControllerWebhook.localK8sAPIEndpoint.enabled` and
`.cozystackAPI.localK8sAPIEndpoint.enabled` to `false` in the respective
system Helm releases.
```release-note
[api,lineage] Configure all chatter between the Lineage webhook, the
Cozystack API server and the Kubernetes API server to be confined to a
single controlplane node, improving k8s API latency.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
depends on https://github.com/cozystack/cozyvalues-gen/pull/16
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dx] JSDoc compatible syntax for values.yaml
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Standardized and tightened wording, punctuation and examples across
many charts, READMEs and schemas for clearer parameter docs and
defaults.
* **Refactor**
* Replaced many inline parameter blocks with consistent typedefs/enums
and typed maps (resources, presets, components, addons, storage,
sources, etc.) to unify configuration surfaces.
* **Chores**
* Workflow: updated pre-commit generate step to a newer generator
release.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
With this release, the new Flux 2.7 version becomes generally available
The Flux 2.7 upgrade may require some API bumps. (This PR only upgrades
Flux Operator.)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added support for an optional “source-watcher” component in Flux
installations, selectable via chart configuration.
- Documentation
- Updated READMEs to reflect v0.30.0 and clarified that charts can
install, configure, and automatically upgrade Flux. Version badges
refreshed.
- Chores
- Bumped chart and app versions to 0.30.0 across Flux Operator and Flux
Instance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Turns off kubeovn enableLb, kube-proxy implementation of kube-ovn.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[system] kube-ovn: turn off kube-proxy implementation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a new load balancing configuration option to system settings
(disabled by default).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch moves the build of the Redis operator into the Cozystack
organization and patches it to prevent overwriting third-party labels on
owned resources.
### Release note
```release-note
[redis-operator] Move operator into tree and patch it to retain
third-party labels on owned resources, reducing noisy traffic to the API
server.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Implemented automated Docker image build pipeline with version
tracking and caching.
* Updated image configuration to include repository reference and digest
for reproducibility.
* **Bug Fixes**
* Improved label and annotation handling to preserve existing Kubernetes
resource metadata instead of overwriting it.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch changes all clients in the Cozystack API server to typed ones
from the controller runtime. This should improve the performance of the
API server and simplifies the code by removing work with unstructured
objects and dynamic clients.
### Release note
```release-note
[api] Use typed and cache-backed k8s clients in the Cozystack API to
improve performance. Get rid of operations on unstructured objects and
use of dynamic clients.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Backend migrated to a controller-runtime manager with typed clients
for Kubernetes resources, improving watch reliability and cache sync.
* Storage paths for applications, tenant modules, namespaces, and
secrets now use strongly-typed resource handling for more consistent
behavior.
* **Chores**
* Cluster role expanded to include services in core API permissions.
* **Notes**
* No user-facing API schema changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch changes all clients in the Cozystack API server to typed
ones from the controller runtime. This should improve the performance of
the API server and simplifies the code by removing work with
unstructured objects and dynamic clients.
```release-note
[api] Use typed and cache-backed k8s clients in the Cozystack API to
improve performance. Get rid of operations on unstructured objects and
use of dynamic clients.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch moves the build of the Redis operator into the Cozystack
organization and patches it to prevent overwriting third-party labels on
owned resources.
```release-note
[redis-operator] Move operator into tree and patch it to retain
third-party labels on owned resources, reducing noisy traffic to the API
server.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Previously, all system charts have had their versions bumped with every
update of Cozystack, even if the underlying images and templates did not
change. Additionally, some components would have their docker images
rebuilt and get updated digests because of a changed docker context,
even though no relevant code had been changed. This would cause an
upgrade of several HelmReleases on every upgrade of Cozystack, even when
in practice nothing had changed. This is especially noticeable for
charts, such as kubeovn, which is a dependency for almost all other
system HelmReleases. By moving kubeovn out of the monorepo and
decoupling its release cycle from the release cycle of Cozystack, this
issue can be resolved and is another step towards decomposing Cozystack.
```release-note
[kubeovn] Decouple the release cycle of kubeovn from Cozystack's release
cycle, preventing unnecessary redeployments of all Helm releases, when
upgrading Cozystack.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
@@ -31,4 +31,5 @@ This list is sorted in chronological order, based on the submission date.
| [gohost](https://gohost.kz/) | @karabass_off | 2024-02-01 | Our company has been working in the market of Kazakhstan for more than 15 years, providing clients with a standard set of services: VPS/VDC, IaaS, shared hosting, etc. Now we are expanding the lineup by introducing Bare Metal Kubenetes cluster under Cozystack management. |
| [Urmanac](https://urmanac.com) | @kingdonb | 2024-12-04 | Urmanac is the future home of a hosting platform for the knowledge base of a community of personal server enthusiasts. We use Cozystack to provide support services for web sites hosted using both conventional deployments and on SpinKube, with WASM. |
| [Hidora](https://hikube.cloud) | @matthieu-robin | 2025-09-17 | Hidora is a Swiss cloud provider delivering managed services and infrastructure solutions through datacenters located in Switzerland, ensuring data sovereignty and reliability. Its sovereign cloud platform, Hikube, is designed to run workloads with high availability across multiple datacenters, providing enterprises with a secure and scalable foundation for their applications based on Cozystack. |
|
| [QOSI](https://qosi.kz) | @tabu-a | 2025-10-04 | QOSI is a non-profit organization driving open-source adoption and digital sovereignty across Kazakhstan and Central Asia. We use Cozystack as a platform for deploying sovereign, GPU-enabled clouds and educational environments under the National AI Program. Our goal is to accelerate the region’s transition toward open, self-hosted cloud-native technologies |
| [Cloupard](https://cloupard.kz/) | @serjiott | 2025-12-18 | Cloupard is a public cloud provider offering IaaS and PaaS services via datacenters in Kazakhstan and Uzbekistan. Uses CozyStack on bare metal to extend its managed PaaS offerings. |
This file provides structured guidance for AI coding assistants and agents
working with the **Cozystack** project.
## Activation
**CRITICAL**: When the user asks you to do something that matches the scope of a documented process, you MUST read the corresponding documentation file and follow the instructions exactly as written.
- **Commits, PRs, git operations** (e.g., "create a commit", "make a PR", "fix review comments", "rebase", "cherry-pick")
- Action: Read the file to understand git workflow, commit format, PR process
**Important rules:**
- ✅ **ONLY read the file if the task matches the documented process scope** - do not read files for tasks that don't match their purpose
- ✅ **ALWAYS read the file FIRST** before starting the task (when applicable)
- ✅ **Follow instructions EXACTLY** as written in the documentation
- ✅ **Do NOT skip mandatory steps** (especially in changelog.md)
- ✅ **Do NOT assume** you know the process - always check the documentation when the task matches
- ❌ **Do NOT read files** for tasks that are outside their documented scope
- 📖 **Note**: [`overview.md`](./docs/agents/overview.md) can be useful as a reference to understand project structure and conventions, even when not explicitly required by the task
## Project Overview
**Cozystack** is a Kubernetes-based platform for building cloud infrastructure with managed services (databases, VMs, K8s clusters), multi-tenancy, and GitOps delivery.
# Cozystack Backups – Core API & Contracts (Draft)
## 1. Overview
Cozystack’s backup subsystem provides a generic, composable way to back up and restore managed applications:
* Every **application instance** can have one or more **backup plans**.
* Backups are stored in configurable **storage locations**.
* The mechanics of *how* a backup/restore is performed are delegated to **strategy drivers**, each implementing driver-specific **BackupStrategy** CRDs.
The core API:
* Orchestrates **when** backups happen and **where** they’re stored.
* Tracks **what** backups exist and their status.
* Defines contracts with drivers via shared resources (`BackupJob`, `Backup`, `RestoreJob`).
It does **not** implement the backup logic itself.
This document covers only the **core** API and its contracts with drivers, not driver implementations.
* Set `ownerReferences` so the `BackupJob` is owned by the `Plan`.
**Note:** The `BackupJob` controller resolves the `BackupClass` to determine the appropriate strategy and parameters, based on the `ApplicationRef`. The strategy template is processed with a context containing the `Application` object and `Parameters` from the `BackupClass`.
The Plan controller does **not**:
* Execute backups itself.
* Modify driver resources or `Backup` objects.
* Touch `BackupJob.spec` after creation.
---
### 4.2 BackupClass
**Group/Kind**
`backups.cozystack.io/v1alpha1, Kind=BackupClass`
**Purpose**
Define a class of backup configurations that encapsulate strategy and parameters per application type. `BackupClass` is a cluster-scoped resource that allows admins to configure backup strategies and parameters in a reusable way.
**Key fields (spec)**
```go
typeBackupClassSpecstruct{
// Strategies is a list of backup strategies, each matching a specific application type.
Drivers must **not** modify `BackupJob.spec` or delete `BackupJob` themselves.
---
### 4.4 Backup
**Group/Kind**
`backups.cozystack.io/v1alpha1, Kind=Backup`
**Purpose**
Represent a single **backup artifact** for a given application, decoupled from a particular run. usable as a stable, listable “thing you can restore from”.
**Note:** Parameters are not stored directly in `Backup`. Instead, they are resolved from `BackupClass` parameters when the backup was created. The storage location is managed by the driver (e.g., Velero's `BackupStorageLocation`) and referenced via parameters in the `BackupClass`.
**Key fields (status)**
```go
typeBackupStatusstruct{
PhaseBackupPhase`json:"phase,omitempty"`// Pending, Ready, Failed, etc.
`BackupArtifact` describes the artifact (URI, size, checksum).
**Backup contract with drivers**
* On successful completion of a `BackupJob`, the **driver**:
* Creates a `Backup` in the same namespace (typically owned by the `BackupJob`).
* Populates `spec` fields with:
* The application reference.
* The strategy reference (resolved from `BackupClass` during `BackupJob` execution).
*`takenAt`.
* Optional `driverMetadata`.
* Sets `status` with:
*`phase = Ready` (or equivalent when fully usable).
*`artifact` describing the stored object.
* Core:
* Treats `Backup` spec as mostly immutable and opaque.
* Uses it to:
* List backups for a given application/plan.
* Anchor `RestoreJob` operations.
* Implement higher-level policies (retention) if needed.
**Note:** Parameters are resolved from `BackupClass` when the `BackupJob` is created. The driver uses these parameters to determine where to store backups. The storage location itself is managed by the driver (e.g., Velero's `BackupStorageLocation` CRD) and is not directly referenced in the `Backup` resource. When restoring, the driver resolves the storage location from the original `BackupClass` parameters or from the driver's own metadata.
---
### 4.5 RestoreJob
**Group/Kind**
`backups.cozystack.io/v1alpha1, Kind=RestoreJob`
**Purpose**
Represent a single **restore operation** from a `Backup`, either back into the same application or into a new target application.
* RestoreJob is created either manually or by core.
* Driver controller:
1. Watches `RestoreJob`.
2. On reconcile:
* Fetches the referenced `Backup`.
* Determines effective:
* **Strategy**: `backup.spec.strategyRef`.
* **Storage**: Resolved from driver metadata or `BackupClass` parameters (e.g., `backupStorageLocationName` stored in `driverMetadata` or resolved from the original `BackupClass`).
* **Target application**: `spec.targetApplicationRef` or `backup.spec.applicationRef`.
* If effective strategy’s GVK is one of its supported strategy types → driver is responsible.
3. Behaviour:
* On first reconcile, set `status.startedAt` and `phase = Running`.
Drivers are interchangeable as long as they respect:
* The `BackupJob` and `RestoreJob` contracts.
* The shapes and semantics of `Backup` objects.
---
## 6. Summary
The Cozystack backups core API:
* Uses a single group, `backups.cozystack.io`, for all core CRDs.
* Cleanly separates:
* **When** (Plan schedule) – core-owned.
* **How & where** (BackupClass) – central configuration unit that encapsulates strategy and parameters (e.g., storage reference) per application type, resolved per BackupJob/Plan.
* **Execution** (BackupJob) – created by Plan when schedule fires, resolves BackupClass to get strategy and parameters, then delegates to driver.
* **What backup artifacts exist** (Backup) – driver-created but cluster-visible.
"Interval between telemetry data collection (e.g. 15m, 1h)")
flag.StringVar(&platformSourceURL,"platform-source-url","","Platform source URL (oci:// or https://). If specified, generates OCIRepository or GitRepository resource.")
flag.StringVar(&platformSourceName,"platform-source-name","cozystack-packages","Name for the generated platform source resource (default: cozystack-packages)")
flag.StringVar(&platformSourceRef,"platform-source-ref","","Reference specification as key=value pairs (e.g., 'branch=main' or 'digest=sha256:...,tag=v1.0'). For OCI: digest, semver, semverFilter, tag. For Git: branch, tag, semver, name, commit.")
flag.StringVar(&cozyValuesSecretName,"cozy-values-secret-name","cozystack-values","The name of the secret containing cluster-wide configuration values.")
flag.StringVar(&cozyValuesSecretNamespace,"cozy-values-secret-namespace","cozy-system","The namespace of the secret containing cluster-wide configuration values.")
flag.StringVar(&cozyValuesNamespaceSelector,"cozy-values-namespace-selector","cozystack.io/system=true","The label selector for namespaces where the cluster-wide configuration values must be replicated.")
This file contains detailed instructions for AI-powered IDE on how to generate changelogs for Cozystack releases.
## When to use these instructions
Follow these instructions when the user explicitly asks to generate a changelog.
## Required Tools
Before generating changelogs, ensure you have access to `gh` (GitHub CLI) tool, which is used to fetch commit and PR author information. The GitHub CLI is used to correctly identify PR authors from commits and pull requests.
## Changelog Generation Process
When the user asks to generate a changelog, follow these steps in the specified order:
**CHECKLIST - All actions that must be completed:**
- [ ] Step 1: Update information from remote (git fetch)
- [ ] Step 2: Check current branch (must be main)
- [ ] Step 3: Determine release type and previous version (minor vs patch release)
- [ ]**MANDATORY**: For EVERY PR in main repo, get PR author via `gh pr view <PR_NUMBER> --json author --jq .author.login` (do NOT skip this step)
- [ ]**MANDATORY**: Extract PR numbers from commit messages, then use `gh pr view` for each PR to get the PR author. Do NOT use commit author. Only for commits without PR numbers (rare), fall back to `gh api repos/cozystack/cozystack/commits/<hash> --jq '.author.login'`
- [ ] Step 8: Form new changelog (structure, format, generate contributors list)
- [ ] Step 9: Verify completeness and save
### 1. Updating information from remote
```bash
git fetch --tags --force --prune
```
This is necessary to get up-to-date information about tags and commits from the remote repository.
### 2. Checking current branch
Make sure we are on the `main` branch:
```bash
git branch --show-current
```
### 3. Determining release type and previous version
**Important**: Determine if you're generating a changelog for a **minor release** (vX.Y.0) or a **patch release** (vX.Y.Z where Z > 0).
**For minor releases (vX.Y.0):**
- Each minor version lives and evolves in its own branch (`release-X.Y`)
- You MUST compare with the **previous minor version** (v(X-1).Y.0), not the last patch release
- This ensures you capture all changes from the entire minor version cycle, including all patch releases
- Example: For v0.38.0, compare with v0.37.0 (not v0.37.8)
- Run a separate cycle to check the diff with the zero version of the previous minor release
**For patch releases (vX.Y.Z where Z > 0):**
- Compare with the previous patch version (vX.Y.(Z-1))
- Example: For v0.37.2, compare with v0.37.1
### 4. Determining versions and analyzing existing changelogs
**Determine the last published version:**
1. Get the list of version tags:
```bash
git tag -l 'v[0-9]*.[0-9]*.[0-9]*' | sort -V
```
2. Get the last tag:
```bash
git tag -l 'v[0-9]*.[0-9]*.[0-9]*' | sort -V | tail -1
```
3. Compare tags with existing changelog files in `docs/changelogs/` to determine the last published version (the newest file `vX.Y.Z.md`)
**Study existing changelog format:**
- Review recent changelog files to understand the format and structure
- Pay attention to:
- **Feature Highlights format** (for minor releases): Use `## Feature Highlights` with `### Feature Name` subsections containing detailed descriptions (2-4 paragraphs each). See v0.35.0 and v0.36.0 for examples.
- Section structure (Major Features and Improvements, Security, Fixes, Dependencies, etc.)
- PR link format (e.g., `[**@username**](https://github.com/username) in #1234`)
- Change description style
- Presence of Breaking changes sections, etc.
### 5. Getting the list of commits
**Important**: Determine if you're generating a changelog for a **minor release** (vX.Y.0) or a **patch release** (vX.Y.Z where Z > 0).
**For patch releases (vX.Y.Z where Z > 0):**
Get the list of commits starting from the previous patch version to HEAD:
**⚠️ CRITICAL: Do NOT use --first-parent flag! It will skip merge commits including backports!**
```bash
# Get all commits including merge commits (backports)
This will include all commits from v0.37.1, v0.37.2, v0.37.3, etc., up to v0.38.0.
**⚠️ IMPORTANT: Always check merge commits:**
- Merge commits may contain backports that need to be included
- Check all commits in the range, including merge commits
- For backports, always find and reference the original PR
### 6. Analyzing additional repositories
**⚠️ CRITICAL: This step is MANDATORY and must NOT be skipped!**
Cozystack release may include changes from related repositories. Check and include commits from these repositories if tags were released during the release period:
- **MANDATORY**: Always check this repository for documentation changes during the release period
- **MANDATORY**: Get GitHub username for EVERY commit. Extract PR number from commit message, then use `gh pr view <PR_NUMBER> --repo cozystack/website --json author --jq .author.login` to get PR author. Only if no PR number, fall back to `gh api repos/cozystack/website/commits/<hash> --jq '.author.login'`
**Optional repositories (MUST check ALL of them for tags during release period):**
**⚠️ IMPORTANT**: You MUST check ALL optional repositories for tags created during the release period. Do NOT skip this step even if you think there might not be any tags. Use the process below to verify.
- Check ALL optional repositories (talm, boot-to-talos, cozyhr, cozy-proxy)
- Look for tags created during the release period
- Get commits between tags (if tags exist) or by date range (if no tags)
- Extract PR numbers from commit messages
- For EVERY commit with PR number, get PR author via CLI: `gh pr view <PR_NUMBER> --repo cozystack/<repo> --json author --jq .author.login` (ALWAYS use PR author, not commit author)
- For commits without PR numbers (rare), fallback to: `gh api repos/cozystack/<repo>/commits/<hash> --jq '.author.login'`
- Output results in format: `commit_hash|subject|author_name|github_username|cozystack/repo#PR_NUMBER` or `cozystack/repo@commit_hash`
4. **Extract PR numbers and authors using GitHub CLI:**
- **ALWAYS use PR author, not commit author** for commits from additional repositories
- For each commit, extract PR number from commit message first: Extract `#123` pattern from commit message
- If PR number found, use `gh pr view <PR_NUMBER> --repo cozystack/<repo> --json author --jq .author.login` to get PR author (the person who wrote the code)
- Only if no PR number found (rare), fallback to commit author: `gh api repos/cozystack/<repo>/commits/<hash> --jq '.author.login'`
- **Prefer PR numbers**: Use format `cozystack/website#123` if PR number found in commit message
- **Fallback to commit hash**: Use format `cozystack/website@abc1234` if no PR number
- **ALWAYS include author**: Every entry from additional repositories MUST include author in format `([**@username**](https://github.com/username) in cozystack/repo#123)`
- Determine user impact and categorize appropriately
- Format entries with repository prefix: `[website]`, `[talm]`, etc.
**Example entry format for additional repositories:**
```markdown
# If PR number found in commit message (REQUIRED format):
* **[website] Update installation documentation**: Improved installation guide with new examples ([**@username**](https://github.com/username) in cozystack/website#123).
# If no PR number (fallback, use commit hash):
* **[website] Update installation documentation**: Improved installation guide with new examples ([**@username**](https://github.com/username) in cozystack/website@abc1234).
# For optional repositories:
* **[talm] Add new feature**: Description of the change ([**@username**](https://github.com/username) in cozystack/talm#456).
```
**CRITICAL**:
- **ALWAYS include author** for every entry from additional repositories
- **ALWAYS include PR link or commit hash** for every entry
- Never add entries without author and PR/commit reference
- **ALWAYS use PR author, not commit author**: Extract PR number from commit message, then use `gh pr view <PR_NUMBER> --repo cozystack/<repo> --json author --jq .author.login` to get the PR author (the person who wrote the code)
- Only if no PR number found (rare), fallback to commit author: `gh api repos/cozystack/<repo>/commits/<hash> --jq '.author.login'`
- The commit author (especially for squash/merge commits) is usually the person who merged the PR, not the person who wrote the code
### 7. Analyzing commits and PRs
**⚠️ CRITICAL: You MUST get the author from PR, not from commit! Always use `gh pr view` to get the PR author. Do NOT use commit author!**
**Get all PR numbers from commits:**
**⚠️ CRITICAL: Do NOT use --no-merges flag! It will skip merge commits including backports!**
```bash
# Extract all PR numbers from commit messages in the release range (including merge commits)
- Backport PRs have format: `[Backport release-X.Y] <original title> (#BACKPORT_PR_NUMBER)`
- The backport commit message or PR description usually mentions the original PR number
- For backport entries in changelog, use the original PR author (not the backport PR author)
- Include both original and backport PR numbers in the changelog entry (e.g., `#1606, #1609`)
- To find original PR from backport: Check the backport PR description or commit message for "Backport of #ORIGINAL_PR"
**For each PR number, get the author:**
**CRITICAL**: The commit author (especially for squash/merge commits) is usually the person who merged the PR (or GitHub bot), NOT the person who wrote the code. **ALWAYS use the PR author**, not the commit author.
**⚠️ MANDATORY: ALWAYS use `gh pr view` to get the PR author. Do NOT use commit author!**
**ALWAYS use GitHub CLI** to get the PR author:
```bash
# Usage: Get PR author - MANDATORY for EVERY PR
# Loop through ALL PR numbers and get PR author (including backports)
git log <previous_version>..<new_version> --format="%s%n%b" | grep -oE '#[0-9]+' | sort -u | tr -d '#' | while read PR_NUMBER; do
# Check if this is a backport PR
BACKPORT_INFO=$(gh pr view "$PR_NUMBER" --json body --jq '.body' 2>/dev/null | grep -i "backport of #" || echo "")
if [ -n "$BACKPORT_INFO" ]; then
# Extract original PR number from backport description
ORIGINAL_PR=$(echo "$BACKPORT_INFO" | grep -oE 'backport of #([0-9]+)' | grep -oE '[0-9]+' | head -1)
**⚠️ IMPORTANT**: You must run this for EVERY PR in the release period. Do NOT skip any PRs or assume the GitHub username based on the git author name.
**CRITICAL**: Always use `gh pr view <PR_NUMBER> --json author --jq .author.login` to get the PR author. This correctly identifies the person who wrote the code, not the person who merged it (which is especially important for squash merges).
**Why this matters**: Using the wrong author in changelogs gives incorrect credit and can confuse contributors. The merge/squash commit is created by the person who clicks "Merge" in GitHub, not the PR author.
**For commits without PR numbers (rare):**
- Only if a commit has no PR number, fall back to commit author: `gh api repos/cozystack/cozystack/commits/<hash> --jq '.author.login'`
- But this should be very rare - most commits should have PR numbers
**Extract PR number from commit messages:**
- Check commit message subject (`%s`) and body (`%b`) for PR references: `#1234` or `(#1234)`
- **Primary method**: Extract from commit message format `(#PR_NUMBER)` or `in #PR_NUMBER` or `Merge pull request #1234`
- Use regex: `grep -oE '#[0-9]+'` to find all PR numbers
**⚠️ CRITICAL: Verify PR numbers match commit messages!**
- Always verify that the PR number in the changelog matches the PR number in the commit message
- Common mistake: Using wrong PR number (e.g., #1614 instead of #1617) when multiple similar commits exist
- To verify: Check the actual commit message: `git log <commit_hash> -1 --format="%s%n%b" | grep -oE '#[0-9]+'`
- If multiple PR numbers appear in a commit, use the one that matches the PR title/description
- For merge commits, check the merged branch commits, not just the merge commit message
3. **Understand the change:**
```bash
# Get PR details (preferred method)
gh pr view <PR_NUMBER> --json title,body,url
# Or get commit details if no PR number
git show <commit_hash> --stat
git show <commit_hash>
```
- Review PR description and changed files
- Understand functionality added/changed/fixed
- **Determine user impact**: What can users do now? What problems are fixed? What improvements do users experience?
4. **For release branches (backports):**
- If commit is from `release-X.Y` branch, check if it's a backport
- Find original commit in `main` to get correct PR number:
```bash
git log origin/main --grep="<part of commit message>" --oneline
```
### 8. Forming a new changelog
Create a new changelog file in the format matching previous versions:
1. **Determine the release type:**
- **Minor release (vX.Y.0)** - use full format with **Feature Highlights** section. **Must include all changes from patch releases of the previous minor version** (e.g., v0.38.0 should include changes from v0.37.1, v0.37.2, v0.37.3, etc.)
- **Patch release (vX.Y.Z, where Z > 0)** - use more compact format, includes only changes since the previous patch release
**Feature Highlights format for minor releases:**
- Use section header: `## Feature Highlights`
- Include 3-6 major features as subsections with `### Feature Name` headers
- **Development, Testing, and CI/CD** - development and testing improvements
- **Documentation** (include changes from website repository here - **MUST include authors and PR links for all entries**)
- **Migration and Upgrades** (if applicable)
**Note**: When including changes from additional repositories, group them logically with main repository changes, or create separate subsections if there are many changes from a specific repository.
3. **Entry format:**
- Use the format: `* **Brief description**: detailed description ([**@username**](https://github.com/username) in #PR_NUMBER)`
- **CRITICAL - Get authorship correctly**:
- **ALWAYS use PR author, not commit author**: Extract PR number from commit message, then use `gh pr view` to get the PR author. The commit author (especially for squash/merge commits) is usually the person who merged the PR (or GitHub bot), NOT the person who wrote the code.
# Only fallback to commit author if no PR number found (rare)
GITHUB_USERNAME=$(gh api repos/cozystack/cozystack/commits/<commit_hash> --jq '.author.login')
fi
```
**Example**: For PR #1507, the squash commit has author "kvaps" (who merged), but the PR author is "lllamnyp" (who wrote the code). Using `gh pr view 1507 --json author --jq .author.login` correctly returns "lllamnyp".
- **For regular commits**: Use the commit author directly:
```bash
git log <commit_hash> -1 --format="%an|%ae"
```
- **Validation**: Before adding to changelog, verify the author by checking:
- For merge commits: Compare merge commit author vs PR author (they should be different)
- Check existing changelogs for author name to GitHub username mappings
Example: For v0.38.0, use `v0.37.0...v0.38.0` (NOT `v0.37.8...v0.38.0`)
**Important**: Minor releases must reference the previous minor release (vX.Y.0), not the last patch release, to include all changes from the entire minor version cycle.
5. **Generate contributors list:**
**⚠️ SIMPLIFIED APPROACH: Extract contributors from the generated changelog itself!**
Since you've already generated the changelog with all PR authors correctly identified, simply extract GitHub usernames from the changelog entries:
```bash
# Extract all GitHub usernames from the current release changelog
# This method is simpler and more reliable than extracting from git history
# For patch releases: extract from the current changelog file
- [ ] Step 5 completed: **ALL commits included** (including merge commits and backports) - do not skip any commits
- [ ] Step 5 completed: **Backports identified and handled correctly** - original PR author used, both original and backport PR numbers included
- [ ] Step 6 completed: Website repository checked for documentation changes WITH authors and PR links via GitHub CLI
- [ ] Step 6 completed: **ALL** optional repositories (talm, boot-to-talos, cozyhr, cozy-proxy) checked for tags during release period
- [ ] Step 6 completed: For ALL commits from additional repos, GitHub username obtained via GitHub CLI (not skipped). For commits with PR numbers, PR author used via `gh pr view` (not commit author)
- [ ] Step 7 completed: For EVERY PR in main repo (including backports), PR author obtained via `gh pr view <PR_NUMBER> --json author --jq .author.login` (not skipped or assumed). Commit author NOT used - always use PR author
- [ ] Step 7 completed: **Backports verified** - for each backport PR, original PR found and original PR author used in changelog
- [ ] Step 8 completed: Contributors list generated
- [ ] All commits from main repository included (including merge commits)
- [ ] User impact described for each change
- [ ] Format matches existing changelogs
**For patch releases:**
- [ ] All commits from the release period are included (including merge commits with backports)
- [ ] PR numbers match commit messages
- [ ] Backports are properly identified and linked to original PRs
**For minor releases (vX.Y.0):**
- [ ] All changes from patch releases (vX.Y.1, vX.Y.2, etc.) are included
- [ ] Contributors section is present and complete
- [ ] Full Changelog link references previous minor version (vX.Y.0), not last patch
- [ ] Verify all PRs from patch releases are included:
# For each missing PR, check if it's a backport and verify change is included by description
fi
```
**Only proceed to save after all checkboxes are verified!**
**Save the changelog:**
Save the changelog to file `docs/changelogs/v<version>.md` according to the version for which the changelog is being generated.
### Important notes
- **After fetch with --force** local tags are up-to-date, use them for work
- **For release branches** always check original commits in `main` to get correct PR numbers
- **Preserve the format** of existing changelog files
- **Group related changes** logically
- **Be accurate** in describing changes, based on actual commit diffs
- **Check for PR numbers** and commit authors
- **CRITICAL - Get authorship from PR, not from commit**:
- **ALWAYS use PR author**: Extract PR number from commit message, then use `gh pr view <PR_NUMBER> --json author --jq .author.login` to get the PR author
- Do NOT use commit author - the commit author (especially for squash/merge commits) is usually the person who merged the PR, not the person who wrote the code
- For commits without PR numbers (rare), fall back to commit author: `gh api repos/cozystack/cozystack/commits/<commit_hash> --jq '.author.login'`
- **Workflow**: Extract PR numbers from commits → Use `gh pr view` for each PR → Get PR author (the person who wrote the code)
- Example: For PR #1507, the commit author is `@kvaps` (who merged), but `gh pr view 1507 --json author --jq .author.login` correctly returns `@lllamnyp` (who wrote the code)
- Check existing changelogs for author name to GitHub username mappings
- **Validation**: Before adding to changelog, always verify the author using `gh pr view` - never use commit author for PRs
- **MANDATORY**: Always describe user impact: Every changelog entry must explain how the change affects end users, not just what was changed technically. Focus on user value and practical implications.
- **⚠️ CRITICAL**: Always check the **website** repository for documentation changes during the release period. This is a required step and MUST NOT be skipped.
- **⚠️ CRITICAL**: You MUST check ALL optional repositories (talm, boot-to-talos, cozyhr, cozy-proxy) for tags during the release period. Do NOT skip any repository even if you think there might not be tags.
- **CRITICAL**: For ALL entries from additional repositories (website and optional), you MUST:
- **MANDATORY**: Extract PR number from commit message first
- **MANDATORY**: For commits with PR numbers, ALWAYS use `gh pr view <PR_NUMBER> --repo cozystack/<repo> --json author --jq .author.login` to get PR author (not commit author)
- **MANDATORY**: Only for commits without PR numbers (rare), fallback to: `gh api repos/cozystack/<repo>/commits/<hash> --jq '.author.login'`
- **MANDATORY**: Do NOT skip getting GitHub username via CLI - do this for EVERY commit
- **MANDATORY**: Do NOT use commit author for PRs - always use PR author
- Include PR link or commit hash reference
- Format: `* **[repo] Description**: details ([**@username**](https://github.com/username) in cozystack/repo#123)`
- For **optional repositories** (talm, boot-to-talos, cozyhr, cozy-proxy), you MUST check ALL of them for tags during the release period. Use the loop provided in Step 6 to check each repository systematically.
- When including changes from additional repositories, use the format: `[repo-name] Description` and link to the repository's PR/issue if available
- **Prefer PR numbers over commit hashes**: For commits from additional repositories, extract PR number from commit message using GitHub API. Use PR format (`cozystack/website#123`) instead of commit hash (`cozystack/website@abc1234`) when available
- **Never add entries without author and PR/commit reference**: Every entry from additional repositories must have both author and link
- Group changes from additional repositories with main repository changes, or create separate subsections if there are many changes from a specific repository
- **⚠️ CRITICAL**: You MUST get the author from PR using `gh pr view`, NOT from commit
- **⚠️ CRITICAL**: Extract PR numbers from commit messages, then use `gh pr view <PR_NUMBER> --json author --jq .author.login` for each PR
- **⚠️ CRITICAL**: Do NOT use commit author - commit author is usually the person who merged, not the person who wrote the code
- **⚠️ CRITICAL**: Do NOT skip this step for any PR, even if the author seems obvious
- For commits without PR numbers (rare), fall back to: `gh api repos/cozystack/cozystack/commits/<hash> --jq '.author.login'`
- This ensures correct attribution and prevents errors in changelog entries (especially important for squash/merge commits)
- **Contributors list (Step 8)**:
- For minor releases (vX.Y.0): You must generate a list of all contributors and identify first-time contributors.
- For patch releases: Contributors section is optional, but recommended for significant releases
- Extract GitHub usernames from PR links in commit messages or changelog entries
- This helps recognize community contributions and welcome new contributors
- **Minor releases (vX.Y.0)**:
- Must include **all changes** from patch releases of the previous minor version (e.g., v0.38.0 includes all changes from v0.37.1, v0.37.2, v0.37.3, etc.)
- The "Full Changelog" link must reference the previous minor release (v0.37.0...v0.38.0), NOT the last patch release (v0.37.8...v0.38.0)
- This ensures users can see the complete set of changes for the entire minor version cycle
- **Verification step**: After creating the changelog, extract all PR numbers from patch release changelogs and verify they all appear in the minor release changelog to prevent missing entries
- **Backport handling**: Patch releases may contain backports with different PR numbers (e.g., #1624 in patch release vs #1622 in main). For minor releases, use original PR numbers from main when available, but verify that all changes from patch releases are included regardless of PR number differences
- **Content verification**: Don't rely solely on PR number matching - verify that change descriptions from patch releases appear in the minor release changelog, as backports may have different PR numbers
When the user asks to fix comments from AI bot reviewers (like Qodo, Copilot, etc.):
### 1. Get PR Comments
View all comments on the pull request:
```bash
gh pr view <PR-number> --comments
```
Or for the current branch:
```bash
gh pr view --comments
```
### 2. Review Each Comment Carefully
**Important**: Do NOT blindly apply all suggestions. Each comment should be evaluated:
- **Consider context** - Does the suggestion make sense for this specific case?
- **Check project conventions** - Does it align with Cozystack patterns?
- **Evaluate impact** - Will this improve code quality or introduce issues?
- **Question validity** - AI bots can be wrong or miss context
**When to apply:**
- ✅ Legitimate bugs or security issues
- ✅ Clear improvements to code quality
- ✅ Better error handling or edge cases
- ✅ Conformance to project conventions
**When to skip:**
- ❌ Stylistic preferences that don't match project style
- ❌ Over-engineering simple code
- ❌ Changes that break existing patterns
- ❌ Suggestions that show misunderstanding of the code
### 3. Apply Valid Fixes
Make changes addressing the valid comments. Use your judgment.
### 4. Leave Changes Uncommitted
**Critical**: Do NOT commit or push the changes automatically.
Leave the changes in the working directory so the user can:
- Review the fixes
- Decide whether to commit them
- Make additional adjustments if needed
```bash
# After making changes, show status but DON'T commit
git status
git diff
```
The user will commit and push when ready.
## Code Review Comments
When asked to fix code review comments, **always work only with unresolved (open) comments**. Resolved comments should be ignored as they have already been addressed.
### Getting Unresolved Review Comments
Use GitHub GraphQL API to fetch only unresolved review comments from a pull request:
The key filter is `select(.isResolved == false)` which ensures only unresolved review threads are processed. Each thread can contain multiple comments, but if the thread is resolved, all its comments should be ignored.
### Working with Review Comments
1.**Fetch unresolved comments** using the GraphQL query above
2.**Parse the results** to identify:
- File path (`path`)
- Line number (`line` or `originalLine`)
- Comment text (`bodyText`)
- Author (`author.login`)
3.**Address each unresolved comment** by:
- Locating the relevant code section
- Making the requested changes
- Ensuring the fix addresses the concern raised
4.**Do NOT process resolved comments** - they have already been handled
- **REST API limitation**: The REST endpoint `/pulls/{pr}/reviews` returns review summaries, not individual review comments. Use GraphQL API for accessing `reviewThreads` with `isResolved` status.
- **Thread-based resolution**: Comments are organized in threads. If a thread is resolved (`isResolved: true`), ignore all comments in that thread.
- **Always filter**: Never process comments from resolved threads, even if they appear in the results.
### Example Workflow
```bash
# Get PR comments
gh pr view 1234 --comments
# Review comments and identify valid ones
# Make necessary changes to address valid comments
# ... edit files ...
# Show what was changed (but don't commit)
git status
git diff
# Tell the user what was fixed and what was skipped
```
## Git Permissions
Request these permissions when needed:
-`git_write` - For commit, rebase, cherry-pick, branch operations
-`network` - For push, fetch, pull operations
## Common Issues
**PR has extra commits?**
→ Rebase on `upstream/main` and cherry-pick only your commits
* [lineage] Break webhook out into a separate daemonset. Reduce unnecessary webhook calls by marking handled resources and excluding them from consideration by the webhook's object selector (@lllamnyp in #1515).
* **[api] Efficient listing of TenantNamespaces**: Optimized TenantNamespace listing by replacing per-namespace SubjectAccessReview calls with group-based rolebinding checks, significantly reducing API latency and improving performance ([**@lllamnyp**](https://github.com/lllamnyp) in #1507).
## Fixes
* **[api] Fix RBAC for listing of TenantNamespaces and handle system:masters**: Fixed regression in TenantNamespace listing RBAC and added proper handling for system:masters group to ensure correct authorization ([**@kvaps**](https://github.com/kvaps) in #1511).
* **[dashboard] Fix logout**: Fixed dashboard logout functionality to properly clear session and redirect users ([**@kvaps**](https://github.com/kvaps) in #1510).
* **[installer] Add additional check to wait for lineage-webhook**: Added additional readiness check to ensure lineage-webhook is fully ready before proceeding with installation, improving upgrade reliability ([**@kvaps**](https://github.com/kvaps) in #1506).
## Development, Testing, and CI/CD
* **[tests] Make Kubernetes tests POSIX-compatible**: Replaced bash-specific constructs with POSIX-compliant code, ensuring tests work reliably with /bin/sh and improving compatibility across different shell environments ([**@IvanHunters**](https://github.com/IvanHunters) in #1509).
## Documentation
* **[website] Update troubleshooting documentation**: Updated Kubernetes installation troubleshooting guide with additional information and fixes ([**@lb0o**](https://github.com/lb0o) in cozystack/website@82beddd).
* **[website] Add LLDPD disabling documentation**: Added minimal patch documentation for disabling lldpd based on official LLDPD usage guide ([**@lb0o**](https://github.com/lb0o) in cozystack/website@7ec5d7b).
* **[website] Fix typo in utility command**: Fixed typo in utility command documentation ([**@lb0o**](https://github.com/lb0o) in cozystack/website@6c76cb5).
* **[website] Update backup and recovery docs**: Updated backup and recovery documentation with latest information ([**@kvaps**](https://github.com/kvaps) in cozystack/website@2781aa5).
* **[website] Add Troubleshooting checklist**: Added troubleshooting checklist to help users diagnose and resolve common issues ([**@kvaps**](https://github.com/kvaps) in cozystack/website@59fc304).
* **[virtual-machine] Improve check for resizing job**: Improved storage resize logic to only expand persistent volume claims when storage is being increased, preventing unintended storage reduction operations. Added validation to accurately compare current and desired storage sizes before triggering resize operations ([**@kvaps**](https://github.com/kvaps) in #1688, #1702).
## Fixes
* **[dashboard] Fix CustomFormsOverride schema to nest properties under spec.properties**: Fixed the logic for generating CustomFormsOverride schema to properly nest properties under `spec.properties` instead of directly under `properties`, ensuring correct form schema generation in the dashboard ([**@kvaps**](https://github.com/kvaps) in #1692, #1699).
* **[lineage] Separate webhook from cozy controller**: Separated the lineage-controller-webhook from cozystack-controller into a separate daemonset component deployed on all control-plane nodes, reducing API server latency and improving performance by decreasing outgoing API calls. Introduced internal label to track resources already handled by the webhook ([**@lllamnyp**](https://github.com/lllamnyp) in #1515).
## Fixes
* **[api] Fix listing tenantnamespaces for non-oidc users**: Fixed TenantNamespace listing functionality for users not using OIDC authentication, ensuring proper namespace visibility for all authentication methods ([**@kvaps**](https://github.com/kvaps) in #1517, #1519).
## Migration and Upgrades
* **[platform] Better migration for 0.36.2->0.37.2+**: Improved migration script for users upgrading directly from 0.36.2 to 0.37.2+, ensuring the new lineage webhook daemonset is properly deployed and fixing a bug where webhook readiness was not appropriately verified during migration ([**@lllamnyp**](https://github.com/lllamnyp) in #1521, #1522).
* **[apps] Make VM service user facing**: Virtual machine services are now marked as user-facing, improving service discovery and visibility in the dashboard ([**@lllamnyp**](https://github.com/lllamnyp) in #1523).
* **[seaweedfs] Allow users to discover their buckets**: Users can now discover and list their S3 buckets in SeaweedFS, improving usability and bucket management ([**@kvaps**](https://github.com/kvaps) in #1528).
* **[seaweedfs] Update SeaweedFS v3.99 and deploy S3 as stacked service**: Updated SeaweedFS to version 3.99 and deployed S3 gateway as a stacked service for better integration and performance ([**@kvaps**](https://github.com/kvaps) in #1562).
* **[dashboard] Show service LB IP**: Fixed JSON path issue to correctly display Service LoadBalancer IPs in the dashboard table view, improving visibility of service endpoints ([**@lllamnyp**](https://github.com/lllamnyp) in #1524).
* **[dashboard] Update openapi-ui v1.0.3 + fixes**: Updated OpenAPI UI to version 1.0.3 with various fixes and improvements ([**@kvaps**](https://github.com/kvaps) in #1564).
* **[kubernetes] Use controlPlane.replicas field**: Fixed managed Kubernetes app to properly use the `controlPlane.replicas` field instead of hardcoding the value, allowing users to configure control plane replica count ([**@lllamnyp**](https://github.com/lllamnyp) in #1556).
* **[monitoring] add settings alert for slack**: Added Slack integration configuration for Alerta alerts, enabling notifications to Slack channels ([**@scooby87**](https://github.com/scooby87) in #1545).
## Fixes
* **[lineage] Check for nil chart in HelmRelease**: Added nil check to prevent crashes when lineage webhook encounters HelmReleases using `chartRef` instead of `chart`, improving stability ([**@lllamnyp**](https://github.com/lllamnyp) in #1525).
* **[kamaji] Respect 3rd party labels**: Applied patch to Kamaji controller to respect third-party labels, preventing reconciliation loops between lineage webhook and Kamaji controller ([**@lllamnyp**](https://github.com/lllamnyp) in #1531, #1534).
* **[redis-operator] Build patched operator in-tree**: Moved Redis operator build into Cozystack organization and patched it to prevent overwriting third-party labels on owned resources ([**@lllamnyp**](https://github.com/lllamnyp) in #1547).
* **[mariadb-operator] Add post-delete job to remove PVCs**: Added post-delete job to automatically remove PersistentVolumeClaims when MariaDB instances are deleted, preventing orphaned storage resources ([**@IvanHunters**](https://github.com/IvanHunters) in #1553).
* **[velero] Set defaultItemOperationTimeout=24h**: Set default item operation timeout to 24 hours for Velero backups, preventing timeouts on large backup operations ([**@kvaps**](https://github.com/kvaps) in #1542).
## Dependencies
* **Update LINSTOR v1.32.3**: Updated LINSTOR to version 1.32.3 with latest features and bug fixes ([**@kvaps**](https://github.com/kvaps) in #1565).
## System Configuration
* **[system] kube-ovn: turn off enableLb**: Disabled load balancer functionality in Kube-OVN configuration ([**@nbykov0**](https://github.com/nbykov0) in #1548).
## Documentation
* **[website] Update LINSTOR documentation**: Updated LINSTOR guide and set failmode=continue for ZFS configurations ([**@kvaps**](https://github.com/kvaps) in cozystack/website@033804e).
* **[website] Update golden image documentation**: Updated documentation for creating golden images for virtual machines ([**@kvaps**](https://github.com/kvaps) in cozystack/website@34c2f3a, cozystack/website@ef65593).
* **[website] Fix documentation formatting**: Fixed alerts, infoboxes, tabs styles and main page formatting ([**@kvaps**](https://github.com/kvaps) in cozystack/website@e992e97, cozystack/website@b2c4dee).
* **[website] Fix typo in blog article**: Fixed typo in blog article ([**@kvaps**](https://github.com/kvaps) in cozystack/website@0a4bbf3).
* **[tenant] Allow listing workloads**: Enabled listing of workloads for tenants, improving visibility and management of tenant resources ([**@kvaps**](https://github.com/kvaps) in #1576, #1577).
## Fixes
* **[seaweedfs] Fix migration to v3.99**: Fixed migration issues when upgrading SeaweedFS to version 3.99, ensuring smooth upgrades ([**@kvaps**](https://github.com/kvaps) in #1572, #1575).
* **[nats] Merge container spec, not podTemplate**: Fixed NATS configuration to properly merge container specifications instead of podTemplate, ensuring correct container configuration ([**@lllamnyp**](https://github.com/lllamnyp) in #1571, #1574).
## Development, Testing, and CI/CD
* **[e2e] Increase Kubernetes connection timeouts**: Increased connection and request timeouts in E2E tests when communicating with Kubernetes API, improving test stability under high load and slow cluster response conditions ([**@IvanHunters**](https://github.com/IvanHunters) in #1570, #1573).
## Documentation
* **[website] Optimize website for mobile devices**: Improved website layout and responsiveness for mobile devices ([**@kvaps**](https://github.com/kvaps) in cozystack/website@3ab2338).
* **[website] Add OpenAPI UI**: Added OpenAPI UI documentation and integration ([**@kvaps**](https://github.com/kvaps) in cozystack/website@b1c1668).
* **[website] Update Cozystack video in hero banner**: Updated hero banner with new Cozystack video ([**@kvaps**](https://github.com/kvaps) in cozystack/website@e351137).
* **[website] Add screenshots carousel**: Added screenshots carousel to showcase Cozystack features ([**@kvaps**](https://github.com/kvaps) in cozystack/website@8422bd0).
* **[dashboard-controller] Move badges generation logic to internal dashboard component**: Moved badges generation logic to internal dashboard component for better code organization and maintainability ([**@kvaps**](https://github.com/kvaps) in #1567).
## Security
* **[redis] Bump Redis image version for security fixes**: Updated Redis image version to include latest security fixes, improving cluster security ([**@IvanHunters**](https://github.com/IvanHunters) in #1580).
* **[flux] Close Flux Operator ports to external access**: Removed hostPort and hostNetwork from Flux Operator Deployment, ensuring ports 8080 and 8081 are only accessible within the cluster, preventing external exposure and improving security ([**@IvanHunters**](https://github.com/IvanHunters) in #1581).
* **[ingress] Enforce HTTPS-only for API**: Added force-ssl-redirect annotation to default API Ingress, ensuring all HTTP traffic is redirected to HTTPS, preventing unencrypted external access and improving security ([**@IvanHunters**](https://github.com/IvanHunters) in #1582, #1585).
## Fixes
* **[nats] Fixes for NATS App Helm chart, fix template issues with config.merge**: Fixed template issues in NATS Helm chart related to config.merge value, ensuring correct configuration ([**@insignia96**](https://github.com/insignia96) in #1583, #1591).
* **[api] Use shared informer cache**: Optimized API server by using shared informer cache, reducing API server load and improving performance ([**@lllamnyp**](https://github.com/lllamnyp) in #1539).
* **[dashboard] sync with upstream & enhancements**: Synchronized dashboard with upstream and added various enhancements ([**@kvaps**](https://github.com/kvaps) in #1603).
* **[cozystack-api][dashboard] Fix filtering for application services/ingresses/secrets**: Fixed filtering functionality for application services, ingresses, and secrets in both API and dashboard ([**@kvaps**](https://github.com/kvaps) in #1612).
## Fixes
* **[controller] Remove crdmem, handle DaemonSet**: Removed crdmem and improved DaemonSet handling in controller ([**@lllamnyp**](https://github.com/lllamnyp) in #1555).
* **[dashboard] Revert reconciler removal**: Reverted reconciler removal to restore proper dashboard functionality ([**@lllamnyp**](https://github.com/lllamnyp) in #1559).
* **[dashboard-controller] Fix static resources reconciliation and showing secrets**: Fixed static resources reconciliation and improved secret display in dashboard controller ([**@kvaps**](https://github.com/kvaps) in #1605).
* **[api,lineage] Ensure node-local traffic**: Ensured node-local traffic handling for API and lineage components ([**@lllamnyp**](https://github.com/lllamnyp) in #1606).
* **[virtual-machine] Revert per-vm network policies**: Reverted per-VM network policies to previous behavior ([**@lllamnyp**](https://github.com/lllamnyp) in #1611).
* **[cozy-lib] Fix: handling resources=nil**: Fixed handling of nil resources in cozy-lib templates ([**@kvaps**](https://github.com/kvaps) in #1607).
* **[nats] Use dig function to check for existing secret and prevent nil indexing**: Fixed NATS app chart to use dig function for checking existing secrets and prevent nil indexing errors ([**@kvaps**](https://github.com/kvaps) in #1609, #1610).
## Development, Testing, and CI/CD
* **[cozystack-controller] improve API tests**: Improved API tests for cozystack-controller ([**@lllamnyp**](https://github.com/lllamnyp) in #1599).
* **[kubernetes] Helm hooks for cleanup**: Added Helm hooks for cleanup operations in Kubernetes app ([**@lllamnyp**](https://github.com/lllamnyp) in #1616).
* **[kubernetes] Cleanup loadbalancer services**: Added cleanup functionality for load balancer services in Kubernetes app ([**@lllamnyp**](https://github.com/lllamnyp) in #1622).
* **[rbac] Fix permissions for high-privilege users**: Fixed RBAC permissions for high-privilege users, ensuring proper access control ([**@lllamnyp**](https://github.com/lllamnyp) in #1624).
## System Configuration
* **[system] kubeovn: increase limits**: Increased resource limits for Kube-OVN components to improve stability and performance ([**@nbykov0**](https://github.com/nbykov0) in #1629).
* **[cozy-lib] Fix malformed ResourceQuota rendering for LoadBalancer services**: Fixed malformed ResourceQuota rendering for LoadBalancer services in cozy-lib templates ([**@IvanHunters**](https://github.com/IvanHunters) in #1642).
* **[extra] ingress: rm spaces from external ip list**: Removed spaces from external IP list in ingress configuration, fixing formatting issues ([**@nbykov0**](https://github.com/nbykov0) in #1652).
* **scripts: fix 20 migration**: Fixed migration script #20 to ensure proper execution during upgrades ([**@nbykov0**](https://github.com/nbykov0) in #1653).
## System Configuration
* **Increase strimzi memory limit**: Increased memory limit for Strimzi Kafka operator to improve stability and performance ([**@nbykov0**](https://github.com/nbykov0) in #1651).
* **[seaweedfs] Extended CA certificate duration to reduce disruptive CA rotations**: Extended CA certificate duration to reduce disruptive CA rotations. ([**@IvanHunters**](https://github.com/IvanHunters) in #1657, #1666).
* **[dashboard] Add config hash annotations to restart pods on config changes**: Added config hash annotations to restart pods when configuration changes, ensuring pods are automatically restarted when their configuration is updated ([**@kvaps**](https://github.com/kvaps) in #1662, #1665).
## Fixes
* **[tenant][kubernetes] Introduce better cleanup logic**: Improved cleanup logic for tenant Kubernetes resources, ensuring proper resource cleanup when tenants are deleted or updated ([**@kvaps**](https://github.com/kvaps) in #1661).
* **[dashboard] Fix loading arrays in forms when editing existing objects**: Fixed issue where arrays in forms were not loading correctly when editing existing objects in the dashboard ([**@kvaps**](https://github.com/kvaps)).
This release introduces **Virtual Private Cloud (VPC)** support, enabling advanced networking capabilities for tenant applications. We've also added VNC console support in the dashboard, made Kubernetes worker versions configurable, and delivered numerous improvements and fixes across the platform.
### Virtual Private Cloud (VPC) Networking
Cozystack v0.38.0 introduces Virtual Private Cloud (VPC) support, enabling platform administrators to create isolated network segments for tenant applications. VPCs provide network isolation and allow fine-grained control over network topology, subnets, and routing. Each VPC can contain multiple subnets, and administrators can configure subnet details including IP ranges, gateway settings, and DNS configuration.
The VPC feature integrates seamlessly with the Cozystack dashboard, allowing users to view and manage VPCs and their subnets through an intuitive interface. Subnet details are exposed in the dashboard as tables, making it easy to understand network configuration at a glance. VPC configuration is stored in ConfigMaps with predictable naming, ensuring reliable access to subnet information.
This feature is particularly valuable for multi-tenant environments where network isolation is critical, and for applications that require specific network configurations or routing rules.
### VNC Console for Virtual Machines
The Cozystack dashboard now includes a built-in VNC console for virtual machines, enabling users to access VM console directly from the web interface without requiring external tools. This feature provides immediate access to virtual machine consoles for troubleshooting, configuration, and maintenance tasks. The VNC console integration streamlines VM management workflows and improves the user experience by keeping all VM operations within the Cozystack dashboard.
## Highlights
* **Virtual Private Cloud (VPC)**: New VPC system module enables advanced networking with Multus CNI, subnet management, and network isolation for tenant applications ([**@nbykov0**](https://github.com/nbykov0) in #1543; [**@lllamnyp**](https://github.com/lllamnyp) in #1587, #1590, #1600, #1621, #1638).
* **VNC Console in Dashboard**: Users can now access virtual machine consoles directly from the dashboard, improving VM management experience ([**@kvaps**](https://github.com/kvaps) in #1627).
* **Configurable Kubernetes Worker Versions**: Platform administrators can now configure Kubernetes worker node versions independently, providing more flexibility in cluster management ([**@lllamnyp**](https://github.com/lllamnyp) in #1619).
* **Security Enhancements**: Multiple security improvements including HTTPS-only enforcement for API, closed Flux Operator ports, and Redis security updates ([**@IvanHunters**](https://github.com/IvanHunters) in #1580, #1581, #1582).
* **Cozy-lib Improvements**: Enhanced flatten function with better ResourceQuota handling and nil resource support ([**@lllamnyp**](https://github.com/lllamnyp) in #1647; [**@IvanHunters**](https://github.com/IvanHunters) in #1642; [**@kvaps**](https://github.com/kvaps) in #1607).
---
## New features
### VPC (Virtual Private Cloud)
* **[system] Add VPC**: Introduced Virtual Private Cloud system module with Multus CNI integration, enabling advanced networking capabilities for tenant applications ([**@nbykov0**](https://github.com/nbykov0) in #1543).
* **[vpc] Install Multus by default**: Multus CNI is now installed by default when VPC is enabled, providing multi-network interface support ([**@lllamnyp**](https://github.com/lllamnyp) in #1587).
* **[vpc] Give predictable name to subnet configmap**: Subnet configuration maps now use predictable naming for better management and debugging ([**@lllamnyp**](https://github.com/lllamnyp) in #1590).
* **[vpc] Entry per subnet in the subnets configmap**: Each subnet now has its own entry in the subnets configmap, improving subnet organization and management ([**@lllamnyp**](https://github.com/lllamnyp) in #1600).
* **[vpc,dashboard] Print subnet details as table**: Subnet details are now displayed as a table in the dashboard, improving visibility and management ([**@lllamnyp**](https://github.com/lllamnyp) in #1621).
* **[apps] Add VPC app**: Added VPC application for tenant use, enabling users to create and manage VPCs ([**@nbykov0**](https://github.com/nbykov0) in #1543).
### Dashboard
* **[dashboard] Introduce VNC console**: Added VNC console support in the dashboard, allowing users to access virtual machine consoles directly from the web interface ([**@kvaps**](https://github.com/kvaps) in #1627).
* **[dashboard] sync with upstream & enhancements**: Synchronized dashboard with upstream project and added various enhancements ([**@kvaps**](https://github.com/kvaps) in #1603).
* **[dashboard] Migrate patches to upstream project**: Migrated dashboard patches to upstream project for better maintainability ([**@kvaps**](https://github.com/kvaps) in #1569).
### Kubernetes
* **[kubernetes] Make worker version configurable**: Platform administrators can now configure Kubernetes worker node versions independently from control plane versions, providing more flexibility ([**@lllamnyp**](https://github.com/lllamnyp) in #1619).
* **[kubernetes] Use controlPlane.replicas field**: Fixed managed Kubernetes app to properly use the `controlPlane.replicas` field instead of hardcoding the value ([**@lllamnyp**](https://github.com/lllamnyp) in #1556).
* **[kubernetes] Helm hooks for cleanup**: Added Helm hooks for cleanup operations in Kubernetes app ([**@lllamnyp**](https://github.com/lllamnyp) in #1606).
### API & Platform
* **[api] Efficient listing of TenantNamespaces**: Optimized TenantNamespace listing by replacing per-namespace SubjectAccessReview calls with group-based rolebinding checks, significantly reducing API latency ([**@lllamnyp**](https://github.com/lllamnyp) in #1507).
* **[api] Use shared informer cache**: Optimized API server by using shared informer cache, reducing API server load and improving performance ([**@lllamnyp**](https://github.com/lllamnyp) in #1539).
* **[api] Fix representation of dynamic list kinds**: Fixed API representation of dynamic list kinds for better compatibility ([**@lllamnyp**](https://github.com/lllamnyp) in #1630).
* **[api] Delete previous instance when changing type**: API now properly deletes previous instance when changing application type ([**@lllamnyp**](https://github.com/lllamnyp) in #1579).
### Applications
* **[tenant] Allow listing workloads**: Enabled listing of workloads for tenants, improving visibility and management of tenant resources ([**@kvaps**](https://github.com/kvaps) in #1576).
* **[apps] Make VM service user facing**: Virtual machine services are now marked as user-facing, improving service discovery and visibility in the dashboard ([**@lllamnyp**](https://github.com/lllamnyp) in #1523).
* **[foundationdb] Upgrade FDB app for latest Cozy**: Upgraded FoundationDB application for compatibility with latest Cozystack version ([**@lllamnyp**](https://github.com/lllamnyp) in #1505).
### Storage & Backups
* **[seaweedfs] Update SeaweedFS v3.99 and deploy S3 as stacked service**: Updated SeaweedFS to version 3.99 and deployed S3 gateway as a stacked service for better integration and performance ([**@kvaps**](https://github.com/kvaps) in #1562).
* **[seaweedfs] Allow users to discover their buckets**: Users can now discover and list their S3 buckets in SeaweedFS, improving usability and bucket management ([**@kvaps**](https://github.com/kvaps) in #1528).
* **[velero] Set defaultItemOperationTimeout=24h**: Set default item operation timeout to 24 hours for Velero backups, preventing timeouts on large backup operations ([**@kvaps**](https://github.com/kvaps) in #1542).
### Monitoring & Operations
* **[monitoring] add settings alert for slack**: Added Slack integration configuration for Alerta alerts, enabling notifications to Slack channels ([**@scooby87**](https://github.com/scooby87) in #1545).
---
## Improvements (minor)
* **[lineage] Separate webhook from cozy controller**: Separated the lineage-controller-webhook from cozystack-controller into a separate daemonset component deployed on all control-plane nodes, reducing API server latency ([**@lllamnyp**](https://github.com/lllamnyp) in #1515).
* **[dashboard] Show service LB IP**: Fixed JSON path issue to correctly display Service LoadBalancer IPs in the dashboard table view ([**@lllamnyp**](https://github.com/lllamnyp) in #1524).
* **[dashboard] Update openapi-ui v1.0.3 + fixes**: Updated OpenAPI UI to version 1.0.3 with various fixes and improvements ([**@kvaps**](https://github.com/kvaps) in #1564).
* **[dashboard-controller] Move badges generation logic to internal dashboard component**: Moved badges generation logic to internal dashboard component for better code organization ([**@kvaps**](https://github.com/kvaps) in #1567).
* **[bucket] Expose bucket name in secrets**: Bucket names are now exposed in secrets for better integration with applications ([**@lllamnyp**](https://github.com/lllamnyp) in #1518).
* **[platform] Better migration for 0.36.2->0.37.2+**: Improved migration script for users upgrading directly from 0.36.2 to 0.37.2+ ([**@lllamnyp**](https://github.com/lllamnyp) in #1521).
* **[cozy-lib] Improve flatten function**: Improved flatten function in cozy-lib with better handling of complex resource structures ([**@lllamnyp**](https://github.com/lllamnyp) in #1647).
* **[dx] JSDoc compatible syntax for values.yaml**: Added JSDoc compatible syntax for values.yaml documentation ([**@kvaps**](https://github.com/kvaps) in #1536).
* **[system] Tune kubevirt rollout and eviction settings**: Tuned KubeVirt rollout and eviction settings for better stability ([**@nbykov0**](https://github.com/nbykov0) in #1544).
* **[system] multus: update to the latest version**: Updated Multus CNI to the latest version ([**@nbykov0**](https://github.com/nbykov0) in #1628).
* **[system] kubeovn: increase limits**: Increased resource limits for Kube-OVN components to improve stability and performance ([**@nbykov0**](https://github.com/nbykov0) in #1629).
* **[linstor] Update Piraeus Operator to v2.10.1 to enable RWX support**: Updated Piraeus Operator to v2.10.1, enabling ReadWriteMany (RWX) volume support ([**@kvaps**](https://github.com/kvaps) in #1650).
* **[ci,dx] Bump MariaDB operator version**: Bumped MariaDB operator version for latest features and bug fixes ([**@IvanHunters**](https://github.com/IvanHunters) in #1646).
---
## Bug fixes
* **[api] Fix RBAC for listing of TenantNamespaces and handle system:masters**: Fixed regression in TenantNamespace listing RBAC and added proper handling for system:masters group ([**@kvaps**](https://github.com/kvaps) in #1511).
* **[api] Fix listing tenantnamespaces for non-oidc users**: Fixed TenantNamespace listing functionality for users not using OIDC authentication ([**@kvaps**](https://github.com/kvaps) in #1517).
* **[dashboard] Fix logout**: Fixed dashboard logout functionality to properly clear session and redirect users ([**@kvaps**](https://github.com/kvaps) in #1510).
* **[installer] Add additional check to wait for lineage-webhook**: Added additional readiness check to ensure lineage-webhook is fully ready before proceeding with installation ([**@kvaps**](https://github.com/kvaps) in #1506).
* **[lineage] Check for nil chart in HelmRelease**: Added nil check to prevent crashes when lineage webhook encounters HelmReleases using `chartRef` instead of `chart` ([**@lllamnyp**](https://github.com/lllamnyp) in #1525).
* **[kamaji] Respect 3rd party labels**: Applied patch to Kamaji controller to respect third-party labels, preventing reconciliation loops ([**@lllamnyp**](https://github.com/lllamnyp) in #1531).
* **[redis-operator] Build patched operator in-tree**: Moved Redis operator build into Cozystack organization and patched it to prevent overwriting third-party labels ([**@lllamnyp**](https://github.com/lllamnyp) in #1547).
* **[mariadb-operator] Add post-delete job to remove PVCs**: Added post-delete job to automatically remove PersistentVolumeClaims when MariaDB instances are deleted ([**@IvanHunters**](https://github.com/IvanHunters) in #1553).
* **[seaweedfs] Fix migration to v3.99**: Fixed migration issues when upgrading SeaweedFS to version 3.99 ([**@kvaps**](https://github.com/kvaps) in #1572).
* **[nats] Merge container spec, not podTemplate**: Fixed NATS configuration to properly merge container specifications instead of podTemplate ([**@lllamnyp**](https://github.com/lllamnyp) in #1571).
* **[nats] Fixes for NATS App Helm chart, fix template issues with config.merge**: Fixed template issues in NATS Helm chart related to config.merge value ([**@insignia96**](https://github.com/insignia96) in #1583).
* **[nats] Fix NATS app chart to use existing secret credentials when present**: Fixed NATS app chart to use existing secret credentials when present, preventing credential regeneration ([**@insignia96**](https://github.com/insignia96) in #1599).
* **[controller] Remove crdmem, handle DaemonSet**: Removed crdmem and improved DaemonSet handling in controller ([**@lllamnyp**](https://github.com/lllamnyp) in #1555).
* **[dashboard] Revert reconciler removal**: Reverted reconciler removal to restore proper dashboard functionality ([**@lllamnyp**](https://github.com/lllamnyp) in #1559).
* **[dashboard-controller] Fix static resources reconciliation and showing secrets**: Fixed static resources reconciliation and improved secret display in dashboard controller ([**@kvaps**](https://github.com/kvaps) in #1615).
* **[cozystack-api][dashboard] Fix filtering for application services/ingresses/secrets**: Fixed filtering functionality for application services, ingresses, and secrets in both API and dashboard ([**@kvaps**](https://github.com/kvaps) in #1612).
* **[virtual-machine] Revert per-vm network policies**: Reverted per-VM network policies to previous behavior ([**@kvaps**](https://github.com/kvaps) in #1611).
* **[cozy-lib] Fix: handling resources=nil**: Fixed handling of nil resources in cozy-lib templates ([**@kvaps**](https://github.com/kvaps) in #1607).
* **[cozy-lib] Fix malformed ResourceQuota rendering for LoadBalancer services**: Fixed malformed ResourceQuota rendering for LoadBalancer services in cozy-lib templates ([**@IvanHunters**](https://github.com/IvanHunters) in #1642).
* **[kubernetes] Cleanup loadbalancer services**: Added cleanup functionality for load balancer services in Kubernetes app ([**@lllamnyp**](https://github.com/lllamnyp) in #1631).
* **[rbac] Fix permissions for high-privilege users**: Fixed RBAC permissions for high-privilege users, ensuring proper access control ([**@lllamnyp**](https://github.com/lllamnyp) in #1622).
* **[vpc] Fix access to subnet details configmap**: Fixed access to subnet details configmap in VPC functionality ([**@lllamnyp**](https://github.com/lllamnyp) in #1638).
* **[api,lineage] Ensure node-local traffic**: Ensured node-local traffic handling for API and lineage components ([**@lllamnyp**](https://github.com/lllamnyp) in #1554).
* **[extra] ingress: rm spaces from external ip list**: Removed spaces from external IP list in ingress configuration, fixing formatting issues ([**@nbykov0**](https://github.com/nbykov0) in #1652).
* **scripts: fix 20 migration**: Fixed migration script #20 to ensure proper execution during upgrades ([**@nbykov0**](https://github.com/nbykov0) in #1653).
---
## Security
* **[redis] Bump Redis image version for security fixes**: Updated Redis image version to include latest security fixes, improving cluster security ([**@IvanHunters**](https://github.com/IvanHunters) in #1580).
* **[flux] Close Flux Operator ports to external access**: Removed hostPort and hostNetwork from Flux Operator Deployment, ensuring ports 8080 and 8081 are only accessible within the cluster ([**@IvanHunters**](https://github.com/IvanHunters) in #1581).
* **[ingress] Enforce HTTPS-only for API**: Added force-ssl-redirect annotation to default API Ingress, ensuring all HTTP traffic is redirected to HTTPS ([**@IvanHunters**](https://github.com/IvanHunters) in #1582).
---
## Dependencies & version updates
* **Update LINSTOR v1.32.3**: Updated LINSTOR to version 1.32.3 with latest features and bug fixes ([**@kvaps**](https://github.com/kvaps) in #1565).
* **Update Talos Linux v1.11.3**: Updated Talos Linux to version 1.11.3 ([**@kvaps**](https://github.com/kvaps) in #1527).
* **Update Kube-OVN v1.14.11**: Updated Kube-OVN to version 1.14.11 ([**@kvaps**](https://github.com/kvaps) in #1514).
* **[linstor] Update Piraeus Operator to v2.10.1**: Updated Piraeus Operator to v2.10.1 to enable RWX support ([**@kvaps**](https://github.com/kvaps) in #1650).
* **[system] multus: update to the latest version**: Updated Multus CNI to the latest version ([**@nbykov0**](https://github.com/nbykov0) in #1628).
* **[ci,dx] Bump MariaDB operator version**: Bumped MariaDB operator version ([**@IvanHunters**](https://github.com/IvanHunters) in #1646).
* **Increase strimzi memory limit**: Increased memory limit for Strimzi Kafka operator to improve stability and performance ([**@nbykov0**](https://github.com/nbykov0) in #1651).
---
## System Configuration
* **[system] kube-ovn: turn off enableLb**: Disabled load balancer functionality in Kube-OVN configuration ([**@nbykov0**](https://github.com/nbykov0) in #1548).
* **[core] rm talos lldp extension**: Removed Talos LLDP extension from core configuration ([**@nbykov0**](https://github.com/nbykov0) in #1586).
---
## Development, Testing, and CI/CD
* **[tests] Make Kubernetes tests POSIX-compatible**: Replaced bash-specific constructs with POSIX-compliant code, ensuring tests work reliably with /bin/sh ([**@IvanHunters**](https://github.com/IvanHunters) in #1509).
* **[ferretdb] fix tests**: Fixed FerretDB tests to ensure proper execution ([**@IvanHunters**](https://github.com/IvanHunters) in #1540).
* **[e2e] Increase Kubernetes connection timeouts**: Increased connection and request timeouts in E2E tests when communicating with Kubernetes API ([**@IvanHunters**](https://github.com/IvanHunters) in #1570).
* **[cozystack-controller] improve API tests**: Improved API tests for cozystack-controller ([**@kvaps**](https://github.com/kvaps) in #1617).
* **[ci] Fix build from external forks**: Fixed build process to work correctly from external forks ([**@kvaps**](https://github.com/kvaps) in #1530).
* **[ci,dx] Add unit tests for cozy-lib**: Added unit tests for cozy-lib to improve code quality and reliability ([**@lllamnyp**](https://github.com/lllamnyp) in #1643).
---
## Documentation
* **[website] Add VPC page**: Added VPC documentation page explaining VPC features and usage ([**@nbykov0**](https://github.com/nbykov0) in cozystack/website@9ccac78).
* **[website] Add VPC to auto-update list**: Added VPC to auto-update list in documentation ([**@nbykov0**](https://github.com/nbykov0) in cozystack/website@ca2bce6).
* **[website] Update dashboard part in OIDC configuration doc**: Updated OIDC configuration documentation with dashboard information ([**@nbykov0**](https://github.com/nbykov0) in cozystack/website@6c44b93).
* **[website] Add System Resource Planning Recommendations**: Added system resource planning recommendations documentation ([**@kvaps**](https://github.com/kvaps) in cozystack/website@c877c2a).
* **[website] Optimize website for mobile devices**: Improved website layout and responsiveness for mobile devices ([**@kvaps**](https://github.com/kvaps) in cozystack/website@3ab2338).
* **[website] Add OpenAPI UI**: Added OpenAPI UI documentation and integration ([**@kvaps**](https://github.com/kvaps) in cozystack/website@b1c1668).
* **[website] Update Cozystack video in hero banner**: Updated hero banner with new Cozystack video ([**@kvaps**](https://github.com/kvaps) in cozystack/website@e351137).
* **[website] Add screenshots carousel**: Added screenshots carousel to showcase Cozystack features ([**@kvaps**](https://github.com/kvaps) in cozystack/website@8422bd0).
* **[website] Update LINSTOR documentation**: Updated LINSTOR guide and set failmode=continue for ZFS configurations ([**@kvaps**](https://github.com/kvaps) in cozystack/website@033804e).
* **[website] Update golden image documentation**: Updated documentation for creating golden images for virtual machines ([**@kvaps**](https://github.com/kvaps) in cozystack/website@34c2f3a, cozystack/website@ef65593).
* **[website] Fix documentation formatting**: Fixed alerts, infoboxes, tabs styles and main page formatting ([**@kvaps**](https://github.com/kvaps) in cozystack/website@e992e97, cozystack/website@b2c4dee).
* **[website] Fix typo in blog article**: Fixed typo in blog article ([**@kvaps**](https://github.com/kvaps) in cozystack/website@0a4bbf3).
* **[apps] vpc: more docs**: Added more VPC documentation ([**@nbykov0**](https://github.com/nbykov0) in #1594).
* **[apps] vpc: fix typo in README**: Fixed typo in VPC README ([**@nbykov0**](https://github.com/nbykov0) in #1637).
---
## Additional Repositories
### boot-to-talos
* **[boot-to-talos] Introduce boot/install mode**: Introduced boot/install mode in boot-to-talos tool ([**@kvaps**](https://github.com/kvaps) in cozystack/boot-to-talos#5).
### cozypkg
* **[cozypkg] Handle valuesFiles from cozypkg.cozystack.io/values-files annotation**: Added support for handling valuesFiles from annotation in cozypkg ([**@kvaps**](https://github.com/kvaps) in cozystack/cozypkg#8).
---
## Refactors & chores
* **[dashboard] Migrate patches to upstream project**: Migrated dashboard patches to upstream project for better maintainability ([**@kvaps**](https://github.com/kvaps) in #1569).
* **Update CODEOWNERS**: Updated CODEOWNERS file ([**@nbykov0**](https://github.com/nbykov0) in #1537).
* **Add QOSI to ADOPTERS.md**: Added QOSI to adopters list ([**@tabu-a**](https://github.com/tabu-a) in #1589).
---
## Breaking changes & upgrade notes
No breaking changes in this release.
---
## Contributors
We'd like to thank all contributors who made this release possible:
* **[seaweedfs] Extended CA certificate duration to reduce disruptive CA rotations**: Extended CA certificate duration to reduce disruptive CA rotations. ([**@IvanHunters**](https://github.com/IvanHunters) in #1657, #1666).
* **[dashboard] Add config hash annotations to restart pods on config changes**: Added config hash annotations to restart pods when configuration changes, ensuring pods are automatically restarted when their configuration is updated ([**@kvaps**](https://github.com/kvaps) in #1662, #1665).
## Fixes
* **[tenant][kubernetes] Introduce better cleanup logic**: Improved cleanup logic for tenant Kubernetes resources, ensuring proper resource cleanup when tenants are deleted or updated ([**@kvaps**](https://github.com/kvaps) in #1661).
* **[dashboard] Fix loading arrays in forms when editing existing objects**: Fixed issue where arrays in forms were not loading correctly when editing existing objects in the dashboard ([**@kvaps**](https://github.com/kvaps)).
* **[api] Revert dynamic list kinds representation fix (fixes namespace deletion regression)**: Reverted changes from #1630 that caused a regression affecting namespace deletion and upgrades from previous versions. The regression caused namespace deletion failures with errors like "content is not a list: []unstructured.Unstructured" during namespace finalization. This revert restores compatibility with namespace deletion controller and fixes upgrade issues from previous versions, particularly when running migration 20 ([**@kvaps**](https://github.com/kvaps) in #1677).
* **[core:installer] Address buildx warnings for installer image builds**: Aligns Dockerfile syntax casing to remove buildx warnings, keeping installer builds clean ([**@nbykov0**](https://github.com/nbykov0) in #1682).
* **[system:coredns] Align CoreDNS app labels with Talos defaults**: Matches CoreDNS labels to Talos conventions so services select pods consistently across platform and tenant clusters ([**@nbykov0**](https://github.com/nbykov0) in #1675).
* **[system:monitoring-agents] Rename CoreDNS metrics service to avoid conflicts**: Renames the metrics service so it no longer clashes with the CoreDNS service used for name resolution in tenant clusters ([**@nbykov0**](https://github.com/nbykov0) in #1676).
* **[linstor] Update piraeus-operator v2.10.2 to handle fsck checks reliably**: Upgrades LINSTOR CSI to avoid failed mounts when fsck sees mounted volumes, improving volume publish reliability ([**@kvaps**](https://github.com/kvaps) in #1689, #1697).
* **[dashboard] Nest CustomFormsOverride properties under spec.properties**: Fixes schema generation so custom form properties are placed under `spec.properties`, preventing mis-rendered or missing form fields ([**@kvaps**](https://github.com/kvaps) in #1692, #1700).
* **[virtual-machine] Guard PVC resize to only expand storage**: Ensures resize jobs run only when storage size increases, avoiding unintended shrink attempts during VM updates ([**@kvaps**](https://github.com/kvaps) in #1688, #1701).
## Documentation
* **[website] Clarify GPU check command**: Makes the kubectl command for validating GPU binding more explicit, including namespace context ([**@nbykov0**](https://github.com/nbykov0) in cozystack/website#379).
* **[virtual-machine,vm-instance] Add nodeAffinity for Windows VMs based on scheduling config**: Added nodeAffinity configuration to virtual-machine and vm-instance charts to support dedicated nodes for Windows VMs. When `dedicatedNodesForWindowsVMs` is enabled in the `cozystack-scheduling` ConfigMap, Windows VMs are scheduled on nodes with label `scheduling.cozystack.io/vm-windows=true`, while non-Windows VMs prefer nodes without this label ([**@kvaps**](https://github.com/kvaps) in #1693, #1744).
* **[cilium] Enable automatic pod rollout on configmap updates**: Cilium and Cilium operator pods now automatically restart when the cilium-config ConfigMap is updated, ensuring configuration changes are applied immediately without manual intervention ([**@kvaps**](https://github.com/kvaps) in #1728, #1745).
* **Update SeaweedFS v4.02**: Updated SeaweedFS to version 4.02 with improved S3 daemon performance and fixes. This update includes better S3 compatibility and performance improvements ([**@kvaps**](https://github.com/kvaps) in #1725, #1732).
## Fixes
* **[apps] Refactor apiserver to use typed objects and fix UnstructuredList GVK**: Refactored the apiserver REST handlers to use typed objects (`appsv1alpha1.Application`) instead of `unstructured.Unstructured`, eliminating the need for runtime conversions and simplifying the codebase. Additionally, fixed an issue where `UnstructuredList` objects were using the first registered kind from `typeToGVK` instead of the kind from the object's field when multiple kinds are registered with the same Go type. This fix includes the upstream fix from kubernetes/kubernetes#135537 ([**@kvaps**](https://github.com/kvaps) in #1679, #1709).
* **[kubevirt-operator] Fix typo in VMNotRunningFor10Minutes alert**: Fixed typo in VM alert name, ensuring proper alert triggering and monitoring for virtual machines that are not running for extended periods ([**@lexfrei**](https://github.com/lexfrei) in #1770).
* **[kubevirt-operator] Revert incorrect case change in VM alerts**: Reverted incorrect case change in VM alert names to maintain consistency with alert naming conventions ([**@lexfrei**](https://github.com/lexfrei) in #1804, #1805).
* **[multus] Remove memory limit**: Removed memory limit for Multus daemonset due to unpredictable memory consumption spikes during startup after node reboots (reported up to 3Gi). This temporary change prevents out-of-memory issues while the root cause is addressed in future releases ([**@nbykov0**](https://github.com/nbykov0) in #1834).
This release introduces topology-aware routing for Cilium services, automatic pod rollouts on configuration changes, improved monitoring capabilities, and numerous bug fixes and improvements across the platform.
## Highlights
* **Topology-Aware Routing**: Enabled topology-aware routing for Cilium services, improving traffic distribution and reducing latency by routing traffic to endpoints in the same zone when possible ([**@nbykov0**](https://github.com/nbykov0) in #1734).
* **Automatic Pod Rollouts**: Cilium and Cilium operator pods now automatically restart when configuration changes, ensuring configuration updates are applied immediately ([**@kvaps**](https://github.com/kvaps) in #1728).
* **Windows VM Scheduling**: Added nodeAffinity configuration for Windows VMs based on scheduling config, enabling dedicated nodes for Windows workloads ([**@kvaps**](https://github.com/kvaps) in #1693).
* **SeaweedFS Updates**: Updated to SeaweedFS v4.02 with improved S3 daemon performance and fixes ([**@kvaps**](https://github.com/kvaps) in #1725).
---
## Major Features and Improvements
### Networking
* **[system/cilium] Enable topology-aware routing for services**: Enabled topology-aware routing for services, improving traffic distribution and reducing latency by routing traffic to endpoints in the same zone when possible. This feature helps optimize network performance in multi-zone deployments ([**@nbykov0**](https://github.com/nbykov0) in #1734).
* **[cilium] Enable automatic pod rollout on configmap updates**: Cilium and Cilium operator pods now automatically restart when the cilium-config ConfigMap is updated, ensuring configuration changes are applied immediately without manual intervention ([**@kvaps**](https://github.com/kvaps) in #1728).
### Virtual Machines
* **[virtual-machine,vm-instance] Add nodeAffinity for Windows VMs based on scheduling config**: Added nodeAffinity configuration to virtual-machine and vm-instance charts to support dedicated nodes for Windows VMs. When `dedicatedNodesForWindowsVMs` is enabled in the `cozystack-scheduling` ConfigMap, Windows VMs are scheduled on nodes with label `scheduling.cozystack.io/vm-windows=true`, while non-Windows VMs prefer nodes without this label ([**@kvaps**](https://github.com/kvaps) in #1693).
### Storage
* **Update SeaweedFS v4.02**: Updated SeaweedFS to version 4.02 with improved performance for S3 daemon and fixes for known issues. This update includes better S3 compatibility and performance improvements ([**@kvaps**](https://github.com/kvaps) in #1725).
### Tools
* **[talm] feat(init)!: require --name flag for cluster name**: Breaking change: The `talm init` command now requires the `--name` flag to specify the cluster name. This ensures consistent cluster naming and prevents accidental initialization without a name ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#86).
* **[talm] feat(template): preserve extra YAML documents in output**: Templates now preserve extra YAML documents in the output, allowing for more flexible template processing ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#87).
* **[talm] feat: add directory expansion for -f flag**: Added directory expansion support for the `-f` flag, allowing users to specify directories instead of individual files ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@ca5713e).
* **[talm] Introduce automatic root detection**: Added automatic root detection logic to simplify talm usage and reduce manual configuration ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@d165162).
* **[talm] Introduce talm kubeconfig --login command**: Added new `talm kubeconfig --login` command for easier kubeconfig management ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@5f7e05b).
* **[talm] Introduce encryption**: Added encryption support to talm for secure configuration management ([**@kvaps**](https://github.com/kvaps) in cozystack/talm#81).
* **[talm] Replace code-generation with wrapper on talosctl**: Refactored talm to use a wrapper on talosctl instead of code generation, simplifying the codebase and improving maintainability ([**@kvaps**](https://github.com/kvaps) in cozystack/talm#80).
* **[talm] Use go embed instead of code generation**: Migrated from code generation to go embed for better build performance and simpler dependency management ([**@kvaps**](https://github.com/kvaps) in cozystack/talm#79).
* **[boot-to-talos] Cozystack: Update Talos Linux v1.11.3**: Updated boot-to-talos to use Talos Linux v1.11.3 ([**@kvaps**](https://github.com/kvaps) in cozystack/boot-to-talos#7).
## Improvements
* **[seaweedfs] Extended CA certificate duration to reduce disruptive CA rotations**: Extended CA certificate duration to reduce disruptive CA rotations, improving long-term certificate management and reducing operational overhead ([**@IvanHunters**](https://github.com/IvanHunters) in #1657).
* **[dashboard] Add config hash annotations to restart pods on config changes**: Added config hash annotations to dashboard deployment templates to ensure pods are automatically restarted when their configuration changes, ensuring configuration updates are applied immediately ([**@kvaps**](https://github.com/kvaps) in #1662).
* **[tenant][kubernetes] Introduce better cleanup logic**: Improved cleanup logic for tenant Kubernetes resources, ensuring proper resource cleanup when tenants are deleted or updated. Added automated pre-delete cleanup job for tenant namespaces to remove tenant-related releases during uninstall ([**@kvaps**](https://github.com/kvaps) in #1661).
* **[system:coredns] update coredns app labels to match Talos coredns labels**: Updated coredns app labels to match Talos coredns labels, ensuring consistency across the platform ([**@nbykov0**](https://github.com/nbykov0) in #1675).
* **[system:monitoring-agents] rename coredns metrics service**: Renamed coredns metrics service to avoid interference with coredns service used for name resolution in tenant k8s clusters ([**@nbykov0**](https://github.com/nbykov0) in #1676).
* **[core:installer] Address buildx warnings**: Fixed Dockerfile syntax warnings from buildx, ensuring clean builds without warnings ([**@nbykov0**](https://github.com/nbykov0) in #1682).
* **[talm] Refactor root detection logic into single file**: Improved code organization by consolidating root detection logic into a single file ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@487b479).
* **[talm] Refactor init logic, better upgrade**: Improved initialization logic and upgrade process for better reliability ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@c512777).
* **[talm] Sugar for kubeconfig command**: Added convenience features to the kubeconfig command for improved usability ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@a4010b3).
* **[talm] wrap upgrade command**: Wrapped upgrade command for better integration and error handling ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@2e1afbf).
* **[talm] docs(readme): add Homebrew installation option**: Added Homebrew installation option to the README for easier installation on macOS ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm@12bd4f2).
* **[talm] cozystack: disable nodeCIDRs allocation**: Disabled nodeCIDRs allocation in talm for better network configuration control ([**@kvaps**](https://github.com/kvaps) in cozystack/talm#82).
* **[talm] Update license to Apache2.0**: Updated license to Apache 2.0 for better compatibility and clarity ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@eda1032).
## Fixes
* **[apps] Refactor apiserver to use typed objects and fix UnstructuredList GVK**: Refactored the apiserver REST handlers to use typed objects (`appsv1alpha1.Application`) instead of `unstructured.Unstructured`, eliminating the need for runtime conversions and simplifying the codebase. Additionally, fixed an issue where `UnstructuredList` objects were using the first registered kind from `typeToGVK` instead of the kind from the object's field when multiple kinds are registered with the same Go type. This fix includes the upstream fix from kubernetes/kubernetes#135537 ([**@kvaps**](https://github.com/kvaps) in #1679).
* **[dashboard] Fix CustomFormsOverride schema to nest properties under spec.properties**: Fixed the logic for generating CustomFormsOverride schema to properly nest properties under `spec.properties` instead of directly under `properties`, ensuring correct form schema generation ([**@kvaps**](https://github.com/kvaps) in #1692).
* **[virtual-machine] Improve check for resizing job**: Improved storage resize logic to only expand persistent volume claims when storage is being increased, preventing unintended storage reduction operations. Added validation to accurately compare current and desired storage sizes before triggering resize operations ([**@kvaps**](https://github.com/kvaps) in #1688).
* **[linstor] Update piraeus-operator v2.10.2**: Updated LINSTOR CSI to fix issues with the new fsck behaviour, resolving mount failures when fsck attempts to run on mounted devices ([**@kvaps**](https://github.com/kvaps) in #1689).
* **[api] Revert dynamic list kinds representation fix (fixes namespace deletion regression)**: Reverted changes from #1630 that caused a regression affecting namespace deletion and upgrades from previous versions. The regression caused namespace deletion failures with errors like "content is not a list: []unstructured.Unstructured" during namespace finalization. This revert restores compatibility with namespace deletion controller and fixes upgrade issues from previous versions ([**@kvaps**](https://github.com/kvaps) in #1677).
* **[talm] fix: normalize template paths for Windows compatibility**: Fixed template path handling to ensure Windows compatibility by normalizing paths ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#88).
## Dependencies
* **Update SeaweedFS v4.02**: Updated SeaweedFS to version 4.02 ([**@kvaps**](https://github.com/kvaps) in #1725).
* **[linstor] Update piraeus-operator v2.10.2**: Updated piraeus-operator to version 2.10.2 ([**@kvaps**](https://github.com/kvaps) in #1689).
* **[talm] Cozystack: Update Talos Linux v1.11.3**: Updated talm to use Talos Linux v1.11.3 ([**@kvaps**](https://github.com/kvaps) in cozystack/talm#83).
## Documentation
* **[website] Add article: Talm v0.17: Built-in Age Encryption for Secrets Management**: Added comprehensive blog post announcing Talm v0.17 and its built-in age-based encryption for secrets. Covers initial setup and key generation, encryption/decryption workflows, idempotent encryption behavior, automatic .gitignore handling, file permission safeguards, security best practices, and guidance for GitOps and CI/CD integration ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#384](https://github.com/cozystack/website/pull/384)).
* **[website] docs(talm): update talm init syntax for mandatory --preset and --name flags**: Updated documentation to reflect breaking changes in talm, adding mandatory `--preset` and `--name` flags to the talm init command ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#386).
---
## Contributors
We'd like to thank all contributors who made this release possible:
* **[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant monitoring**: Introduced the SLACK_SEVERITY_FILTER environment variable in the Alerta deployment to enable filtering of alert severities for Slack notifications based on the disabledSeverity configuration. Additionally, added a VMAgent resource template for scraping metrics within tenant namespaces, improving monitoring granularity and control. This enhancement allows administrators to configure which alert severities are sent to Slack and enables tenant-specific metrics collection for better observability ([**@IvanHunters**](https://github.com/IvanHunters) in #1712).
* **[vm] Always expose VMs with a service**: Virtual machines are now always exposed with at least a ClusterIP service, ensuring they have in-cluster DNS names and can be accessed from other pods even without public IP addresses ([**@lllamnyp**](https://github.com/lllamnyp) in #1738, #1751).
* **[tenant] Allow egress to parent ingress pods**: Updated tenant network policies to allow egress traffic to parent cluster ingress pods, enabling proper communication patterns between tenant namespaces and parent cluster ingress controllers ([**@lexfrei**](https://github.com/lexfrei) in #1765, #1776).
* **[system] Add resource requests and limits to etcd-defrag**: Added resource requests and limits to etcd-defrag job to ensure proper resource allocation and prevent resource contention during etcd maintenance operations ([**@matthieu-robin**](https://github.com/matthieu-robin) in #1785, #1786).
* **[tenant] Run cleanup job from system namespace**: Moved tenant cleanup job to run from system namespace, improving security and resource isolation for tenant cleanup operations ([**@lllamnyp**](https://github.com/lllamnyp) in #1774, #1777).
## Fixes
* **[kubevirt-operator] Fix typo in VMNotRunningFor10Minutes alert**: Fixed typo in VM alert name, ensuring proper alert triggering and monitoring for virtual machines that are not running for extended periods ([**@lexfrei**](https://github.com/lexfrei) in #1770, #1775).
* **[seaweedfs] Traffic locality**: Upgraded SeaweedFS to v4.05 with traffic locality capabilities, new admin component with web-based UI, worker component for distributed operations, and enhanced S3 monitoring with Grafana dashboards. Improves S3 service performance by routing requests to nearest available volume servers ([**@nbykov0**](https://github.com/nbykov0) in #1748, #1830).
* **[kube-ovn] Update to v1.14.25**: Updated Kube-OVN to version 1.14.25 with improved stability and new features ([**@kvaps**](https://github.com/kvaps) in #1819, #1837).
* **[linstor] Build linstor-server with custom patches**: Added custom patches to linstor-server build process, enabling platform-specific optimizations and fixes ([**@kvaps**](https://github.com/kvaps) in #1726, #1818).
* **[api, lineage] Tolerate all taints**: Updated API and lineage webhook to tolerate all taints, ensuring controllers can run on any node regardless of taint configuration ([**@nbykov0**](https://github.com/nbykov0) in #1781, #1827).
* **[ingress] Add topology anti-affinities**: Added topology anti-affinity rules to ingress controller deployment for better pod distribution across nodes ([**@kvaps**](https://github.com/kvaps) in commit 25f31022).
## Fixes
* **[linstor] fix: prevent DRBD device race condition in updateDiscGran**: Fixed race condition in DRBD device management during granularity updates, preventing potential data corruption or device conflicts ([**@kvaps**](https://github.com/kvaps) in #1829, #1836).
* **fix(linstor): prevent orphaned DRBD devices during toggle-disk retry**: Fixed issue where retry logic during disk toggle operations could leave orphaned DRBD devices, now properly cleans up devices during retry attempts ([**@kvaps**](https://github.com/kvaps) in #1823, #1825).
* **[kubernetes] Fix endpoints for cilium-gateway**: Fixed endpoint configuration for cilium-gateway, ensuring proper service discovery and connectivity ([**@kvaps**](https://github.com/kvaps) in #1729, #1808).
* **[kubevirt-operator] Revert incorrect case change in VM alerts**: Reverted incorrect case change in VM alert names to maintain consistency with alert naming conventions ([**@lexfrei**](https://github.com/lexfrei) in #1804, #1806).
## System Configuration
* **[kubeovn] Package from external repo**: Extracted Kube-OVN packaging from main repository to external repository, improving modularity ([**@lllamnyp**](https://github.com/lllamnyp) in #1535).
## Development, Testing, and CI/CD
* **[testing] Add aliases and autocomplete**: Added shell aliases and autocomplete support for testing commands, improving developer experience ([**@lllamnyp**](https://github.com/lllamnyp) in #1803, #1809).
## Dependencies
* **[seaweedfs] Traffic locality**: Upgraded SeaweedFS to v4.05 with traffic locality capabilities ([**@nbykov0**](https://github.com/nbykov0) in #1748, #1830).
* **[kube-ovn] Update to v1.14.25**: Updated Kube-OVN to version 1.14.25 ([**@kvaps**](https://github.com/kvaps) in #1819, #1837).
* **[paas-full] Add multus dependencies similar to other CNIs**: Added Multus as a dependency in the paas-full package, consistent with how other CNIs are included. This ensures proper dependency management and simplifies the installation process for environments using Multus networking ([**@nbykov0**](https://github.com/nbykov0) in #1835).
* **[linstor] Update piraeus-server patches with critical fixes**: Backported critical patches to piraeus-server that address storage stability issues and improve DRBD resource handling. These patches fix edge cases in device management and ensure more reliable storage operations ([**@kvaps**](https://github.com/kvaps) in #1850, #1853).
This release introduces LINSTOR scheduler for optimal pod placement, SeaweedFS traffic locality, a new valuesFrom-based configuration mechanism, auto-diskful for LINSTOR, automated version management systems, and numerous improvements across the platform.
## Feature Highlights
### LINSTOR Scheduler for Optimal Pod Placement
Cozystack now includes a custom Kubernetes scheduler extender that works alongside the default kube-scheduler to optimize pod placement on nodes with LINSTOR storage. When a pod requests LINSTOR-backed storage, the scheduler communicates with the LINSTOR controller to find nodes that have local replicas of the requested volumes, prioritizing placement on nodes with existing data to minimize network traffic and improve I/O performance.
The scheduler includes an admission webhook that automatically routes pods using LINSTOR CSI volumes to the custom scheduler, ensuring seamless integration without manual configuration. This feature significantly improves performance for workloads using LINSTOR storage by reducing network latency and improving data locality.
Learn more about LINSTOR in the [documentation](https://cozystack.io/docs/operations/storage/linstor/).
### SeaweedFS Traffic Locality
SeaweedFS has been upgraded to version 4.05 with new traffic locality capabilities that optimize S3 service traffic distribution. The update includes a new admin component with a web-based UI and authentication support, as well as a worker component for distributed operations. These enhancements improve S3 service performance and provide better visibility through enhanced Grafana dashboard panels for buckets, API calls, costs, and performance metrics.
The traffic locality feature ensures that S3 requests are routed to the nearest available volume servers, reducing latency and improving overall performance for distributed storage operations. TLS certificate support for admin and worker components adds an extra layer of security for management operations.
### ValuesFrom Configuration Mechanism
Cozystack now uses FluxCD's valuesFrom mechanism to replace Helm lookup functions for configuration propagation. This architectural improvement provides cleaner config propagation and eliminates the need for force reconcile controllers. Configuration from ConfigMaps (cozystack, cozystack-branding, cozystack-scheduling) and namespace service references (etcd, host, ingress, monitoring, seaweedfs) is now centrally managed through a `cozystack-values` Secret in each namespace.
This change simplifies Helm chart templates by replacing complex lookup functions with direct value references, improves configuration consistency, and reduces the reconciliation overhead. All HelmReleases now automatically receive cluster and namespace configuration through the valuesFrom mechanism, making configuration management more transparent and maintainable.
### Auto-diskful for LINSTOR
The LINSTOR integration now includes automatic diskful functionality that converts diskless nodes to diskful when they hold DRBD resources in Primary state for an extended period (30 minutes). This feature addresses scenarios where workloads are scheduled on nodes without local storage replicas by automatically creating local disk replicas when needed, improving I/O performance for long-running workloads.
When enabled with cleanup options, the system can automatically remove disk replicas that are no longer needed, preventing storage waste from temporary replicas. This intelligent storage management reduces network traffic for frequently accessed data while maintaining efficient storage utilization.
### Automated Version Management Systems
Cozystack now includes automated version management systems for PostgreSQL, Kubernetes, MariaDB, and Redis applications. These systems automatically track upstream versions and provide mechanisms for automated version updates, ensuring that platform users always have access to the latest stable versions while maintaining compatibility with existing deployments.
The version management systems integrate with the Cozystack API and dashboard, providing administrators with visibility into available versions and update paths. This infrastructure sets the foundation for future automated upgrade workflows and version compatibility management.
---
## Major Features and Improvements
### Storage
* **[linstor] Add linstor-scheduler package**: Added LINSTOR scheduler extender for optimal pod placement on nodes with LINSTOR storage. Includes admission webhook that automatically routes pods using LINSTOR CSI volumes to the custom scheduler, ensuring pods are placed on nodes with local replicas to minimize network traffic and improve I/O performance ([**@kvaps**](https://github.com/kvaps) in #1824).
* **[linstor] Enable auto-diskful for diskless nodes**: Enabled DRBD auto-diskful functionality to automatically convert diskless nodes to diskful when they hold volumes in Primary state for more than 30 minutes. Improves I/O performance for long-running workloads by creating local replicas and includes automatic cleanup options to prevent storage waste ([**@kvaps**](https://github.com/kvaps) in #1826).
* **[linstor] Build linstor-server with custom patches**: Added custom patches to linstor-server build process, enabling platform-specific optimizations and fixes ([**@kvaps**](https://github.com/kvaps) in #1726).
* **[seaweedfs] Traffic locality**: Upgraded SeaweedFS to v4.05 with traffic locality capabilities, new admin component with web-based UI, worker component for distributed operations, and enhanced S3 monitoring with Grafana dashboards. Improves S3 service performance by routing requests to nearest available volume servers ([**@nbykov0**](https://github.com/nbykov0) in #1748).
* **[linstor] fix: prevent DRBD device race condition in updateDiscGran**: Fixed race condition in DRBD device management during granularity updates, preventing potential data corruption or device conflicts ([**@kvaps**](https://github.com/kvaps) in #1829).
* **fix(linstor): prevent orphaned DRBD devices during toggle-disk retry**: Fixed issue where retry logic during disk toggle operations could leave orphaned DRBD devices, now properly cleans up devices during retry attempts ([**@kvaps**](https://github.com/kvaps) in #1823).
### Platform Architecture
* **[platform] Replace Helm lookup with valuesFrom mechanism**: Replaced Helm lookup functions with FluxCD valuesFrom mechanism for configuration propagation. Configuration from ConfigMaps and namespace references is now managed through `cozystack-values` Secret, simplifying templates and eliminating force reconcile controllers ([**@kvaps**](https://github.com/kvaps) in #1787).
* **[platform] refactor: split cozystack-resource-definitions into separate packages**: Refactored cozystack-resource-definitions into separate packages for better organization and maintainability, improving code structure and reducing coupling between components ([**@kvaps**](https://github.com/kvaps) in #1778).
* **[platform] Separate assets server into dedicated deployment**: Separated assets server from main platform deployment, improving scalability and allowing independent scaling of asset delivery infrastructure ([**@kvaps**](https://github.com/kvaps) in #1705).
* **[core] Extract Talos package from installer**: Extracted Talos package configuration from installer into a separate package, improving modularity and enabling independent updates ([**@kvaps**](https://github.com/kvaps) in #1724).
* **[registry] Add application labels and update filtering mechanism**: Added application labels to registry resources and improved filtering mechanism for better resource discovery and organization ([**@kvaps**](https://github.com/kvaps) in #1707).
* **fix(registry): implement field selector filtering for label-based resources**: Implemented field selector filtering for label-based resources in the registry, improving query performance and resource lookup efficiency ([**@kvaps**](https://github.com/kvaps) in #1845).
* **[platform] Add alphabetical sorting to registry resource lists**: Added alphabetical sorting to registry resource lists in the API and dashboard, improving user experience when browsing available applications ([**@lexfrei**](https://github.com/lexfrei) in #1764).
### Version Management
* **[postgres] Add version management system with automated version updates**: Introduced version management system for PostgreSQL with automated version tracking and update mechanisms ([**@kvaps**](https://github.com/kvaps) in #1671).
* **[kubernetes] Add version management system with automated version updates**: Added version management system for Kubernetes tenant clusters with automated version tracking and update capabilities ([**@kvaps**](https://github.com/kvaps) in #1672).
* **[mariadb] Add version management system with automated version updates**: Implemented version management system for MariaDB with automated version tracking and update mechanisms ([**@kvaps**](https://github.com/kvaps) in #1680).
* **[redis] Add version management system with automated version updates**: Added version management system for Redis with automated version tracking and update capabilities ([**@kvaps**](https://github.com/kvaps) in #1681).
### Networking
* **[kube-ovn] Update to v1.14.25**: Updated Kube-OVN to version 1.14.25 with improved stability and new features ([**@kvaps**](https://github.com/kvaps) in #1819).
* **[kubeovn] Package from external repo**: Extracted Kube-OVN packaging from main repository to external repository, improving modularity ([**@lllamnyp**](https://github.com/lllamnyp) in #1535).
* **[cilium] Update Cilium to v1.18.5**: Updated Cilium to version 1.18.5 with latest features and bug fixes ([**@lexfrei**](https://github.com/lexfrei) in #1769).
* **[system/cilium] Enable topology-aware routing for services**: Enabled topology-aware routing for Cilium services, improving traffic distribution and reducing latency by routing traffic to endpoints in the same zone when possible ([**@nbykov0**](https://github.com/nbykov0) in #1734).
* **[cilium] Enable automatic pod rollout on configmap updates**: Cilium and Cilium operator pods now automatically restart when the cilium-config ConfigMap is updated, ensuring configuration changes are applied immediately ([**@kvaps**](https://github.com/kvaps) in #1728).
* **[kubernetes] Fix endpoints for cilium-gateway**: Fixed endpoint configuration for cilium-gateway, ensuring proper service discovery and connectivity ([**@kvaps**](https://github.com/kvaps) in #1729).
* **[multus] Increase memory limit**: Increased memory limits for Multus components to handle larger network configurations and reduce out-of-memory issues ([**@nbykov0**](https://github.com/nbykov0) in #1773).
* **[main][paas-full] Add multus dependencies similar to other CNIs**: Added Multus as a dependency in the paas-full package, consistent with how other CNIs are included ([**@nbykov0**](https://github.com/nbykov0) in #1842).
### Virtual Machines
* **[vm] Always expose VMs with a service**: Virtual machines are now always exposed with at least a ClusterIP service, ensuring they have in-cluster DNS names and can be accessed from other pods even without public IP addresses ([**@lllamnyp**](https://github.com/lllamnyp) in #1738).
* **[virtual-machine] Improve check for resizing job**: Improved storage resize logic to only expand persistent volume claims when storage is being increased, preventing unintended storage reduction operations ([**@kvaps**](https://github.com/kvaps) in #1688).
* **[virtual-machine,vm-instance] Add nodeAffinity for Windows VMs based on scheduling config**: Added nodeAffinity configuration to virtual-machine and vm-instance charts to support dedicated nodes for Windows VMs ([**@kvaps**](https://github.com/kvaps) in #1693).
### Monitoring
* **[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant monitoring**: Introduced SLACK_SEVERITY_FILTER environment variable in Alerta deployment to enable filtering of alert severities for Slack notifications. Added VMAgent resource template for scraping metrics within tenant namespaces, improving monitoring granularity ([**@IvanHunters**](https://github.com/IvanHunters) in #1712).
* **[monitoring] Improve tenant metrics collection**: Improved tenant metrics collection mechanisms for better observability and monitoring coverage ([**@IvanHunters**](https://github.com/IvanHunters) in #1684).
### System Configuration
* **[api, lineage] Tolerate all taints**: Updated API and lineage webhook to tolerate all taints, ensuring controllers can run on any node regardless of taint configuration ([**@nbykov0**](https://github.com/nbykov0) in #1781).
* **[system] Add resource requests and limits to etcd-defrag**: Added resource requests and limits to etcd-defrag job to ensure proper resource allocation and prevent resource contention ([**@matthieu-robin**](https://github.com/matthieu-robin) in #1785).
* **[system:coredns] update coredns app labels to match Talos coredns labels**: Updated coredns app labels to match Talos coredns labels, ensuring consistency across the platform ([**@nbykov0**](https://github.com/nbykov0) in #1675).
* **[system:monitoring-agents] rename coredns metrics service**: Renamed coredns metrics service to avoid interference with coredns service used for name resolution in tenant k8s clusters ([**@nbykov0**](https://github.com/nbykov0) in #1676).
### Tenants and Namespaces
* **[tenant] Allow egress to parent ingress pods**: Updated tenant network policies to allow egress traffic to parent cluster ingress pods, enabling proper communication patterns ([**@lexfrei**](https://github.com/lexfrei) in #1765).
* **[tenant] Run cleanup job from system namespace**: Moved tenant cleanup job to run from system namespace, improving security and resource isolation ([**@lllamnyp**](https://github.com/lllamnyp) in #1774).
### FluxCD
* **[fluxcd] Add flux-aio module and migration**: Added FluxCD all-in-one module with migration support, simplifying FluxCD installation and management ([**@kvaps**](https://github.com/kvaps) in #1698).
* **[fluxcd] Enable source-watcher**: Enabled source-watcher in FluxCD configuration for improved GitOps synchronization and faster update detection ([**@kvaps**](https://github.com/kvaps) in #1706).
### Applications
* **[dashboard] Fix CustomFormsOverride schema to nest properties under spec.properties**: Fixed CustomFormsOverride schema generation to properly nest properties under `spec.properties` instead of directly under `properties`, ensuring correct form schema generation ([**@kvaps**](https://github.com/kvaps) in #1692).
* **[apps] Refactor apiserver to use typed objects and fix UnstructuredList GVK**: Refactored apiserver REST handlers to use typed objects instead of unstructured.Unstructured, eliminating runtime conversions. Fixed UnstructuredList GVK issue where objects were using the first registered kind instead of the correct kind ([**@kvaps**](https://github.com/kvaps) in #1679).
* **[keycloak] Make kubernetes client public**: Made Kubernetes client public in Keycloak configuration, enabling broader access patterns for Kubernetes integrations ([**@lllamnyp**](https://github.com/lllamnyp) in #1802).
## Improvements
* **[granular kubernetes application extensions dependencies]**: Improved dependency management for Kubernetes application extensions with more granular control over dependencies ([**@nbykov0**](https://github.com/nbykov0) in #1683).
* **[core:installer] Address buildx warnings**: Fixed Dockerfile syntax warnings from buildx, ensuring clean builds without warnings ([**@nbykov0**](https://github.com/nbykov0) in #1682).
* **[linstor] Update piraeus-operator v2.10.2**: Updated LINSTOR CSI to version 2.10.2 with improved stability and bug fixes ([**@kvaps**](https://github.com/kvaps) in #1689).
* **Update SeaweedFS v4.02**: Updated SeaweedFS to version 4.02 with improved S3 daemon performance and fixes ([**@kvaps**](https://github.com/kvaps) in #1725).
* **[installer,dx] Rename cozypkg to cozyhr**: Renamed cozypkg tool to cozyhr for better branding and consistency ([**@kvaps**](https://github.com/kvaps) in #1763).
## Fixes
* **fix(platform): fix migrations for v0.40 release**: Fixed platform migrations for v0.40 release, ensuring smooth upgrades from previous versions ([**@kvaps**](https://github.com/kvaps) in #1846).
* **[platform] fix migration for removing fluxcd-operator**: Fixed migration logic for removing fluxcd-operator, ensuring clean removal without leaving orphaned resources ([**@kvaps**](https://github.com/kvaps) in commit 4a83d2c7).
* **[kubevirt-operator] Fix typo in VMNotRunningFor10Minutes alert**: Fixed typo in VM alert name, ensuring proper alert triggering and monitoring ([**@kvaps**](https://github.com/kvaps) in #1770).
* **[kubevirt-operator] Revert incorrect case change in VM alerts**: Reverted incorrect case change in VM alert names to maintain consistency ([**@lexfrei**](https://github.com/lexfrei) in #1804).
* **[cozystack-controller] Fix: move crds to definitions**: Fixed CRD placement by moving them to definitions directory, ensuring proper resource organization ([**@kvaps**](https://github.com/kvaps) in #1759).
## Dependencies
* **Update SeaweedFS v4.02**: Updated SeaweedFS to version 4.02 ([**@kvaps**](https://github.com/kvaps) in #1725).
* **[seaweedfs] Traffic locality**: Upgraded SeaweedFS to v4.05 with traffic locality capabilities ([**@nbykov0**](https://github.com/nbykov0) in #1748).
* **[linstor] Update piraeus-operator v2.10.2**: Updated piraeus-operator to version 2.10.2 ([**@kvaps**](https://github.com/kvaps) in #1689).
* **[kube-ovn] Update to v1.14.25**: Updated Kube-OVN to version 1.14.25 ([**@kvaps**](https://github.com/kvaps) in #1819).
* **[cilium] Update Cilium to v1.18.5**: Updated Cilium to version 1.18.5 ([**@lexfrei**](https://github.com/lexfrei) in #1769).
* **Update go modules**: Updated Go modules to latest versions ([**@kvaps**](https://github.com/kvaps) in #1736).
## Development, Testing, and CI/CD
* **[ci] Fix auto-release workflow**: Fixed auto-release workflow to ensure correct release publishing and tagging ([**@kvaps**](https://github.com/kvaps) in commit 526af294).
* **fix(ci): ensure correct latest release after backport publishing**: Fixed CI workflow to correctly identify and tag the latest release after backport publishing ([**@kvaps**](https://github.com/kvaps) in #1800).
* **[workflows] Add auto patch release workflow**: Added automated patch release workflow for streamlined release management ([**@kvaps**](https://github.com/kvaps) in #1754).
* **[workflow] Add GitHub Action to update release notes from changelogs**: Added GitHub Action to automatically update release notes from changelog files ([**@kvaps**](https://github.com/kvaps) in #1752).
* **[ci] Improve backport workflow with merge_commits skip and conflict resolution**: Improved backport workflow with better merge commit handling and conflict resolution ([**@kvaps**](https://github.com/kvaps) in #1694).
* **[testing] Add aliases and autocomplete**: Added shell aliases and autocomplete support for testing commands, improving developer experience ([**@lllamnyp**](https://github.com/lllamnyp) in #1803).
* **[kubernetes] Add lb tests for tenant k8s**: Added load balancer tests for tenant Kubernetes clusters, improving test coverage ([**@IvanHunters**](https://github.com/IvanHunters) in #1783).
* **[agents] Add instructions for working with unresolved code review comments**: Added documentation and instructions for working with unresolved code review comments in agent workflows ([**@kvaps**](https://github.com/kvaps) in #1710).
* **feat(ci): add /retest command to rerun tests from Prepare environment**: Added `/retest` command to rerun tests from Prepare environment workflow ([**@kvaps**](https://github.com/kvaps) in commit 30c1041e).
* **fix(ci): remove GITHUB_TOKEN extraheader to trigger workflows**: Removed GITHUB_TOKEN extraheader to properly trigger workflows ([**@kvaps**](https://github.com/kvaps) in commit 68a639b3).
* **Fix: Add missing components to `distro-full` bundle**: Fixed missing components in distro-full bundle, ensuring all required components are included ([**@LoneExile**](https://github.com/LoneExile) in #1620).
* **Update Flux Operator (v0.33.0)**: Updated Flux Operator to version 0.33.0 ([**@kingdonb**](https://github.com/kingdonb) in #1649).
* **Add changelogs for v0.38.3 and v.0.38.4**: Added missing changelogs for v0.38.3 and v0.38.4 releases ([**@androndo**](https://github.com/androndo) in #1743).
* **Add changelogs to v.0.39.1**: Added changelog for v0.39.1 release ([**@androndo**](https://github.com/androndo) in #1750).
* **Add Cloupard to ADOPTERS.md**: Added Cloupard to the adopters list ([**@SerjioTT**](https://github.com/SerjioTT) in #1733).
## Documentation
* **[website] docs: expand monitoring and alerting documentation**: Expanded monitoring and alerting documentation with comprehensive guides, examples, and troubleshooting information ([**@IvanHunters**](https://github.com/IvanHunters) in [cozystack/website#388](https://github.com/cozystack/website/pull/388)).
* **[website] fix auto-generation of documentation**: Fixed automatic documentation generation process, ensuring all documentation is properly generated and formatted ([**@IvanHunters**](https://github.com/IvanHunters) in [cozystack/website#391](https://github.com/cozystack/website/pull/391)).
* **[website] secure boot**: Added documentation for Secure Boot support in Talos Linux ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#387](https://github.com/cozystack/website/pull/387)).
## Tools
* **[talm] feat(helpers): add bond interface discovery helpers**: Added bond interface discovery helpers to talm for easier network configuration ([**@kvaps**](https://github.com/kvaps) in [cozystack/talm#94](https://github.com/cozystack/talm/pull/94)).
* **[talm] feat(talosconfig): add certificate regeneration from secrets.yaml**: Added certificate regeneration functionality to talm talosconfig command, allowing certificates to be regenerated from secrets.yaml ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@1319dde).
* **[talm] fix(init): make name optional for -u flag**: Made name parameter optional for init command with -u flag, improving flexibility ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@da29320).
* **[talm] fix(wrapper): copy NoOptDefVal when remapping -f to -F flag**: Fixed wrapper to properly copy NoOptDefVal when remapping flags, ensuring correct default value handling ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@f6a6f1d).
* **[talm] fix(root): detect project root with secrets.encrypted.yaml**: Fixed root detection to properly identify project root when secrets.encrypted.yaml is present ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@cf56780).
* **[talm] Fix interfaces helper for Talos v1.12**: Fixed interfaces helper to work correctly with Talos v1.12 ([**@kvaps**](https://github.com/kvaps) in cozystack/talm@34984ae).
* **[talm] Fix typo on README.md**: Fixed typo in README documentation ([**@diegolakatos**](https://github.com/diegolakatos) in [cozystack/talm#92](https://github.com/cozystack/talm/pull/92)).
* **[talm] fix(template): return error for invalid YAML in template output**: Fixed template command to return proper error for invalid YAML output ([**@kvaps**](https://github.com/kvaps) in [cozystack/talm#93](https://github.com/cozystack/talm/pull/93)).
* **[talm] feat(cozystack): enable allocateNodeCIDRs by default**: Enabled allocateNodeCIDRs by default in talm cozystack preset ([**@lexfrei**](https://github.com/lexfrei) in [cozystack/talm#91](https://github.com/cozystack/talm/pull/91)).
* **[boot-to-talos] feat(network): add VLAN interface support via netlink**: Added VLAN interface support via netlink in boot-to-talos for advanced network configuration ([**@kvaps**](https://github.com/kvaps) in cozystack/boot-to-talos@02874d7).
* **[boot-to-talos] feat(network): add bond interface support via netlink**: Added bond interface support via netlink in boot-to-talos for network bonding configurations ([**@kvaps**](https://github.com/kvaps) in cozystack/boot-to-talos@067822d).
* **[boot-to-talos] Draft EFI Support**: Added draft EFI support in boot-to-talos for UEFI boot scenarios ([**@kvaps**](https://github.com/kvaps) in cozystack/boot-to-talos@e194bc8).
* **[boot-to-talos] Change default install image size from 2GB to 3GB**: Changed default install image size from 2GB to 3GB to accommodate larger installations ([**@kvaps**](https://github.com/kvaps) in cozystack/boot-to-talos@3bfb035).
* **[cozyhr] feat(values): add valuesFrom support for HelmRelease**: Added valuesFrom support for HelmRelease in cozyhr tool, enabling better configuration management ([**@kvaps**](https://github.com/kvaps) in cozystack/cozyhr@7dff0c8).
* **[cozyhr] Rename cozypkg to cozyhr**: Renamed cozypkg tool to cozyhr for better branding ([**@kvaps**](https://github.com/kvaps) in cozystack/cozyhr@1029461).
---
## Contributors
We'd like to thank all contributors who made this release possible:
* **[linstor] Update piraeus-server patches with critical fixes**: Backported critical patches to piraeus-server that address storage stability issues and improve DRBD resource handling. These patches fix edge cases in device management and ensure more reliable storage operations ([**@kvaps**](https://github.com/kvaps) in #1850, #1852).
* **[linstor] Refactor node-level RWX validation**: Refactored the node-level ReadWriteMany (RWX) validation logic in LINSTOR CSI. The validation has been moved to the CSI driver level with a custom linstor-csi image build, providing more reliable RWX volume handling and clearer error messages when RWX requirements cannot be satisfied ([**@kvaps**](https://github.com/kvaps) in #1856, #1857).
## Fixes
* **[linstor] Remove node-level RWX validation**: Removed the problematic node-level RWX validation that was causing issues with volume provisioning. The validation logic has been refactored and moved to a more appropriate location in the LINSTOR CSI driver ([**@kvaps**](https://github.com/kvaps) in #1851).
* **[apiserver] Fix Watch resourceVersion and bookmark handling**: Fixed issues with Watch API handling of resourceVersion and bookmarks, ensuring proper event streaming and state synchronization for API clients ([**@kvaps**](https://github.com/kvaps) in #1860).
## Dependencies
* **[cilium] Update Cilium to v1.18.6**: Updated Cilium CNI to v1.18.6 with security fixes and performance improvements ([**@sircthulhu**](https://github.com/sircthulhu) in #1868, #1870).
* **[kubernetes] Increase default apiServer resourcesPreset to large**: Increased the default resource preset for kube-apiserver to `large` to ensure more reliable operation under higher workloads and prevent resource constraints ([**@kvaps**](https://github.com/kvaps) in #1875, #1882).
* **[kubernetes] Increase kube-apiserver startup probe threshold**: Increased the startup probe threshold for kube-apiserver to allow more time for the API server to become ready, especially in scenarios with slow storage or high load ([**@kvaps**](https://github.com/kvaps) in #1876, #1883).
* **[etcd] Increase probe thresholds for better recovery**: Increased etcd probe thresholds to provide more time for recovery operations, improving cluster resilience during network issues or temporary slowdowns ([**@kvaps**](https://github.com/kvaps) in #1874, #1878).
## Fixes
* **[dashboard] Fix view of loadbalancer IP in services window**: Fixed an issue where load balancer IP addresses were not displayed correctly in the services window of the dashboard ([**@IvanHunters**](https://github.com/IvanHunters) in #1884, #1887).
## Dependencies
* **Update Talos Linux v1.11.6**: Updated Talos Linux to v1.11.6 with latest security patches and improvements ([**@kvaps**](https://github.com/kvaps) in #1879).
This release introduces MongoDB as a new managed application, expanding Cozystack's database offerings alongside existing PostgreSQL, MySQL, and Redis services. The release also includes storage improvements, Kubernetes stability enhancements, and updated documentation.
## Feature Highlights
### MongoDB Managed Application
Cozystack now includes MongoDB as a fully managed database service. Users can deploy production-ready MongoDB instances directly from the application catalog with minimal configuration.
Key capabilities:
- **Replica Set deployment**: Automatic configuration of MongoDB replica sets for high availability
- **Persistent storage**: Integration with Cozystack storage backends for reliable data persistence
- **Resource management**: Configurable CPU, memory, and storage resources
- **Monitoring integration**: Built-in metrics export for platform monitoring
Deploy MongoDB through the Cozystack dashboard or using the standard application deployment workflow ([**@lexfrei**](https://github.com/lexfrei) in #1822, #1881).
## Improvements
* **[linstor] Update piraeus-server patches with critical fixes**: Backported critical patches to piraeus-server that address storage stability issues and improve DRBD resource handling. These patches fix edge cases in device management and ensure more reliable storage operations ([**@kvaps**](https://github.com/kvaps) in #1850, #1852).
* **[linstor] Refactor node-level RWX validation**: Refactored the node-level ReadWriteMany (RWX) validation logic in LINSTOR CSI. The validation has been moved to the CSI driver level with a custom linstor-csi image build, providing more reliable RWX volume handling and clearer error messages when RWX requirements cannot be satisfied ([**@kvaps**](https://github.com/kvaps) in #1856, #1857).
* **[kubernetes] Increase default apiServer resourcesPreset to large**: Increased the default resource preset for kube-apiserver to `large` to ensure more reliable operation under higher workloads and prevent resource constraints ([**@kvaps**](https://github.com/kvaps) in #1875, #1882).
* **[kubernetes] Increase kube-apiserver startup probe threshold**: Increased the startup probe threshold for kube-apiserver to allow more time for the API server to become ready, especially in scenarios with slow storage or high load ([**@kvaps**](https://github.com/kvaps) in #1876, #1883).
* **[etcd] Increase probe thresholds for better recovery**: Increased etcd probe thresholds to provide more time for recovery operations, improving cluster resilience during network issues or temporary slowdowns ([**@kvaps**](https://github.com/kvaps) in #1874, #1878).
## Fixes
* **[linstor] Remove node-level RWX validation**: Removed the problematic node-level RWX validation that was causing issues with volume provisioning. The validation logic has been refactored and moved to a more appropriate location in the LINSTOR CSI driver ([**@kvaps**](https://github.com/kvaps) in #1851).
* **[apiserver] Fix Watch resourceVersion and bookmark handling**: Fixed issues with Watch API handling of resourceVersion and bookmarks, ensuring proper event streaming and state synchronization for API clients ([**@kvaps**](https://github.com/kvaps) in #1860).
* **[dashboard] Fix view of loadbalancer IP in services window**: Fixed an issue where load balancer IP addresses were not displayed correctly in the services window of the dashboard ([**@IvanHunters**](https://github.com/IvanHunters) in #1884, #1887).
## Dependencies
* **[cilium] Update cilium to v1.18.6**: Updated Cilium CNI to v1.18.6 with security fixes and performance improvements ([**@sircthulhu**](https://github.com/sircthulhu) in #1868, #1870).
* **Update Talos Linux v1.11.6**: Updated Talos Linux to v1.11.6 with latest security patches and improvements ([**@kvaps**](https://github.com/kvaps) in #1879).
## Documentation
* **[website] Add documentation for creating and managing cloned virtual machines**: Added comprehensive guide for VM cloning operations ([**@sircthulhu**](https://github.com/sircthulhu) in [cozystack/website#401](https://github.com/cozystack/website/pull/401)).
* **[website] Simplify NFS driver setup instructions**: Improved NFS driver setup documentation with clearer instructions ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#399](https://github.com/cozystack/website/pull/399)).
* **[website] Update Talos installation docs for Hetzner and Servers.com**: Updated installation documentation with improved instructions for Hetzner and Servers.com environments ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#395](https://github.com/cozystack/website/pull/395)).
* **[website] Add Hetzner RobotLB documentation**: Added documentation for configuring public IP with Hetzner RobotLB ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#394](https://github.com/cozystack/website/pull/394)).
* **[website] Add Hidora organization support details**: Added Hidora to the support page with organization details ([**@matthieu-robin**](https://github.com/matthieu-robin) in [cozystack/website#397](https://github.com/cozystack/website/pull/397), [cozystack/website#398](https://github.com/cozystack/website/pull/398)).
* **[kubernetes] Add enum validation for IngressNginx exposeMethod**: Added enum validation for the `exposeMethod` field in IngressNginx configuration, preventing invalid values and improving user experience with clear valid options ([**@sircthulhu**](https://github.com/sircthulhu) in #1895, #1897).
* **[monitoring-agents] Set minReplicas to 1 for VPA for VMAgent**: Configured VPA (Vertical Pod Autoscaler) to maintain at least 1 replica for VMAgent, ensuring monitoring availability during scaling operations ([**@sircthulhu**](https://github.com/sircthulhu) in #1894, #1905).
* **[mongodb] Remove user-configurable images from MongoDB chart**: Removed user-configurable image options from the MongoDB chart to simplify configuration and ensure consistency with tested image versions ([**@kvaps**](https://github.com/kvaps) in #1901, #1904).
* **[kubernetes] Show Service and Ingress resources for Kubernetes app in dashboard**: Added visibility of Service and Ingress resources for Kubernetes applications in the dashboard, improving resource management and monitoring capabilities ([**@sircthulhu**](https://github.com/sircthulhu) in #1912, #1915).
## Fixes
* **[dashboard] Fix filtering on Pods tab for Service**: Fixed an issue where pod filtering was not working correctly on the Pods tab when viewing Services in the dashboard ([**@sircthulhu**](https://github.com/sircthulhu) in #1909, #1914).
This alpha release introduces a fundamental architectural shift from HelmRelease bundles to Package-based deployment managed by the new cozystack-operator. It includes a comprehensive backup system with Velero integration, significant API changes that rename the core CRD, Flux sharding for improved tenant workload distribution, enhanced monitoring capabilities, and various improvements to virtual machines, tenants, and the build workflow.
> **⚠️ Alpha Release Warning**: This is a pre-release version intended for testing and early adoption. Breaking changes may occur before the stable v1.0.0 release.
## Breaking Changes
### API Rename: CozystackResourceDefinition → ApplicationDefinition
The `CozystackResourceDefinition` CRD has been renamed to `ApplicationDefinition` for better clarity and consistency. This change affects:
- All Go types and controller files
- CRD Helm chart renamed from `cozystack-resource-definition-crd` to `application-definition-crd`
- All cozyrds YAML manifests updated to use `kind: ApplicationDefinition`
A migration (v24) is included to handle the transition automatically.
### Package-Based Deployment
The platform now uses Package resources managed by cozystack-operator instead of HelmRelease bundles. Key changes:
- Restructured values.yaml with full configuration support (networking, publishing, authentication, scheduling, branding, resources)
- Added values-isp-full.yaml and values-isp-hosted.yaml for bundle variants
- Package resources replace old HelmRelease templates
- PackageSources moved from sources/ to templates/sources/
- Migration script `hack/migrate-to-version-1.0.sh` provided for converting ConfigMaps to Package resources
---
## Major Features and Improvements
### Cozystack Operator
A new operator has been introduced to manage Package and PackageSource resources, providing declarative package management for the platform:
* **[cozystack-operator] Introduce API objects: packages and packagesources**: Added new CRDs for declarative package management, defining the API for Package and PackageSource resources ([**@kvaps**](https://github.com/kvaps) in #1740).
* **[cozystack-operator] Introduce Cozystack-operator core logic**: Implemented core reconciliation logic for the operator, handling Package and PackageSource lifecycle management ([**@kvaps**](https://github.com/kvaps) in #1741).
* **[cozystack-operator] Add Package and PackageSource reconcilers**: Added controllers for Package and PackageSource resources with full reconciliation support ([**@kvaps**](https://github.com/kvaps) in #1755).
* **[cozystack-operator] Add deployment files**: Added Kubernetes deployment manifests for running cozystack-operator in the cluster ([**@kvaps**](https://github.com/kvaps) in #1761).
* **[platform] Add PackageSources for cozystack-operator**: Added PackageSource definitions for cozystack-operator integration ([**@kvaps**](https://github.com/kvaps) in #1760).
* **[cozypkg] Add tool for managing Package and PackageSources**: Added CLI tool for managing Package and PackageSource resources ([**@kvaps**](https://github.com/kvaps) in #1756).
### Backup System
Comprehensive backup functionality has been added with Velero integration for managing application backups:
* **[backups] Implement core backup Plan controller**: Core controller for managing backup schedules and plans, providing the foundation for backup orchestration ([**@lllamnyp**](https://github.com/lllamnyp) in #1640).
* **[backups] Build and deploy backup controller**: Deployment infrastructure for the backup controller, including container image builds and Kubernetes manifests ([**@lllamnyp**](https://github.com/lllamnyp) in #1685).
* **[backups] Scaffold a backup strategy API group**: Added API group for backup strategies, enabling pluggable backup implementations ([**@lllamnyp**](https://github.com/lllamnyp) in #1687).
* **[backups] Add indices to core backup resources**: Added indices to backup resources for improved query performance ([**@lllamnyp**](https://github.com/lllamnyp) in #1719).
* **[backups] Stub the Job backup strategy controller**: Added stub implementation for Job-based backup strategy ([**@lllamnyp**](https://github.com/lllamnyp) in #1720).
* **[backups] Implement Velero strategy controller**: Integration with Velero for backup operations, enabling enterprise-grade backup capabilities ([**@androndo**](https://github.com/androndo) in #1762).
* **[backups,dashboard] User-facing UI**: Dashboard interface for managing backups and backup jobs, providing visibility into backup status and history ([**@lllamnyp**](https://github.com/lllamnyp) in #1737).
### Platform Architecture
* **[platform] Migrate from HelmRelease bundles to Package-based deployment**: Replaced HelmRelease bundle system with Package resources managed by cozystack-operator. Includes restructured values.yaml with full configuration support and migration tooling ([**@kvaps**](https://github.com/kvaps) in #1816).
* **refactor(api): rename CozystackResourceDefinition to ApplicationDefinition**: Renamed CRD and all related types for better clarity and consistency. Updated all Go types, controllers, and 25+ YAML manifests ([**@kvaps**](https://github.com/kvaps) in #1864).
* **feat(flux): implement flux sharding for tenant HelmReleases**: Added Flux sharding support to distribute tenant HelmRelease reconciliation across multiple controllers, improving scalability in multi-tenant environments ([**@kvaps**](https://github.com/kvaps) in #1816).
* **refactor(installer): migrate installer to cozystack-operator**: Moved installer functionality to cozystack-operator for unified management ([**@kvaps**](https://github.com/kvaps) in #1816).
* **feat(api): add chartRef to ApplicationDefinition**: Added chartRef field to support ExternalArtifact references for flexible chart sourcing ([**@kvaps**](https://github.com/kvaps) in #1816).
* **feat(api): show only hash in version column for applications and modules**: Simplified version display in API responses for cleaner output ([**@kvaps**](https://github.com/kvaps) in #1816).
### Virtual Machines
* **[vm] Always expose VMs with a service**: Virtual machines are now always exposed with at least a ClusterIP service, ensuring they have in-cluster DNS names and can be accessed from other pods even without public IP addresses ([**@lllamnyp**](https://github.com/lllamnyp) in #1738, #1751).
### Monitoring
* **[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant monitoring**: Introduced the SLACK_SEVERITY_FILTER environment variable in the Alerta deployment to enable filtering of alert severities for Slack notifications based on the disabledSeverity configuration. Additionally, added a VMAgent resource template for scraping metrics within tenant namespaces, improving monitoring granularity and control ([**@IvanHunters**](https://github.com/IvanHunters) in #1712).
### Tenants
* **[tenant] Allow egress to parent ingress pods**: Updated tenant network policies to allow egress traffic to parent cluster ingress pods, enabling proper communication patterns between tenant namespaces and parent cluster ingress controllers ([**@lexfrei**](https://github.com/lexfrei) in #1765, #1776).
* **[tenant] Run cleanup job from system namespace**: Moved tenant cleanup job to run from system namespace, improving security and resource isolation for tenant cleanup operations ([**@lllamnyp**](https://github.com/lllamnyp) in #1774, #1777).
### System
* **[system] Add resource requests and limits to etcd-defrag**: Added resource requests and limits to etcd-defrag job to ensure proper resource allocation and prevent resource contention during etcd maintenance operations ([**@matthieu-robin**](https://github.com/matthieu-robin) in #1785, #1786).
### Development and Build
* **feat(cozypkg): add cross-platform build targets with version injection**: Added cross-platform build targets (linux/amd64, linux/arm64, darwin/amd64, darwin/arm64) for cozypkg/cozyhr tool with automatic version injection from git tags ([**@kvaps**](https://github.com/kvaps) in #1862).
* **refactor: move scripts to hack directory**: Reorganized scripts to standard hack/ location following Kubernetes project conventions ([**@kvaps**](https://github.com/kvaps) in #1863).
## Fixes
* **fix(talos): skip rebuilding assets if files already exist**: Improved Talos package build process to avoid redundant asset rebuilds when files are already present, reducing build time ([**@kvaps**](https://github.com/kvaps)).
* **[kubevirt-operator] Fix typo in VMNotRunningFor10Minutes alert**: Fixed typo in VM alert name, ensuring proper alert triggering and monitoring for virtual machines that are not running for extended periods ([**@lexfrei**](https://github.com/lexfrei) in #1770, #1775).
* **[backups] Fix malformed glob and split in template**: Fixed malformed glob pattern and split operation in backup template processing ([**@lllamnyp**](https://github.com/lllamnyp) in #1708).
## Documentation
* **[website] docs(storage): simplify NFS driver setup instructions**: Simplified NFS driver setup documentation with clearer instructions ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#399](https://github.com/cozystack/website/pull/399)).
* **[website] Add Hidora organization support details**: Added Hidora to the support page with organization details ([**@matthieu-robin**](https://github.com/matthieu-robin) in [cozystack/website#397](https://github.com/cozystack/website/pull/397)).
* **[website] Update LinkedIn link for Hidora organization**: Updated LinkedIn link for Hidora organization on the support page ([**@matthieu-robin**](https://github.com/matthieu-robin) in [cozystack/website#398](https://github.com/cozystack/website/pull/398)).
---
## Migration Guide
### From v0.38.x / v0.39.x to v1.0.0-alpha.1
1.**Backup your cluster** before upgrading
2. Run the migration script: `hack/migrate-to-version-1.0.sh`
3. The migration will:
- Convert ConfigMaps to Package resources
- Rename CozystackResourceDefinition to ApplicationDefinition
- Update HelmRelease references to use Package-based deployment
### Known Issues
- This is an alpha release; some features may be incomplete or change before stable release
- Migration script should be tested in a non-production environment first
---
## Contributors
We'd like to thank all contributors who made this release possible:
> **⚠️ Alpha Release Warning**: This is a pre-release version intended for testing and early adoption. Breaking changes may occur before the stable v1.0.0 release.
## Major Features and Improvements
### New Applications
* **[apps] Add MongoDB managed application**: Added MongoDB as a new managed application, providing a fully managed MongoDB database with automatic scaling, backups, and high availability support ([**@lexfrei**](https://github.com/lexfrei) in #1822).
### Networking
* **[kilo] Introduce kilo**: Added Kilo WireGuard mesh networking support. Kilo provides secure WireGuard-based VPN mesh for connecting Kubernetes nodes across different networks and regions ([**@kvaps**](https://github.com/kvaps) in #1691).
* **[local-ccm] Add local-ccm package**: Added local cloud controller manager package for managing load balancer services in local/bare-metal environments without a cloud provider ([**@kvaps**](https://github.com/kvaps) in #1831).
### Platform
* **[platform] Add flux-plunger controller**: Added flux-plunger controller to automatically fix stuck HelmRelease errors by cleaning up failed resources and retrying reconciliation ([**@kvaps**](https://github.com/kvaps) in #1843).
* **[platform] Split telemetry between operator and controller**: Separated telemetry collection between cozystack-operator and cozystack-controller for better metrics isolation and monitoring capabilities ([**@kvaps**](https://github.com/kvaps) in #1869).
* **[platform] Remove cozystack.io/ui label**: Cleaned up deprecated `cozystack.io/ui` labels from platform components ([**@kvaps**](https://github.com/kvaps) in #1872).
## Improvements
* **[kubernetes] Increase default apiServer resourcesPreset to large**: Increased the default resource preset for kube-apiserver to `large` to ensure more reliable operation under higher workloads ([**@kvaps**](https://github.com/kvaps) in #1875).
* **[kubernetes] Increase kube-apiserver startup probe threshold**: Increased the startup probe threshold for kube-apiserver to allow more time for the API server to become ready ([**@kvaps**](https://github.com/kvaps) in #1876).
* **[etcd] Increase probe thresholds for better recovery**: Increased etcd probe thresholds to provide more time for recovery operations, improving cluster resilience ([**@kvaps**](https://github.com/kvaps) in #1874).
## Fixes
* **[apiserver] Fix Watch resourceVersion and bookmark handling**: Fixed issues with Watch API handling of resourceVersion and bookmarks, ensuring proper event streaming and state synchronization ([**@kvaps**](https://github.com/kvaps) in #1860).
* **[dashboard] Fix view of loadbalancer IP in services window**: Fixed an issue where load balancer IP addresses were not displayed correctly in the services window of the dashboard ([**@IvanHunters**](https://github.com/IvanHunters) in #1884).
## Dependencies
* **[cilium] Update cilium to v1.18.6**: Updated Cilium CNI to v1.18.6 with security fixes and performance improvements ([**@sircthulhu**](https://github.com/sircthulhu) in #1868).
* **Update Talos Linux v1.12.1**: Updated Talos Linux to v1.12.1 with latest features, security patches and improvements ([**@kvaps**](https://github.com/kvaps) in #1877).
## Documentation
* **[website] Add documentation for creating and managing cloned virtual machines**: Added comprehensive guide for VM cloning operations ([**@sircthulhu**](https://github.com/sircthulhu) in [cozystack/website#401](https://github.com/cozystack/website/pull/401)).
* **[website] Simplify NFS driver setup instructions**: Improved NFS driver setup documentation with clearer instructions ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#399](https://github.com/cozystack/website/pull/399)).
* **[website] Update Talos installation docs for Hetzner and Servers.com**: Updated installation documentation with improved instructions for Hetzner and Servers.com environments ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#395](https://github.com/cozystack/website/pull/395)).
* **[website] Add Hetzner RobotLB documentation**: Added documentation for configuring public IP with Hetzner RobotLB ([**@kvaps**](https://github.com/kvaps) in [cozystack/website#394](https://github.com/cozystack/website/pull/394)).
* **[website] Add Hidora organization support details**: Added Hidora to the support page with organization details ([**@matthieu-robin**](https://github.com/matthieu-robin) in [cozystack/website#397](https://github.com/cozystack/website/pull/397), [cozystack/website#398](https://github.com/cozystack/website/pull/398)).
Hubble is a network and security observability platform built on top of Cilium. It provides deep visibility into the communication and behavior of services in your Kubernetes cluster.
## Prerequisites
- Cozystack platform running with Cilium as the CNI
- Monitoring hub enabled for Grafana access
## Configuration
Hubble is disabled by default in Cozystack. To enable it, update the Cilium configuration.
### Enable Hubble
Edit the Cilium values in your platform configuration to enable Hubble:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.