Commit Graph

79 Commits

Author SHA1 Message Date
kklinch0
8e79f24c5b add rq for pg
Signed-off-by: kklinch0 <kklinch0@gmail.com>
2025-06-11 22:48:09 +03:00
Andrei Kvapil
646dab497c [e2e] Increase timeout awaiting for machines
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-06-09 12:34:54 +02:00
Andrei Kvapil
ea0dd68e84 [tests] fix: increase qemu system disk size
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-05-30 00:54:09 +02:00
Andrei Kvapil
f798cbd9f9 Update Talos Linux v1.10.3
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-05-29 23:18:53 +02:00
Andrei Kvapil
74e7e5cdfb [tests] Add tests for tenant Kubernetes cluster
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-05-25 09:30:26 +02:00
Andrei Kvapil
2bf4032d5b [tests] Introduce cozytest - a new testing framework
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-05-25 09:29:58 +02:00
Timofei Larkin
a8ba6b1328 Remove user-facing config of limits and requests (#935)
This patch introduces reusable library charts that provide
backward-compatibility for users that specify their resources as
explicit requests and limits for cpu, however this input is processed so
that limits are set equal to requests except for CPU which only gets
requests. Users can now embrace the new form by directly specifying
resources in the first level of nesting (e.g. resources.cpu=100m instead
of .resources.requests.cpu=100m). The order of precedence is top-level,
then requests, then limits, ensuring that nothing will break in terms of
scheduling, however workloads that specified limits much higher than
requests might get a performance hit, now that they cannot use all this
excess capacity. This should only affect memory-hungry workloads in
low-contention environments.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced a reusable Helm library chart, "cozy-lib", providing common
templates and resource helpers for other charts.
- Added resource preset and sanitization templates to standardize
Kubernetes resource configurations.
- ClickHouse chart now depends on "cozy-lib" for improved resource
handling.
- Added a new packaging script and streamlined Helm chart packaging
processes across multiple packages.

- **Bug Fixes**
- Resource configuration logic in the ClickHouse deployment was updated
to use the new library templates, ensuring more consistent resource
definitions.

- **Chores**
- Added new Makefiles and version mapping for streamlined Helm chart
packaging and validation.
- Updated ClickHouse chart version to 0.9.0 and reflected this in
version mapping files.
- Refactored Makefile targets to consolidate packaging logic and improve
maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 18:11:27 +03:00
Timofei Larkin
9592f7fe46 Remove user-facing config of limits and requests
This patch introduces reusable library charts that provide
backward-compatibility for users that specify their resources as
explicit requests and limits for cpu, however this input is processed so
that limits are set equal to requests except for CPU which only gets
requests. Users can now embrace the new form by directly specifying
resources in the first level of nesting (e.g. resources.cpu=100m instead
of .resources.requests.cpu=100m). The order of precedence is top-level,
then requests, then limits, ensuring that nothing will break in terms of
scheduling, however workloads that specified limits much higher than
requests might get a performance hit, now that they cannot use all this
excess capacity. This should only affect memory-hungry workloads in
low-contention environments.

Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2025-05-23 17:32:42 +03:00
Andrei Kvapil
3780925a68 [platform] Introduce expose-services and expose-ingress options
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-05-15 12:35:02 +02:00
dtrdnk
9239852ec8 Update permissions version for CRI containerd
Signed-off-by: dtrdnk <4demenko@gmail.com>
2025-04-25 15:45:07 +03:00
Andrei Kvapil
58d959b305 [tests] refactor tests and remove e2e.applications (#893)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-25 12:55:09 +02:00
Andrei Kvapil
5d7e56bffe [tests] refactor tests and remove e2e.applications
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-25 12:28:43 +02:00
Andrei Kvapil
69b3ddf717 [e2e] Better output in case of failed HelmReleases
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-25 12:21:49 +02:00
Timofei Larkin
4bb524e53d Suppress wget progress bar
In our CI wget spams thousands of lines of the progress bar into the
output, making it hard to read. Turns out, it doesn't have an option to
just remove the progress bar, but explicitly directing wget's log to
stdout and invoking --show-progress sends that to stderr which we
redirect to dev/null. The downloaded size is still reported at regular
intervals, but --progress=dot:giga shortens that to one line per 32M
which is manageable.

Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2025-04-23 17:37:57 +03:00
Andrei Kvapil
1c53a6f9f6 [e2e] fix timeouts for capi and keycloak
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-23 14:01:51 +02:00
Andrei Kvapil
39ede77fec [ci] Fix checkout and improve error output for gen_versions_map.sh
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-22 18:34:50 +02:00
Andrei Kvapil
d8f3547db7 [ci] Fix escaping for gen_versions_map.sh script
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-22 17:52:53 +02:00
kklinch0
2def9f4e83 [ci] Fix escaping for gen_versions_map.sh script
Signed-off-by: kklinch0 <kklinch0@gmail.com>
2025-04-22 16:33:40 +03:00
kklinch0
1d639fda0d [tests] fix tests and add retry for monitoring hr tests
Signed-off-by: kklinch0 <kklinch0@gmail.com>
2025-04-10 15:23:31 +03:00
Andrei Kvapil
559eb8dea9 Fix: versions_map, use awk instead of grep
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-10 10:42:31 +02:00
Andrei Kvapil
f8dfc43cae [e2e] Add mirror.gcr.io as default mirror for docker.io (#782)
related issues:
- https://github.com/cozystack/talm/pull/48
- https://github.com/cozystack/website/pull/154
- https://github.com/cozystack/cozystack/pull/782


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced additional configuration options that enable using Docker
image mirrors. This enhancement can improve image retrieval performance
and provide redundancy while maintaining the existing functionality.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-04-10 10:36:23 +02:00
Andrei Kvapil
3e19bc74d4 [tests] Add mirror.gcr.io as default mirror for docker.io
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-10 10:36:02 +02:00
Andrei Kvapil
1721d397a7 Fix: versions_map, include only versions from tags
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-09 14:07:07 +02:00
Andrei Kvapil
e954ab7f8b Upload kernel and initramfs to release assets
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-08 23:27:37 +02:00
Andrei Kvapil
d10a9ad4e6 Workflows Fix uploading assets 2025-04-03 15:37:48 +02:00
Andrei Kvapil
6980dc59c5 Add workflow to run e2e tests using GitHub CI
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-03 15:02:52 +02:00
Nick Volynkin
c795e4fb68 Prepare release v0.29.0 (#740)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Chores**
- Streamlined the asset release process to automatically replace
existing files during uploads.
  
- **Container Image Updates**
- Upgraded versions across multiple components—including backup,
caching, autoscaling, API, dashboard, monitoring, and more—to align with
the latest release (e.g., updating from v0.28.0 to v0.29.0 and other
minor version increments).
- Updated specific images for Grafana, PostgreSQL, MariaDB, ClickHouse,
and others to their latest versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Nick Volynkin <nick.volynkin@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-02 23:45:25 +02:00
Andrei Kvapil
d971f2ff29 Enhance versions_map generator logic (#741)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Chores**
- Enhanced version mapping with improved error reporting and clearer
version resolution, ensuring more accurate and reliable version
displays.
- Updated version references for multiple packages to maintain
consistency and stability across the system.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-02 21:43:17 +02:00
Nick Volynkin
47ff861f00 Upload build assets with make upload_assets (#737)
# Upload build assets with make upload_assets; keep image digests

* Keep image digests in cozystack-installer.yml
* Remove manifests/cozysаtack-installer.yaml, which is a build asset,
not source file.
* Requires `gh`, the GitHub CLI app.

Co-authored-by:  Andrei Kvapil <kvapss@gmail.com>
Signed-off-by: Nick Volynkin <nick.volynkin@gmail.com>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced an automated process that uploads release assets during
each update cycle, ensuring smoother and more reliable deployments.
	- Added a new target for uploading assets in the build process.

- **Refactor**
- Streamlined the organization and generation of deployment artifacts to
improve efficiency and consistency in our release workflow.
	- Adjusted output file management for better structure.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Nick Volynkin <nick.volynkin@gmail.com>
Co-authored-by: Andrei Kvapil <kvapss@gmail.com>
2025-04-02 14:46:02 +02:00
Andrei Kvapil
3d962685ce Move project from aenix-io to cozystack repository
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-03-10 21:16:58 +01:00
klinch0
d0d62e8847 feature/add-goldpinger (#648)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced a comprehensive Grafana dashboard for Goldpinger, offering
real-time insights into node health, error occurrences, and response
times with intuitive filtering.
- Expanded deployment configurations to include Goldpinger across
environments, streamlining release management and dependency handling.
- Launched a dedicated deployment package featuring customizable
templates for secure, efficient Kubernetes deployments—including
workloads, services, ingress, and monitoring integrations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-02-25 10:08:08 +01:00
Floppy Disk
b75aaf177b add kafka monitoring 2025-02-10 13:29:17 +03:00
klinch0
842d3e55bc feature/add-flux-dashboards (#619)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced a new dashboard for Flux Control Plane monitoring that
visualizes key performance metrics like CPU, memory, API requests, and
more.
- Added a second dashboard for Flux Cluster Stats to display resource
reconciliation, operation durations, and readiness indicators.
- Seamlessly integrated these dashboards into the monitoring workflow
with dynamic querying and periodic refresh options.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-02-06 13:50:57 +01:00
klinch0
861e6c464b feature/add-client-etcd-monitoring (#613)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced etcd monitoring with new metrics exposure, pod scraping
configuration, and comprehensive alert rules for proactive
observability.
- Introduced a new `VMPodScrape` resource for improved pod metrics
collection.
- Added a new PrometheusRule configuration for monitoring etcd clusters
with various alert conditions.
- **Chores**
  - Upgraded the etcd release from version 2.4.0 to 2.5.0.
- Consolidated and renamed monitoring dashboard references for better
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-02-05 13:13:28 +01:00
Andrei Kvapil
33d2b24ff2 Prepare release v0.22.0 (#570)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-01-16 12:24:24 +01:00
Andrei Kvapil
34d6ab032f Update Talos v1.9.1 (#553)
This PR includes a new image based on Talos Linux v1.9.1

- new DRBD module 9.2.12:
https://github.com/LINBIT/drbd/blob/master/ChangeLog
- ZFS fix: https://github.com/siderolabs/extensions/issues/572

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
	- Updated Talos system components to version 1.9.1
	- Added SELinux workaround DaemonSet for KubeVirt

- **Chores**
	- Updated image references for base installer and system extensions
- Modified installation script configuration to enhance Kubernetes setup
process
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-01-15 13:01:31 +01:00
Andrei Kvapil
227848a59d Introduce cozystack-controller (#560)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

Based on the comprehensive summary of changes, here are the release
notes:

- **New Features**
	- Added a new Kubernetes controller for managing workload monitoring
- Introduced telemetry collection capabilities with configurable options
- Added new Custom Resource Definitions (CRDs) for Workload and
WorkloadMonitor

- **Improvements**
	- Enhanced API infrastructure with new API group and version
	- Improved deployment configurations for various system components
	- Added development container and workflow configurations

- **Bug Fixes**
	- Updated import paths to correct domain naming

- **Chores**
	- Updated copyright years
	- Refined module dependencies
	- Standardized code linting and testing configurations

- **Infrastructure**
- Increased `cozystack-api` deployment replicas from 1 to 2 for improved
availability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-01-09 12:24:51 +01:00
Andrei Kvapil
b6e27cb3dc disable node.kubernetes.io/exclude-from-external-load-balancers label (#552) 2024-12-30 15:31:48 +01:00
Andrei Kvapil
007ebd8c9c update Talos Linux v1.8.4 (#544)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-12-27 14:33:17 +01:00
Andrei Kvapil
95e39c951a Prepare release v0.20.0 (#522)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-12-09 18:42:41 +01:00
Andrei Kvapil
21c291c4de Refactor Keycloak (#502)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
  - Integrated OpenID Connect (OIDC) for enhanced authentication.
- Added dynamic Role resource for tenant-specific access to Kubernetes
secrets.
  - Introduced new Keycloak realm groups for improved role management.

- **Improvements**
  - Enhanced error handling for service readiness checks.
- Streamlined configuration files for better clarity and management of
OIDC settings.
- Updated handling of API server address and improved configuration
adaptability based on OIDC settings.

- **Bug Fixes**
- Removed deprecated configurations related to Keycloak, simplifying
deployment.

These updates aim to improve security, usability, and overall system
performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-04 09:31:08 +01:00
Andrei Kvapil
2b10fb25c8 Update Talos Linux v1.8.3 (#497) 2024-12-02 19:23:28 +01:00
Andrei Kvapil
bfbde07c55 Prepare release v0.18.0 (#462)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

- **New Features**
	- Expanded build process to include the `cozystack-api` component.
- Updated image versions for `cozystack`, `darkhttpd`, and other
components to improve performance and stability.

- **Bug Fixes**
- Updated image digests for various components, ensuring the latest
updates and security patches are applied.

- **Documentation**
- Incremented version numbers across multiple configuration files for
clarity and consistency.

- **Chores**
- Updated various package versions in the version map for better
dependency management.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-11-06 09:26:26 +01:00
klinch0
b9e80b9a91 add endpoint for http checks (#469) 2024-11-05 17:06:35 +01:00
klinch0
3c27a1e9bf add metrics agents (#461)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced new HelmRelease configurations for cert-manager, monitoring
agents, and Victoria Metrics Operator in Kubernetes.
- Added resource specifications for `vmselect` in the VMCluster
configuration.
- Enhanced resource management for `vmselect` with defined limits and
requests for memory and CPU.

- **Bug Fixes**
	- Adjusted resource limits for Redis failover memory allocation.

- **Documentation**
- Updated README and release notes for various components, enhancing
clarity and usability.

- **Chores**
- Updated image versions across multiple components for consistency and
performance improvements.
- Modified migration scripts to facilitate transitions and manage
resources effectively.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Andrei Kvapil <kvapss@gmail.com>
2024-11-04 19:01:33 +01:00
klinch0
f06f653744 add values and checks (#454)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced a new configuration file for Kubernetes deployments,
enhancing clarity on parameters and settings.
- Added common parameters for NATS, including external access and
persistent volume settings.

- **Bug Fixes**
- Improved error handling and feedback in Helm release management
scripts.

- **Chores**
- Reduced verbosity in test output by removing unnecessary echo
statements in the testing Makefile.
- Added success return statements in various check scripts to ensure
proper termination.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-11-04 18:43:36 +01:00
Andrei Kvapil
7b78af6092 Introduce Cozystack API (#460)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced a RESTful API for managing `Application` resources,
enabling CRUD operations with HelmRelease integration.
- Added validation functions for `Application` and `ApplicationSpec`,
laying the groundwork for future validation rules.
- Implemented configuration management for resources, allowing for
structured application and release settings.

- **Bug Fixes**
- Addressed API rule violations related to naming conventions and
missing types in the CozyStack API definitions.

- **Tests**
- Added comprehensive tests for round-trip functionality and version
compatibility within the Apps API server.

- **Documentation**
- Introduced documentation for the `v1alpha1` API version, including
licensing and code generation annotations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-11-04 17:33:34 +01:00
klinch0
7eb8864f42 Add tests refactor (#451)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced a new script for managing Helm releases in Kubernetes,
including installation and status monitoring.
- Added a configuration file for tenant settings, enabling monitoring
and SeaweedFS.
- Enhanced PostgreSQL initialization script to manage database roles and
privileges dynamically.
  - Added a new local pre-commit hook for version map checks.

- **Bug Fixes**
  - Updated pre-commit hooks for consistent formatting.

- **Tests**
- Improved testing capabilities for applications in a Kubernetes
environment with new Makefile targets.
  - Enhanced Docker image with tools for YAML and JSON processing.
- Updated testing image to the latest version for improved performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Co-authored-by: Andrei Kvapil <kvapss@gmail.com>
2024-10-22 17:14:04 +02:00
Andrei Kvapil
1ec10165b0 Update talos v1.8.1 (#448) 2024-10-21 17:29:39 +02:00
Andrei Kvapil
c0c3e17033 Fix osx grep have no -P flag (#438) 2024-10-21 11:59:56 +02:00