mirror of
https://github.com/outbackdingo/proxmox-cloud-controller-manager.git
synced 2026-01-27 10:20:13 +00:00
706faa8d08
Add the group.topology.proxmox.sinextra.dev/ label to improve support for node selector and affinity rules. Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
115 lines
3.6 KiB
Markdown
115 lines
3.6 KiB
Markdown
# Kubernetes cloud controller manager for Proxmox
|
|
|
|
To me, it seems like Proxmox is a bit old-fashioned when it comes to creating virtual machines.
|
|
It doesn't have a lot of automation built-in, so you have to do a lot of things manually.
|
|
Proxmox is a good option if you have a static infrastructure or don't create new virtual machines very often.
|
|
|
|
I use Terraform to launch my Kubernetes nodes.
|
|
However, when I need to scale down the cluster, I have to manually delete the corresponding node resource in Kubernetes.
|
|
That's why I created the CCM (Cloud Controller Manager) for Proxmox.
|
|
Originally, it was designed to work with [Talos CCM](https://github.com/siderolabs/talos-cloud-controller-manager), but it was not difficult to make it as standalone solution.
|
|
|
|
The CCM does a few things: it initialises new nodes, applies common labels to them, and removes them when they're deleted. It also supports multiple clusters, meaning you can have one kubernetes cluster across multiple Proxmox clusters.
|
|
|
|
The basic definitions:
|
|
* kubernetes label `topology.kubernetes.io/region` is a Proxmox cluster `clusters[].region`
|
|
* kubernetes label `topology.kubernetes.io/zone` is a hypervisor host machine name
|
|
|
|
This makes it possible for me to use pods affinity/anti-affinity.
|
|
|
|
## Example
|
|
|
|
```yaml
|
|
# cloud provider config
|
|
clusters:
|
|
- url: https://cluster-api-1.exmple.com:8006/api2/json
|
|
insecure: false
|
|
# Proxox auth token
|
|
token_id: "user!token-id"
|
|
token_secret: "secret"
|
|
# Uniq region name
|
|
region: cluster-1
|
|
- url: https://cluster-api-2.exmple.com:8006/api2/json
|
|
insecure: false
|
|
token_id: "user!token-id"
|
|
token_secret: "secret"
|
|
region: cluster-2
|
|
```
|
|
|
|
Node spec result:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Node
|
|
metadata:
|
|
labels:
|
|
...
|
|
# Type generated base on CPU and RAM
|
|
node.kubernetes.io/instance-type: 2VCPU-2GB
|
|
# Proxmox cluster name as in the config
|
|
topology.kubernetes.io/region: cluster-1
|
|
# Proxmox hypervisor host machine name
|
|
topology.kubernetes.io/zone: pve-node-1
|
|
|
|
# Proxmox specific labels
|
|
topology.proxmox.sinextra.dev/region: cluster-1
|
|
topology.proxmox.sinextra.dev/zone: pve-node-1
|
|
# HA group labels - the same idea as node-role
|
|
group.topology.proxmox.sinextra.dev/${HAGroup}: ""
|
|
|
|
name: worker-1
|
|
spec:
|
|
...
|
|
# providerID - magic string:
|
|
# cluster-1 - cluster name as in the config
|
|
# 123 - Proxmox VM ID
|
|
providerID: proxmox://cluster-1/123
|
|
status:
|
|
addresses:
|
|
- address: 172.16.0.31
|
|
type: InternalIP
|
|
- address: worker-1
|
|
type: Hostname
|
|
```
|
|
|
|
## Install
|
|
|
|
See [Install](docs/install.md) for installation instructions.
|
|
|
|
## Controllers
|
|
|
|
Support controllers:
|
|
|
|
* cloud-node
|
|
* Updates node resource.
|
|
* Assigns labels and taints based on Proxmox VM configuration.
|
|
* cloud-node-lifecycle
|
|
* Cleans up node resource when Proxmox VM is deleted.
|
|
|
|
## FAQ
|
|
|
|
See [FAQ](docs/faq.md) for answers to common questions.
|
|
|
|
## Contributing
|
|
|
|
Contributions are welcomed and appreciated!
|
|
See [Contributing](CONTRIBUTING.md) for our guidelines.
|
|
|
|
## License
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
|
|
---
|
|
|
|
`Proxmox®` is a registered trademark of [Proxmox Server Solutions GmbH](https://www.proxmox.com/en/about/company).
|