* We now use GitHub Actions instead of a private Drone which is great
for OSS build transparency, but one drawback to that change is that
we no longer auto-deploy matchbox to internal Poseidon Labs infra
* We don't want public workflows for deploys to our private infra
Some commands were added in the ipxeBootstrap to enable architecture detection based on the architecture the iPXE version used was built for.
This is done through the use of an additional flag in the chain which is the ipxeBootstrap using the buildarch parameter.
* Add a publish workflow that uses the mkdocs-pages re-usable GitHub
Workflow when the `release-docs` branch is updated
* Build the docs site with mkdocs and publish to GitHub Pages
* Migrate from the internal Drone server using a GitHub Workflow
to perform the multi-arch container image build
* Use self-hosted GitHub runners on ARM64 to perform the ARM64
build step faster that QEMU/KVM emulation
* Mandate approval for all workflow runs from outside contributors
since the builds use push credentials and partially run internally
* Regenerate example Ignition using fcct v0.18.0 to produce Ignition
spec v3.4.0 as a followup to https://github.com/poseidon/matchbox/pull/1079
* Fix one profile example where a double escape isn't needed
* Emphasize that Container Linux Configs have been dropped and
that Butane is the way forward to use modern Ignition v2
* Include some links to Flatcar Linux docs
* Update Makefile and docs for v0.10.0 release process
* Update github.com/coreos/go-semver from v0.3.0 to 167f5da to
prevent it from using gopkg.in/yaml.v2 (CVE-2019-11254)
```
go mod why gopkg.in/yaml.v2
(main module does not need package gopkg.in/yaml.v2)
```
* Recommend preparing Ignition configs external to Matchbox (e.g.
with Terraform poseidon/terraform-provider-ct)
* Document that Matchbox Rendering is discouraged
* Add Butane Config support as a replacement for dropping Container
Linux Config support. Perform Matchbox Go template evaluation,
translation to Ignition, and parsing to a forward compatible version
* Both Flatcar Linux and Fedora CoreOS now support Ignition v2
(spec v3.x) for machine consumption
* Drop support for Ignition v0.35.0 (Ignition spec v2.x)
* Drop support for Container Linux Configs (the YAML format that
rendered to Ignition v2.x JSON for Flatcar Linux)
* Deprecate rendering Container Linux Configs
* Use tools like [poseidon/ct](https://github.com/poseidon/terraform-provider-ct) or
[butane](https://coreos.github.io/butane/getting-started/) to validate and convert a
Butane Config (`focs` or `flatcar`) to Ignition (for Matchbox to serve)
* Please migrate to serving CoreOS Ignition directly, the Container
Linux related HTTP and gRPC endpoints will be removed in future
* Discontinue using Matchbox's Container Linux Config features
* Flatcar Linux OS now supports Ignition v2.13+ which means it
can accept Ignition v3.x spec's, like Fedora CoreOS. Matchbox
supports this by serving Ignition documents directly
* Users of the poseidon/matchbox Terraform provider can pass
a `matchbox_profile` `raw_ignition` contents with the desired
Ignition v3.3 spec
* Users of the poseidon/ct Terraform provider can write Butane
Config YAML, perform templating, and render an Ignition document
using either the fcos or the flatcar variant
* Default branch was renamed from master to main
* Internal build and publishing workflows were unaffected,
the GitHub Action is just for public contribution vetting
* Add initrd=main karg directive for UEFI (ignored by BIOS)
* Update Butane config version to v1.4.0 (generates Ignition v3.3.0)
in `fedora-coreos` and `fedora-coreos-install` examples
* Update virt-install flag --os-variant
* Removed virt-install deprecated flag --os-type
* Remove virt-install QEMU/KVM event preserve since it apparently
wasn't implemented anyway and QEMU/KVM now warns about it
* Remove serial consol kernel argument from examples, but still
mention it in docs
Rel:
* https://github.com/coreos/fedora-coreos-docs/pull/282
* https://www.spinics.net/linux/fedora/libvir/msg222078.html
We still support passing the rootfs image as an appended initrd, but we
recommend using coreos.live.rootfs_url instead: it generally boots faster
and requires less RAM.
When coreos-installer is running from the live image, it no longer needs
a separate install image, since by default it installs from content
embedded in the live system.
* Looking at enabling some update automation, the clarity of
using non-vendored Go modules/checksums outweighs the risk
of a module disappearing (mitigated by proxies)
* Update Fedora CoreOS live PXE and disk install examples to
Fedora 33
* Increase libvirt VM memory from 2GB to 3GB to support live
PXE example, which is mostly just for laptop examples/demos.
Reduce the VM count from 3 to 2 to compensate.
* Change `fedora-coreos.ign` to suggest using an ed25519 SSH
key since Fedora CoreOS 33 disables RSA SHA1 (256 is still ok
but most people won't know which they have)
* Update Terraform examples to use Terraform v0.13
* Refresh examples to show Fedora CoreOS and Flatcar Linux
* Remove the etcd3 example, not worth it to maintain
* Refactor examples to boot provision minimal hosts
with Fedora CoreOS or Flatcar Linux
* Remove the etcd3 cluster example or other specific
kinds of hosts
* Update script get-fedora-coreos
* Remove script get-coreos
* Matchbox is published as a binary or container image,
not planning to resume RPM/Copr publishing since its a
pain
* Publishing to Copr repo stopped in v0.6 (3 years ago)
* Use our own infra to perform image builds and push images. Provides
future options for publishing images to multiple image registries and
for multiple architectures, while keeping push permissions in-house
* Remove Travis ability to push to Quay
* Consider splitting repo and also enabling Quay automated builds
* Use our own infra to perform image builds and push images.
Provides future options for publishing images to multiple
image registries and for multiple architectures, while keeping
push permissions in-house
* Remove Travis ability to push to Quay
* Use Travis only for Go tests of Pull Requests
* Use `quay.io/dghubble/protoc` as the standard codegen
environment across projects. Pin a version (v3.10.1)
* Generated code is now automatically ignored by golint
based on the comment header, which will simplify linting
* Upgrade protobuf from 2bba0603135d to v1.3.2
* Upgrade grpc from v1.2.1 to v1.25.1
* Remove `get-protoc` and `codegen` scripts
<!-- READ: Issues are used to receive focused bug reports from users and to track planned future enhancements by the authors. Topics like support, debugging help, advice, and operation are out of scope and should not use issues-->
**Description**
A clear and concise description of what the bug is.
**Steps to Reproduce**
Provide clear steps to reproduce the bug.
- [ ] Relevant error messages if appropriate (concise, not a dump of everything).
**Expected behavior**
A clear and concise description of what you expected to happen.
* Release: Matchbox version or Git SHA (reporting latest is **not** helpful)
**Possible Solution**
<!-- Most bug reports should have some inkling about solutions. Otherwise, your report may be less of a bug and more of a support request (see top).-->
@@ -4,6 +4,65 @@ Notable changes between releases.
## Latest
## v0.11.0
* Update butane from v0.17.0 to v0.18.0 ([#1079](https://github.com/poseidon/matchbox/pull/1079))
* Add support for `fcos` [v1.5.0](https://coreos.github.io/butane/config-fcos-v1_5/) Butane Configs
* Add support for `flatcar` [v1.1.0](https://coreos.github.io/butane/config-flatcar-v1_1/) Butane Configs
* Render Ignition as Ignition spec [v3.4.0](https://coreos.github.io/ignition/configuration-v3_4/)
## v0.10.0
* Remove support for Ignition v0.35.0 (Ignition spec v2.x)
* Remove support for Container Linux Configs (**action required**)
* Container Linux Configs were a YAML format that rendered to Ignition (spec v2.x)
* Flatcar Linux now supports Ignition v2 (spec v3.x)
* Butane is a suitable YAML format that renders Ignition v2 (spec v3.x)
* Upgrade Ignition from v0.35.0 (spec v2.x) to v2.14.0 (spec v3.x)
* Update Go version (v1.20.2) and alpine base image (v3.17.3)
* Add limited support for Matchbox rendering Butane configs ([#997](https://github.com/poseidon/matchbox/pull/997)) ([docs](https://matchbox.psdn.io/ignition/#matchbox-rendering))
* Recommend writing Butane via external tools (**action required**)
* For Terraform, use [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct)
* For a CLI, use [`butane`](https://github.com/coreos/butane)
* Parse Ignition and render forward to Ignition v2 (spec v3.3)
* Ignition is [forward](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61) compatible (e.g. a `v3.1` spec can be rendered as `v3.3` safely)
If you still template Container Linux Configs via Matchbox, [migrate](https://www.flatcar.org/docs/latest/provisioning/config-transpiler/) to Butane by prepending:
```yaml
variant:flatcar
version:1.0.0
```
## v0.9.1
* Add dependabot Go module update automation ([#833](https://github.com/poseidon/matchbox/pull/833))
* Update Go version (v1.18.4) and alpine base image (v3.16.1)
* Move `dnsmasq` container image to its own [repo](https://github.com/poseidon/dnsmasq) ([#840](https://github.com/poseidon/matchbox/pull/840))
* Deprecate rendering Container Linux Configs
* Please migrate to serving CoreOS Ignition directly
* Use tools like [poseidon/ct](https://github.com/poseidon/terraform-provider-ct) or [butane](https://coreos.github.io/butane/getting-started/) to validate and convert a Butane Config (`focs` or `flatcar`) to Ignition (for Matchbox to serve)
### Docs/Examples
* Migrate docs website to GitHub Pages ([#976](https://github.com/poseidon/matchbox/pull/976))
* Update Fedora CoreOS images and configuration ([#972](https://github.com/poseidon/matchbox/pull/972))
* Update Fedora CoreOS initrd karg for UEFI ([#978](https://github.com/poseidon/matchbox/pull/978))
* Update Flatcar Linux examples to use Ignition v3.3.0 ([#980](https://github.com/poseidon/matchbox/pull/980))
## v0.9.0
* Refresh docs and examples for Fedora CoreOS and Flatcar Linux ([#815](https://github.com/poseidon/matchbox/pull/815), [#816](https://github.com/poseidon/matchbox/pull/816))
`matchbox` is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Container Linux or Fedora CoreOS config.
`matchbox` is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Ignition config.
## Features
* Chainload via iPXE and match hardware labels
* Provision Container Linux and Fedora CoreOS (powered by [Ignition](https://github.com/coreos/ignition))
* Provision Fedora CoreOS or Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Authenticated gRPC API for clients (e.g. Terraform)
Matchbox can be installed from a binary or a container image.
* Install Matchbox on [Kubernetes](docs/deployment.md#kubernetes), on a [Linux](docs/deployment.md) host, or as a [container](docs/deployment.md#docker)
* Install Matchbox as a [binary](docs/deployment.md#matchbox-binary), as a [container image](docs/deployment.md#container-image), or on [Kubernetes](docs/deployment.md#kubernetes)
* Setup a PXE-enabled [network](docs/network-setup.md)
## Tutorials
[Getting started](docs/getting-started.md) provisioning machines with Container Linux.
Start provisioning machines with Fedora CoreOS or Flatcar Linux.
*Local QEMU/KVM
*[matchbox with Docker](docs/getting-started-docker.md)
*Clusters
* [etcd3](docs/getting-started-docker.md) - Install a 3-node etcd3 cluster
*[etcd3](https://github.com/poseidon/matchbox/tree/master/examples/terraform/etcd3-install) - Install a 3-node etcd3 cluster (terraform-based)
Notable changes image releases. The dnsmasq project [upstream](http://www.thekelleys.org.uk/dnsmasq/doc.html) has its own [changelog](http://www.thekelleys.org.uk/dnsmasq/CHANGELOG).
## v0.4.1
* Rebuild with alpine:3.6 base image
* Add EXPOSE ports 67 and 69 to Dockerfile
## v0.4.0
*`dnsmasq` package version 2.76
* Rebuild with alpine:3.5 base image to receive patches
* Update CoreOS `grub.efi` to be recent (stable, 1298.7.0)
# dnsmasq [](https://quay.io/repository/poseidon/dnsmasq)
# dnsmasq
`dnsmasq` provides a container image for running DHCP, proxy DHCP, DNS, and/or TFTP with [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html). Use it to test different network setups with clusters of network bootable machines.
The image bundles `undionly.kpxe`, `ipxe.efi`, and `grub.efi` (experimental) for chainloading BIOS and UEFI clients to iPXE.
## Usage
Run the container image as a DHCP, DNS, and TFTP service.
```sh
sudo docker run --rm --cap-add=NET_ADMIN --net=host quay.io/poseidon/dnsmasq \
Configuration arguments can be provided as flags. Check the dnsmasq [man pages](http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html) for a complete list.
| flag | description | example |
|----------|-------------|---------|
| --dhcp-range | Enable DHCP, lease given range | `172.18.0.50,172.18.0.99`, `192.168.1.1,proxy,255.255.255.0` |
| --dhcp-boot | DHCP next server option | `http://matchbox.foo:8080/boot.ipxe` |
| --enable-tftp | Enable serving from tftp-root over TFTP | NA |
| --address | IP address for a domain name | /matchbox.foo/172.18.0.2 |
## Development
Build a container image locally.
```
make docker-image
```
Run the image with Docker on the `docker0` bridge (default).
```
sudo docker run --rm --cap-add=NET_ADMIN poseidon/dnsmasq -d -q
```
Moved to [dnsmasq](https://github.com/poseidon/dnsmasq).
Migrate to [Container Linux Configs](container-linux-config.md). Cloud-Config support will be removed in the future.
Migrate to [Ignition configs](ignition.md). Cloud-Config support will be removed in the future.
CoreOS Cloud-Config is a system for configuring machines with a Cloud-Config file or executable script from user-data. Cloud-Config runs in userspace on each boot and implements a subset of the [cloud-init spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data). See the cloud-config [docs](https://coreos.com/os/docs/latest/cloud-config.html) for details.
A Container Linux Config is a YAML document which declares how Container Linux instances' disks should be provisioned on network boot and first-boot from disk. Configs can declare disk partitions, write files (regular files, systemd units, networkd units, etc.), and configure users. See the Container Linux Config [spec](https://coreos.com/os/docs/latest/configuration.html).
### Ignition
Container Linux Configs are validated and converted to *machine-friendly* Ignition configs (JSON) by matchbox when serving to booting machines. [Ignition](https://coreos.com/ignition/docs/latest/), the provisioning utility shipped in Container Linux, will parse and execute the Ignition config to realize the desired configuration. Matchbox users usually only need to write Container Linux Configs.
*Note: Container Linux directory names are still named "ignition" for historical reasons as outlined below. A future breaking change will rename to "container-linux-config".*
## Adding Container Linux Configs
Container Linux Config templates can be added to the `/var/lib/matchbox/ignition` directory or in an `ignition` subdirectory of a custom `-data-path`. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be evaluated with group metadata, selectors, and query params.
```
/var/lib/matchbox
├── cloud
├── ignition
│ └── k8s-controller.yaml
│ └── etcd.yaml
│ └── k8s-worker.yaml
│ └── raw.ign
└── profiles
```
## Referencing in Profiles
Profiles can include a Container Linux Config for provisioning machines. Specify the Container Linux Config in a [Profile](matchbox.md#profiles) with `ignition_id`. When PXE booting, use the kernel option `coreos.first_boot=1` and `coreos.config.url` to point to the `matchbox` [Ignition endpoint](api-http.md#ignition-config).
## Examples
Here is an example Container Linux Config template. Variables will be interpreted using group metadata, selectors, and query params. Matchbox will convert the config to Ignition to serve Container Linux machines.
ignition/format-disk.yaml.tmpl:
<!-- {% raw %} -->
```yaml
---
storage:
disks:
- device:/dev/sda
wipe_table:true
partitions:
- label:ROOT
filesystems:
- name:root
mount:
device:"/dev/sda1"
format:"ext4"
create:
force:true
options:
- "-LROOT"
files:
- filesystem:root
path:/home/core/foo
mode:0644
user:
id:500
group:
id:500
contents:
inline:|
{{.example_contents}}
{{if index . "ssh_authorized_keys" }}
passwd:
users:
- name:core
ssh_authorized_keys:
{{range $element := .ssh_authorized_keys }}
- {{$element}}
{{end}}
{{end}}
```
<!-- {% endraw %} -->
The Ignition config response (formatted) to a query `/ignition?label=value` for a Container Linux instance supporting Ignition 2.0.0 would be:
```json
{
"ignition":{
"version":"2.0.0",
"config":{}
},
"storage":{
"disks":[
{
"device":"/dev/sda",
"wipeTable":true,
"partitions":[
{
"label":"ROOT",
"number":0,
"size":0,
"start":0
}
]
}
],
"filesystems":[
{
"name":"root",
"mount":{
"device":"/dev/sda1",
"format":"ext4",
"create":{
"force":true,
"options":[
"-LROOT"
]
}
}
}
],
"files":[
{
"filesystem":"root",
"path":"/home/core/foo",
"contents":{
"source":"data:,Example%20file%20contents%0A",
"verification":{}
},
"mode":420,
"user":{
"id":500
},
"group":{
"id":500
}
}
]
},
"systemd":{},
"networkd":{},
"passwd":{}
}
```
See [examples/ignition](../examples/ignition) for numerous Container Linux Config template examples.
### Raw Ignition
If you prefer to design your own templating solution, raw Ignition files (suffixed with `.ign` or `.ignition`) are served directly.
This guide walks through deploying the `matchbox` service on a Linux host (via RPM, docker, or binary) or on a Kubernetes cluster.
This guide walks through deploying the `matchbox` service on a Linux host (as a binary or container image) or on a Kubernetes cluster.
## Provisoner
`matchbox` is a service for network booting and provisioning machines to create CoreOS Container Linux clusters. `matchbox` should be installed on a provisioner machine (Container Linux or any Linux distribution) or cluster (Kubernetes) which can serve configs to client machines in a lab or datacenter.
Matchbox is a service for network booting and provisioning machines to create Fedora CoreOS or Flatcar Linux clusters. Matchbox may installed on a host server or Kubernetes cluster that can serve configs to client machines in a lab or datacenter.
Verify the release has been signed by Dalton Hubble's GPG [Key](https://keyserver.ubuntu.com/pks/lookup?search=0x8F515AD1602065C8&op=vindex)'s signing subkey.
gpg: Good signature from "Dalton Hubble <dghubble@gmail.com>"
```
Untar the release.
```sh
$ tar xzvf matchbox-v0.8.0-linux-amd64.tar.gz
$ cd matchbox-v0.8.0-linux-amd64
$ tar xzvf matchbox-v0.10.0-linux-amd64.tar.gz
$ cd matchbox-v0.10.0-linux-amd64
```
## Install
### RPM-based distro
Run Matchbox as a binary, a container image, or on Kubernetes.
On an RPM-based provisioner (Fedora 24+), install the `matchbox` RPM from the Copr [repository](https://copr.fedorainfracloud.org/coprs/g/CoreOS/matchbox/) using `dnf`.
```sh
dnf copr enable @CoreOS/matchbox
dnf install matchbox
```
RPMs are not currently available for CentOS and RHEL (due to Go version). CentOS and RHEL users should follow the Generic Linux section below.
### Generic Linux
### Matchbox Binary
Pre-built binaries are available for generic Linux distributions. Copy the `matchbox` static binary to an appropriate location on the host.
Customize matchbox by editing the systemd unit or adding a systemd dropin. Find the complete set of `matchbox` flags and environment variables at [config](config.md).
Customize Matchbox by editing the systemd unit or adding a systemd dropin. Find the complete set of `matchbox` flags and environment variables at [config](config.md).
Create machine profiles, groups, or Ignition configs by adding files to `/var/lib/matchbox`.
### Kubernetes
Install Matchbox on a Kubernetes cluster with the example manifests.
```sh
$ kubectl apply -R -f contrib/k8s
$ kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
matchbox 10.3.0.145 <none> 8080/TCP,8081/TCP 46m
```
Example manifests in [contrib/k8s](../contrib/k8s) enable the gRPC API to allow client apps to update matchbox objects. Generate TLS server certificates for `matchbox-rpc.example.com` [as shown](#generate-tls-certificates) and create a Kubernetes secret. Alternately, edit the example manifests if you don't need the gRPC API enabled.
Create an Ingress resource to expose the HTTP read-only and gRPC API endpoints. The Ingress example requires the cluster to have a functioning [Nginx Ingress Controller](https://github.com/kubernetes/ingress).
`matchbox` can serve Container Linux images in development or lab environments to reduce bandwidth usage and increase the speed of Container Linux PXE boots and installs to disk.
Matchbox can serve OS images in development or lab environments to reduce bandwidth usage and increase the speed of PXE boots and installs to disk.
Download a recent Container Linux [release](https://coreos.com/releases/) with signatures.
Download a recent Fedora CoreOS or Flatcar Linux release.
Example manifests in [contrib/k8s](../contrib/k8s) enable the gRPC API to allow client apps to update matchbox objects. Generate TLS server credentials for `matchbox-rpc.example.com` [as shown](#generate-tls-credentials) and create a Kubernetes secret. Alternately, edit the example manifests if you don't need the gRPC API enabled.
Create an Ingress resource to expose the HTTP read-only and gRPC API endpoints. The Ingress example requires the cluster to have a functioning [Nginx Ingress Controller](https://github.com/kubernetes/ingress).
In this tutorial, we'll run `matchbox` on your Linux machine with Docker to network boot and provision a cluster of QEMU/KVM Container Linux machines locally. You'll be able to create Kubernetes clusters, etcd3 clusters, and test network setups.
In this tutorial, we'll run `matchbox` on a Linux machine with Docker to network boot and provision local QEMU/KVM machines as Fedora CoreOS or Flatcar Linux machines. You'll be able to test network setups and Ignition provisioning.
*Note*: To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).
!!! note
To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).
For development convenience, add `/etc/hosts` entries for nodes so they may be referenced by name.
@@ -45,10 +47,10 @@ For development convenience, add `/etc/hosts` entries for nodes so they may be r
Run the `matchbox` and `dnsmasq` services on the `docker0` bridge. `dnsmasq` will run DHCP, DNS and TFTP services to create a suitable network boot environment. `matchbox` will serve configs to machines as they PXE boot.
The `devnet` convenience script can start these services and accepts the name of any example cluster in [examples](../examples).
The `devnet` convenience script can start these services and accepts the name of any example in [examples](https://github.com/poseidon/matchbox/tree/master/examples).
```sh
$ sudo ./scripts/devnet create etcd3
$ sudo ./scripts/devnet create fedora-coreos
```
Inspect the logs.
@@ -57,7 +59,7 @@ Inspect the logs.
$ sudo ./scripts/devnet status
```
Take a look at the [etcd3 groups](../examples/groups/etcd3) to get an idea of how machines are mapped to Profiles. Explore some endpoints exposed by the service, say for QEMU/KVM node1.
Inspect the examples and Matchbox endpoints to see how machines (e.g. node1 with MAC `52:54:00:a1:9c:ae`) are mapped to Profiles, and therefore iPXE and Ignition configs.
In this tutorial, we'll show how to use terraform with `matchbox` to provision Container Linux machines.
In this tutorial, we'll use `matchbox` with Terraform to provision Fedora CoreOS or Flatcar Linux machines.
You'll install the `matchbox` service, setup a PXE network boot environment, and then use terraform configs to describe your infrastructure and the terraform CLI to create those resources on `matchbox`.
We'll install the `matchbox` service, setup a PXE network boot environment, and use Terraform configs to declare infrastructure and apply resources on `matchbox`.
## matchbox
Install `matchbox` on a dedicated server or Kubernetes cluster. Generate TLS credentials and enable the gRPC API as directed. Save the `ca.crt`, `client.crt`, and `client.key` on your local machine (e.g. `~/.matchbox`).
Install `matchbox` on a host server or Kubernetes cluster. Generate TLS credentials and enable the gRPC API as directed. Save the `ca.crt`, `client.crt`, and `client.key` on your local machine (e.g. `~/.matchbox`).
* Installing on [Container Linux / other distros](deployment.md)
* Installing on a [Linux distro](deployment.md)
* Installing on [Kubernetes](deployment.md#kubernetes)
Install [Terraform][terraform-dl] v0.11+ on your system.
Install [Terraform](https://www.terraform.io/downloads.html) v0.13+ on your system.
```sh
$ terraform version
Terraform v0.11.13
Terraform v1.1.8
```
Add the [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
Let's start with the `simple-install` example. With `simple-install`, any machines which PXE boot from matchbox will install Container Linux to `dev/sda`, reboot, and have your SSH key set. Its not much of a cluster, but we'll get to that later.
Select from the Terraform [examples](https://github.com/poseidon/matchbox/tree/master/examples/terraform). For example,
*`fedora-coreos-install` - PXE boot, install Fedora CoreOS to disk, reboot, and machines come up with your SSH authorized key set
*`flatcar-install` - PXE boot, install Flatcar Linux to disk, reboot, and machines come up with your SSH authorized key set
These aren't exactly full clusters, but they show declarations and network provisioning.
```sh
$ cdsimple-install
$ cdfedora-coreos-install # or flatcar-install
```
Configure the variables in `variables.tf` by creating a `terraform.tfvars` file.
!!! note
Fedora CoreOS images are only served via HTTPS, so your iPXE firmware must be compiled to support HTTPS downloads.
Let's review the terraform config and learn a bit about Matchbox.
Terraform can now interact with the matchbox service and create resources.
```sh
$ terraform plan
Plan: 4 to add, 0 to change, 0 to destroy.
```
Let's review the terraform config and learn a bit about matchbox.
#### Provider
### Provider
Matchbox is configured as a provider platform for bare-metal resources.
```hcl
```tf
// Configure the matchbox provider
provider"matchbox"{
endpoint="${var.matchbox_rpc_endpoint}"
client_cert="${file("~/.matchbox/client.crt")}"
client_key="${file("~/.matchbox/client.key")}"
ca="${file("~/.matchbox/ca.crt")}"
endpoint=var.matchbox_rpc_endpoint
client_cert=file("~/.matchbox/client.crt")
client_key=file("~/.matchbox/client.key")
ca=file("~/.matchbox/ca.crt")
}
```
#### Profiles
Machine profiles specify the kernel, initrd, kernel args, Container Linux Config, Cloud-config, or other configs used to network boot and provision a bare-metal machine. This profile will PXE boot machines using the current stable Container Linux kernel and initrd (see [assets](api-http.md#assets) to learn about caching for speed) and supply a Container Linux Config specifying that a disk install and reboot should be performed. Learn more about [Container Linux configs](https://coreos.com/os/docs/latest/configuration.html).
Matcher groups match machines based on labels like MAC, UUID, etc. to different profiles and templates in machine-specific values. This group does not have a `selector` block, so any machines which network boot from matchbox will match this group and be provisioned using the `coreos-install` profile. Machines are matched to the most specific matching group.
Machine profiles specify the kernel, initrd, kernel args, Ignition Config, or other configs (e.g. templated Butane Config, generic) used to network boot and provision a bare-metal machine. The profile below would PXE boot machines using a Fedora CoreOS kernel and initrd (see [assets](api-http.md#assets) to learn about caching for speed), perform a disk install, reboot (first boot from disk), and use a [Fedora CoreOS Config](https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md) to generate an Ignition config to provision.
"--name main https://builds.coreos.fedoraproject.org/prod/streams/${var.os_stream}/builds/${var.os_version}/x86_64/fedora-coreos-${var.os_version}-live-initramfs.x86_64.img"
Matcher groups match machines based on labels like MAC, UUID, etc. to different profiles and templates in machine-specific values. The group below does not have a `selector` block, so any machines which network boot from Matchbox will match this group and be provisioned using the `fedora-coreos-install` profile. Machines are matched to the most specific matching group.
Some Terraform [variables](https://www.terraform.io/docs/configuration/variables.html) are used in the examples. A quick way to set their value is by creating a `terraform.tfvars` file.
Matchbox can integrate with many on-premise network setups. It does not seek to be the DHCP server, TFTP server, or DNS server for the network. Instead, matchbox serves iPXE scripts and GRUB configs as the entrypoint for provisioning network booted machines. PXE clients are supported by chainloading iPXE firmware.
Matchbox can integrate with many on-premise network setups. It does not seek to be the DHCP server, TFTP server, or DNS server for the network. Instead, matchbox serves iPXE scripts as the entrypoint for provisioning network booted machines. PXE clients are supported by chainloading iPXE firmware.
In the simplest case, an iPXE-enabled network can chain to matchbox,
In the simplest case, an iPXE-enabled network can chain to Matchbox,
```
# /var/www/html/ipxe/default.ipxe
@@ -171,7 +188,7 @@ If you've never setup a PXE-enabled network before or you're trying to setup a h
## Boot
Its time to network boot your machines. Use the BMC's remote management capablities (may be vendor-specific) to set the boot device (on the next boot only) to PXE and power on each machine.
Its time to network boot your machines. Use the BMC's remote management capabilities (may be vendor-specific) to set the boot device (on the next boot only) to PXE and power on each machine.
```sh
$ ipmitool -H node1.example.com -U USER -P PASS power off
$ ipmitool -H node1.example.com -U USER -P PASS power on
```
Each machine should chainload iPXE, delegate to `matchbox`, receive its iPXE config (or other supported configs) and begin the provisioning process. The`simple-install` example assumes your machines are configured to boot from disk first and PXE only when requested, but you can write profiles for different cases.
Each machine should chainload iPXE, delegate to Matchbox, receive its iPXE config (or other supported configs) and begin the provisioning process. The examples assume machines are configured to boot from disk first and PXE only when requested, but you can write profiles for different cases.
Once the Container Linux install completes and the machine reboots you can SSH,
Once the install completes and the machine reboots, you can SSH.
```ssh
$ ssh core@node1.example.com
```
To re-provision the machine for another purpose, run `terraform apply` and PXE boot it again.
To re-provision the machine for another purpose, run `terraform apply` and PXE boot machines again.
## Going Further
Matchbox can be used to provision multi-node Container Linux clusters at one or many on-premise sites if deployed in an HA way. Machines can be matched individually by MAC address, UUID, region, or other labels you choose. Installs can be made much faster by caching images in the built-in HTTP [assets](api-http.md#assets) server.
Matchbox can be used to provision multi-node Fedora CoreOS or Flatcar Linux clusters at one or many on-premise sites if deployed in an HA way. Machines can be matched individually by MAC address, UUID, region, or other labels you choose. Installs can be made much faster by caching images in the built-in HTTP [assets](api-http.md#assets) server.
[Container Linux configs](https://coreos.com/os/docs/latest/configuration.html) can be used to partition disks and filesystems, write systemd units, write networkd configs or regular files, and create users. Container Linux nodes can be provisioned into a system that meets your needs. Checkout the examples which create a 3 node [etcd](../examples/terraform/etcd3-install) cluster or a 3 node [Kubernetes](../examples/terraform/bootkube-install) cluster.
[Ignition](https://github.com/coreos/ignition) can be used to partition disks, create filesystems, write systemd units, write networkd configs or regular files, and create users. Nodes can be network provisioned into a complete cluster system that meets your needs. For example, see [Typhoon](https://typhoon.psdn.io/fedora-coreos/bare-metal/).
[Ignition](https://coreos.github.io/ignition/) configs define how disks should be provisioned (on network boot and first-boot from disk) to partition disks, write files (regular files, systemd units, networkd units, etc.), and configure users. Ignition is used by:
* Fedora CoreOS
* RHEL CoreOS
* Flatcar Linux
See the Ignition Config v3.x [specs](https://coreos.github.io/ignition/specs/) for details.
## Usage
Ignition configs can be added to the `/var/lib/matchbox/ignition` directory or in an `ignition` subdirectory of a custom `-data-path`. Ignition configs must end in `.ign` or `ignition`.
```
/var/lib/matchbox
├── ignition
│ └── k8s-controller.ign
│ └── k8s-worker.ign
└── profiles
```
Matchbox Profiles can set an Ignition config for provisioning machines. Specify the Ignition config in a [Profile](matchbox.md#profiles) with `ignition_id`.
```json
{
"id":"worker",
"name":"My Profile",
"boot":{
...
},
"ignition_id":"my-ignition.ign"
}
```
When PXE booting, set kernel arguments depending on the OS (e.g. `ignition.firstboot` on FCOS, `flatcar.first_boot=yes` on Flatcar).
Point the `ignition.config.url` or `flatcar.config.url` to point to the `matchbox` [Ignition endpoint](api-http.md#ignition-config).
Matchbox parses Ignition configs (e.g. `.ign` or `.ignition`) at spec v3.3 or below and renders to the current supported version (v3.3). This relies on Ignition's [forward compatibility](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61).
## Writing Configs
Ignition configs can be prepared externally and loaded via the gRPC API, rather than writing Ignition by hand.
### Terraform
Terraform can be used to prepare Ignition configs, while providing integrations with external systems and rich templating. Using tools like [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct), you can write Butane config (an easier YAML format), validate configs, and load Ignition into Matchbox ([examples](https://github.com/poseidon/matchbox/tree/main/examples/terraform)).
Define a Butane config for Fedora CoreOS or Flatcar Linux:
```yaml
variant:fcos
version:1.5.0
passwd:
users:
- name:core
ssh_authorized_keys:
- ssh-key foo
```
```yaml
variant:flatcar
version:1.1.0
passwd:
users:
- name:core
ssh_authorized_keys:
- ssh-key foo
```
Define a `ct_config` data source with strict validation. Optionally use Terraform [templating](https://github.com/poseidon/terraform-provider-ct).
```tf
data"ct_config""worker"{
content=file("worker.yaml")
strict=true
pretty_print=false
snippets=[
file("units.yaml"),
file("storage.yaml"),
]
}
```
Then render the Butane config to Ignition and use it in a Matchbox Profile.
See the Terraform [examples](https://github.com/poseidon/matchbox/tree/main/examples#terraform-examples) for details.
### Butane
The [Butane](https://coreos.github.io/butane/) command line tool can be used to convert Butane configs (an easier YAML format) to Ignition. Then you can use the Matchbox gRPC API to upload the rendered Ignition to Matchbox for serving to machines on boot.
See [examples/ignition](../examples/ignition) for Butane config examples.
### Matchbox Rendering
While Matchbox recommends preparing Ignition configs externally (e.g. using Terraform's rich templating), Matchbox does still support limited templating and translation features with a builtin Butane converter.
Specify a Butane config in a [Profile](matchbox.md#profiles) with `ignition_id` (file must not end in `.ign` or `.ignition`).
```json
{
"id":"worker",
"name":"My Profile",
"boot":{
...
},
"ignition_id":"butane.yaml"
}
```
Here is an example Butane config with Matchbox template elements. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be interpreted using group metadata, selectors, and query params.
```yaml
variant:flatcar
version:1.1.0
storage:
files:
- path:/var/home/core/foo
mode:0644
contents:
inline:|
{{.example_contents}}
{{if index . "ssh_authorized_keys" }}
passwd:
users:
- name:core
ssh_authorized_keys:
{{range $element := .ssh_authorized_keys }}
- {{$element}}
{{end}}
{{end}}
```
Matchbox will use the Butane library to config to the current supported Ignition version. This relies on Ignition's [forward compatibility](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61).
Matchbox is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Container Linux or Fedora CoreOS config.
Matchbox is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Ignition config.
## Features
* Chainload via iPXE and match hardware labels
* Provision Container Linux and Fedora CoreOS (powered by [Ignition](https://github.com/coreos/ignition))
* Provision Fedora CoreOS or Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Authenticated gRPC API for clients (e.g. Terraform)
## Installation
Matchbox can be installed from a binary or a container image.
* Install Matchbox on [Kubernetes](deployment.md#kubernetes), on a [Linux](deployment.md) host, or as a [container](deployment.md#docker)
* Install Matchbox as a [binary](deployment.md#matchbox-binary), as a [container image](deployment.md#container-image), or on [Kubernetes](deployment.md#kubernetes)
* Setup a PXE-enabled [network](network-setup.md)
## Tutorials
[Getting started](getting-started.md) provisioning machines with Container Linux.
Start provisioning machines with Fedora CoreOS or Flatcar Linux.
*Local QEMU/KVM
*[matchbox with Docker](getting-started-docker.md)
* Clusters
* [etcd3](getting-started-docker.md) - Install a 3-node etcd3 cluster
*[etcd3](https://github.com/poseidon/matchbox/tree/master/examples/terraform/etcd3-install) - Install a 3-node etcd3 cluster (terraform-based)
@@ -128,16 +128,16 @@ Group selectors can use any key/value pairs you find useful. However, several la
### Config templates
Profiles can reference various templated configs. Ignition JSON configs can be generated from [Container Linux Config](https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/configuration.md) template files. Cloud-Config templates files can be used to render a script or Cloud-Config. Generic template files can be used to render arbitrary untyped configs (experimental). Each template may contain [Go template](https://golang.org/pkg/text/template/) elements which will be rendered with machine group metadata, selectors, and query params.
Profiles can reference various templated configs. Ignition configs can be provided directly or rendered fro [Butane Config](https://coreos.github.io/butane/) template files. Cloud-Config templates files can be used to render a script or Cloud-Config. Generic template files can be used to render arbitrary untyped configs (experimental). Each template may contain [Go template](https://golang.org/pkg/text/template/) elements which will be rendered with machine group metadata, selectors, and query params.
For details and examples:
* [Container Linux Config](container-linux-config.md)
* [Ignition (or Butane)](ignition.md)
* [Cloud-Config](cloud-config.md)
#### Variables
Within Container Linux Config templates, Cloud-Config templates, or generic templates, you can use group metadata, selectors, or request-scoped query params. For example, a request `/generic?mac=52-54-00-89-d8-10&foo=some-param&bar=b` would match the `node1.json` machine group shown above. If the group's profile ("etcd") referenced a generic template, the following variables could be used.
Within Butane Config templates, Cloud-Config templates, or generic templates, you can use group metadata, selectors, or request-scoped query params. For example, a request `/generic?mac=52-54-00-89-d8-10&foo=some-param&bar=b` would match the `node1.json` machine group shown above. If the group's profile ("etcd") referenced a generic template, the following variables could be used.
<!-- {% raw %} -->
```
@@ -172,7 +172,7 @@ matchbox.foo/assets/
For example, a `Profile` might refer to a local asset `/assets/coreos/VERSION/coreos_production_pxe.vmlinuz` instead of `http://stable.release.core-os.net/amd64-usr/VERSION/coreos_production_pxe.vmlinuz`.
See the [get-coreos](https://github.com/poseidon/matchbox/blob/master/scripts/get-coreos) script to quickly download, verify, and place Container Linux assets.
See the [get-fedora-coreos](https://github.com/poseidon/matchbox/blob/master/scripts/get-fedora-coreos) or [get-flatcar](https://github.com/poseidon/matchbox/blob/master/scripts/get-flatcar) scripts to quickly download, verify, and place image assets.
This guide shows how to create a DHCP/TFTP/DNS network boot environment to boot and provision BIOS/PXE, iPXE, or UEFI client machines.
Matchbox serves iPXE scripts over HTTP to serve as the entrypoint for provisioning clusters. It does not implement or exec a DHCP, TFTP, or DNS server. Instead, configure your network environment to point to Matchbox or use the convenient [poseidon/dnsmasq](../contrib/dnsmasq) container image (used in local QEMU/KVM setup).
Matchbox serves iPXE scripts over HTTP to serve as the entrypoint for provisioning clusters. It does not implement or exec a DHCP, TFTP, or DNS server. Instead, configure your network environment to point to Matchbox or use the convenient [quay.io/poseidon/dnsmasq](https://quay.io/repository/poseidon/matchbox) container image (used in local QEMU/KVM setup).
*Note*: These are just suggestions. Your network administrator or system administrator should choose the right network setup for your company.
See [dnsmasq](#coreosdnsmasq) below to run dnsmasq with a container.
See [dnsmasq](#poseidon/dnsmasq) below to run dnsmasq with a container.
### Configurable TFTP
@@ -158,7 +158,7 @@ Add ipxe.lkrn to `/var/lib/tftpboot` (see [iPXE docs](http://ipxe.org/embed)).
## poseidon/dnsmasq
The [quay.io/poseidon/dnsmasq](https://quay.io/repository/poseidon/dnsmasq) container image can run DHCP, TFTP, and DNS services via docker. The image bundles `ipxe.efi`, `undionly.kpxe`, and `grub.efi` for convenience. See [contrib/dnsmasq](../contrib/dnsmasq) for details.
The [quay.io/poseidon/dnsmasq](https://quay.io/repository/poseidon/dnsmasq) container image can run DHCP, TFTP, and DNS services via docker. The image bundles `ipxe.efi`, `undionly.kpxe`, and `grub.efi` for convenience. See [contrib/dnsmasq](https://github.com/poseidon/matchbox/tree/master/contrib/dnsmasq) for details.
Matchbox automates network booting and provisioning of clusters. These examples show how to use matchbox on-premise or locally with [QEMU/KVM](scripts/README.md#libvirt).
Matchbox automates network booting and provisioning of clusters. These examples show how to use Matchbox on-premise or locally with QEMU/KVM.
## Terraform Examples
@@ -8,46 +8,55 @@ These examples use [Terraform](https://www.terraform.io/intro/) as a client to M
| [simple-install](terraform/simple-install/) | Install Container Linux with an SSH key |
| [etcd3-install](terraform/etcd3-install/) | Install a 3-node etcd3 cluster |
| [fedora-coreos-install](terraform/fedora-coreos-install) | Fedora CoreOS disk install |
| [flatcar-install](terraform/flatcar-install) | Flatcar Linux disk install |
### Customization
You are encouraged to look through the examples and Terraform modules. Implement your own profiles or package them as modules to meet your needs. We've just provided a starting point. Learn more about [matchbox](../docs/matchbox.md) and [Container Linux configs](../docs/container-linux-config.md).
Look through the examples and Terraform modules and use them as a starting point. Learn more about [matchbox](../docs/matchbox.md).
## Manual Examples
These examples mount raw Matchbox objects into a Matchbox server's `/var/lib/matchbox/` directory.
| Name | Description | CoreOS Container Linux Version | FS | Docs |
| flatcar | Flatcar Linux live PXE | RAM | [docs](https://docs.flatcar-linux.org/os/booting-with-ipxe/) |
| flatcar-install | Flatcar Linux install | Disk | [docs](https://docs.flatcar-linux.org/os/booting-with-ipxe/) |
### Customization
### SSH Access
#### Autologin
For Fedora CoreOS, add an SSH authorized key to the Butane Config (`ignition/fedora-coreos.yaml`) and regenerate the Ignition Config.
Example profiles pass the `coreos.autologin` kernel argument. This skips the password prompt for development and troubleshooting and should be removed **before production**.
Example groups allow `ssh_authorized_keys` to be added for the `core` user as metadata. You might also include this directly in your Ignition.
For Flatcar Linux, add an SSH authorizedkey to the Butane config (`ignition/flatcar.yaml` or `ignition/flatcar-install.yaml`) and regenerate the Ignition Config.
Some examples check the `pxe` variable to determine whether to create a `/dev/sda1` filesystem and partition for PXEing with `root=/dev/sda1` ("pxe":"true") or to write files to the existing filesystem on `/dev/disk/by-label/ROOT` ("pxe":"false").
* Terraform v0.9+ and [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox) installed locally on your system
* 3 machines with known DNS names and MAC addresses
If you prefer to provision QEMU/KVM VMs on your local Linux machine, set up the matchbox [development environment](../../../docs/getting-started-docker.md).
```sh
sudo ./scripts/devnet create
```
## Usage
Clone the [matchbox](https://github.com/poseidon/matchbox) project and take a look at the cluster examples.
Configs in `etcd3-install` configure the matchbox provider, define profiles (e.g. `cached-container-linux-install`, `etcd3`), and define 3 groups which match machines by MAC address to a profile. These resources declare that the machines should PXE boot, install Container Linux to disk, and provision themselves into peers in a 3-node etcd3 cluster.
Note: The `cached-container-linux-install` profile will PXE boot and install Container Linux from matchbox [assets](https://github.com/poseidon/matchbox/blob/master/docs/api.md#assets). If you have not populated the assets cache, use the `container-linux-install` profile to use public images (slower).
### Optional
You may set certain optional variables to override defaults.
```hcl
# install_disk = "/dev/sda"
# container_linux_oem = ""
```
## Apply
Fetch the [profiles](../README.md#modules) Terraform [module](https://www.terraform.io/docs/modules/index.html) which let's you use common machine profiles maintained in the matchbox repo (like `etcd3`).
```sh
$ terraform get
```
Plan and apply to create the resoures on Matchbox.
Power on each machine (with PXE boot device on next boot). Machines should network boot, install Container Linux to disk, reboot, and provision themselves as a 3-node etcd3 cluster.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
