* The reusable wait-for-status-check GitHub Workflow is only used
in public repos which can use actions for free. So we might as
well use GitHub-managed runners
* Motived by re-evaluating runner usage in response to GitHub
pricing changes. Shift costs to Microsoft where possible
* Add wait-for-status-checks reusable GitHub Workflow to wait
for all status checks in a PR. Mark this as a required check
so that GitHub auto-merge can be used
Rel: https://github.com/poseidon/wait-for-status-checks/
* Remove dependency on github.com/coreos/pkg because the module
is no longer maintained and it doesn't follow the usual release
conventions
* The last time flagutils was touched as 9 years ago by me. Bring
the function into the matchbox repo
* A few years ago there was a SVE for gopkg.in/yaml.v2 so we used
a replace to force MVS to pick a patched version of the package.
However, these days, no indirect dependencies use gopkg.in/yaml.v2
at all, they use gopkg.in/yaml.v3 so we can remove this
* We now use GitHub Actions instead of a private Drone which is great
for OSS build transparency, but one drawback to that change is that
we no longer auto-deploy matchbox to internal Poseidon Labs infra
* We don't want public workflows for deploys to our private infra
Some commands were added in the ipxeBootstrap to enable architecture detection based on the architecture the iPXE version used was built for.
This is done through the use of an additional flag in the chain which is the ipxeBootstrap using the buildarch parameter.
* Add a publish workflow that uses the mkdocs-pages re-usable GitHub
Workflow when the `release-docs` branch is updated
* Build the docs site with mkdocs and publish to GitHub Pages
* Migrate from the internal Drone server using a GitHub Workflow
to perform the multi-arch container image build
* Use self-hosted GitHub runners on ARM64 to perform the ARM64
build step faster that QEMU/KVM emulation
* Mandate approval for all workflow runs from outside contributors
since the builds use push credentials and partially run internally
* Regenerate example Ignition using fcct v0.18.0 to produce Ignition
spec v3.4.0 as a followup to https://github.com/poseidon/matchbox/pull/1079
* Fix one profile example where a double escape isn't needed
* Emphasize that Container Linux Configs have been dropped and
that Butane is the way forward to use modern Ignition v2
* Include some links to Flatcar Linux docs
* Update Makefile and docs for v0.10.0 release process
* Update github.com/coreos/go-semver from v0.3.0 to 167f5da to
prevent it from using gopkg.in/yaml.v2 (CVE-2019-11254)
```
go mod why gopkg.in/yaml.v2
(main module does not need package gopkg.in/yaml.v2)
```
* Recommend preparing Ignition configs external to Matchbox (e.g.
with Terraform poseidon/terraform-provider-ct)
* Document that Matchbox Rendering is discouraged
* Add Butane Config support as a replacement for dropping Container
Linux Config support. Perform Matchbox Go template evaluation,
translation to Ignition, and parsing to a forward compatible version
* Both Flatcar Linux and Fedora CoreOS now support Ignition v2
(spec v3.x) for machine consumption
* Drop support for Ignition v0.35.0 (Ignition spec v2.x)
* Drop support for Container Linux Configs (the YAML format that
rendered to Ignition v2.x JSON for Flatcar Linux)
* Deprecate rendering Container Linux Configs
* Use tools like [poseidon/ct](https://github.com/poseidon/terraform-provider-ct) or
[butane](https://coreos.github.io/butane/getting-started/) to validate and convert a
Butane Config (`focs` or `flatcar`) to Ignition (for Matchbox to serve)
* Please migrate to serving CoreOS Ignition directly, the Container
Linux related HTTP and gRPC endpoints will be removed in future
* Discontinue using Matchbox's Container Linux Config features
* Flatcar Linux OS now supports Ignition v2.13+ which means it
can accept Ignition v3.x spec's, like Fedora CoreOS. Matchbox
supports this by serving Ignition documents directly
* Users of the poseidon/matchbox Terraform provider can pass
a `matchbox_profile` `raw_ignition` contents with the desired
Ignition v3.3 spec
* Users of the poseidon/ct Terraform provider can write Butane
Config YAML, perform templating, and render an Ignition document
using either the fcos or the flatcar variant
* Default branch was renamed from master to main
* Internal build and publishing workflows were unaffected,
the GitHub Action is just for public contribution vetting
* Add initrd=main karg directive for UEFI (ignored by BIOS)
* Update Butane config version to v1.4.0 (generates Ignition v3.3.0)
in `fedora-coreos` and `fedora-coreos-install` examples
* Update virt-install flag --os-variant
* Removed virt-install deprecated flag --os-type
* Remove virt-install QEMU/KVM event preserve since it apparently
wasn't implemented anyway and QEMU/KVM now warns about it
* Remove serial consol kernel argument from examples, but still
mention it in docs
Rel:
* https://github.com/coreos/fedora-coreos-docs/pull/282
* https://www.spinics.net/linux/fedora/libvir/msg222078.html
We still support passing the rootfs image as an appended initrd, but we
recommend using coreos.live.rootfs_url instead: it generally boots faster
and requires less RAM.
When coreos-installer is running from the live image, it no longer needs
a separate install image, since by default it installs from content
embedded in the live system.
* Looking at enabling some update automation, the clarity of
using non-vendored Go modules/checksums outweighs the risk
of a module disappearing (mitigated by proxies)
* Update Fedora CoreOS live PXE and disk install examples to
Fedora 33
* Increase libvirt VM memory from 2GB to 3GB to support live
PXE example, which is mostly just for laptop examples/demos.
Reduce the VM count from 3 to 2 to compensate.
* Change `fedora-coreos.ign` to suggest using an ed25519 SSH
key since Fedora CoreOS 33 disables RSA SHA1 (256 is still ok
but most people won't know which they have)
* Update Terraform examples to use Terraform v0.13
* Refresh examples to show Fedora CoreOS and Flatcar Linux
* Remove the etcd3 example, not worth it to maintain
* Refactor examples to boot provision minimal hosts
with Fedora CoreOS or Flatcar Linux
* Remove the etcd3 cluster example or other specific
kinds of hosts
* Update script get-fedora-coreos
* Remove script get-coreos
* Matchbox is published as a binary or container image,
not planning to resume RPM/Copr publishing since its a
pain
* Publishing to Copr repo stopped in v0.6 (3 years ago)
* Use our own infra to perform image builds and push images. Provides
future options for publishing images to multiple image registries and
for multiple architectures, while keeping push permissions in-house
* Remove Travis ability to push to Quay
* Consider splitting repo and also enabling Quay automated builds
* Use our own infra to perform image builds and push images.
Provides future options for publishing images to multiple
image registries and for multiple architectures, while keeping
push permissions in-house
* Remove Travis ability to push to Quay
* Use Travis only for Go tests of Pull Requests
* Use `quay.io/dghubble/protoc` as the standard codegen
environment across projects. Pin a version (v3.10.1)
* Generated code is now automatically ignored by golint
based on the comment header, which will simplify linting
* Upgrade protobuf from 2bba0603135d to v1.3.2
* Upgrade grpc from v1.2.1 to v1.25.1
* Remove `get-protoc` and `codegen` scripts
* Travis creates a Go v1.13.4 environment, checks out the source
repo, and _then_ executes the `install` block to install `golint`
* With module-aware Go, this means installing `golint` caused a
diff, before the Matchbox `make` target was invoked. Builds were
correctly identified as "dirty" as a result
* Release tags v0.8.1 and v0.8.2 have been removed to avoid any
confusion. Container images with binaries considered dirty have
been removed as well
* rkt achieved its mission creating the OCI standard,
pluggable runtimes, and prevented Docker world
domination. We can now proceed with disarmament
* Few, if any, developers still use rkt locally
* Matchbox examples should be simple and educational to
show how to PXE provision machines into clusters. Today,
these goals are achieved well enough by the 3-node etcd
cluster example
* Several years ago, I put together examples PXE booting
Kubernetes clusters with Matchbox. That was before we wrote
Tectonic or Kubernetes was as popular as it is. Today, a
Kubernetes distro is a project in its own right. It no
longer makes sense to maintain (duplicate) a Kubernetes
distro as "an example" inside Matchbox.
* Matchbox is now used for Kubernetes cluster provisioning in
more organizations than ever. It backs the poseidon/Typhoon and
kinvolk/Locomotive distros. These both serve as great external
examples of using Matchbox to provision Kubernetes clusters
Attention: If you relied on Matchbox Kubernetes docs, you can
find a similar guide at https://typhoon.psdn.io/cl/bare-metal/
(same author). https://github.com/poseidon/typhoon/
* Remove files that are unused or refer to CoreOS policies
or points of contact that no longer apply
* Specifically, CoreOS (and by extension Red Hat) no longer
manages this project, events, etc.
* Update base image from alpine:3.6 to alpine:3.9
* Automatically publish dnsmasq image on merge to a branch
named "dnsmasq". Similar to how matchbox is automatically
published from master. Set the version based on git SHA
* Matchbox has moved to a new home in Poseidon
* Update Makefile so container image name uses
poseidon instead of coreos
* Publish container images to quay.io/poseidon/matchbox
* Render Container Linux Configs referenced in Profiles
as Ignition v2.2.0 documents.
* Recall, configs suffixed with .ign/.ignition will be
served as-is, as raw Ignition. Parse warnings will be
shown
* Serve Ignition configs (ending in .ign/.ignition) with v2.1
or v2.2 formats (previously, configs above v2.1 produced warnings
that the config was too new)
* Use Go 1.11 modules with `GO111MODULE=on`
* Change `make vendor` target to call `go mod vendor`
* Enforce builds and tests use the vendor directory by setting
the `-mod=vendor` flag (notice, travis does not fetch)
* Remove glide requirement and glide files
* Squid proxy docs were added as a draft in 2017 to show a
containerized setup for caching images for network boot
environments
* These docs never matured to a point of viability, I don't
make use of squid, and they're unmaintained. Users would be
better served consulting the squid project
* Local QEMU/KVM tutorials use either docker or rkt/acbuild,
but they're dated. rkt and acbuild are no longer in mainstream
use since they successfully drove OCI standardization
* Remove the rkt examples, as they are just more difficult for
newcomers to use and seldom maintained at this point
* Retain Docker examples, although podman will likely supplant
all docker usage in future
* Upstream author changed case of his/her username (breaking)
* Fix to lowercase name so clients using Go modules can avoid
case-sensitive conflicts. Go modules do not handle this real-world
case well
* Define the required versions of terraform-provider-matchbox
and terraform-provider-ct, with install instructions
* Refresh the tutorial for using Matchbox to PXE boot local
QEMU/KVM machines (using Terraform as a client)
* Add ipxe.efi to dnsmasq image's /var/lib/tftpboot directory
* Add initrd kernel argument respected only by UEFI
https://github.com/coreos/bugs/issues/1239
* Improve network-setup docs and scripts to cover UEFI clients
and to support launching UEFI QEMU/KVM clusters locally
* Reduce references to grub.efi flow, its not a happy path
* Upstream fixes to bump all control plane components to v1.7.5
* Stop including etcd-network-checkpointer with on-host etcd
* Remove experimental_self_hosted_etcd support
* Starting in Go 1.7, the standard library http.Request includes
a Context for passing request-scoped values between chained handlers
* Delete the ContextHandler (breaking, should not have been
exported to begin with)
* Mount host /opt/cni/bin in Kubelet to use host's CNI plugins
* Switch /var/run/kubelet-pod.uuid to /var/cache/kubelet-pod.uuid
to persist between reboots and cleanup old Kubelet pods
* Organize Kubelet flags in alphabetical order
In virt-install v1.4.2, the meaning of `--pxe` changed from "allow pxe
boot" to "always pxe boot." This breaks matchbox, since we expect hosts
to pxe-boot only with empty hds. On hosts with v1.4.2, the VMs loop,
re-installing CL over and over.
The flag isn't necessary anyways, since we pass `--boot=hd,network`,
which enables pxe-booting.
* Provide better errors to clients that forget to specify the
port or include a protocol scheme by mistake
* grpc-go uses net.SplitHostPort to validate server listener
addresses are 'host:port', but doesn't validate Dial targets
etcd examples set ETCDCTL_API=3 but are using v2 etcdctl commands. This
works on CL by accident because it ships with 2.3 so etcdctl doesn't
recognize the API env var.
* Users should deploy the Container Linux Update Operator to coordinate
reboots of Container Linux nodes in a Kubernetes cluster
* Write cluster addon docs to describe CLUO
* Terraform modules `bootkube` and `profiles` (Kubernetes) disable
locksmithd
* Use the container linux update operator to coordinate reboots
* Stop using locksmithd for reboot coordination
* etcd TLS assets now only need to be distributed to controller
nodes which are etcd peers
`After=network-online.target` *should* mean this isn't needed in most
cases, but per
https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/, the
definition of "network-online" is a little shaky.
Regardless, being a little more resilient to network flakes and races is
a good thing. The count of `10` was arbitrarily chosen.
* Ensure that Matchbox (open-source) and Tectonic (enterprise)
are kept separate, Tectonic has its own docs
* Matchbox is agnostic to Kubernetes distribution
* Fixes a regression introduced in 6f02107 which upgraded to
Kubernetes v1.6.6 and added self-hosted etcd with TLS
* Both on-host and self-hosted etcd now require clients to use
TLS client certs so locksmithd
* Upgrade to bootkube v0.4.5
* Enable TLS for experimental self-hosted etcd
* Upstream manifest generation changes modify the flannel
Daemonset, switch several control plane components to run
as non-root, and add an explicit UpdateStrategy to the
control plane components
* Add create, status, and destroy subcommands that use docker as
the container runtime for testing local QEMU/KVM clusters. Before,
only rkt could be used.
* Update local QEMU/KVM tutorial documentation
gRPC API verification step has invalid paths to client.crt and client.key; these are created in ~/matchbox-v0.6.1-linux-amd64/scripts/tls (depending on where the matchbox installer is extracted).
Matchbox added generic template support to enable experimenting with
rendering different kinds of templates, beyond Container Linux configs
and cloud-configs. We'd like to add a gRPC endpoint for generic
templates, as is done for other configs to support gRPC clients.
* Module "profiles" provides container-linux-install and
cached-container-linux-install Profiles
* Module bootkube accepts cached_install variable to determine
whether the cluster should install Container Linux from cache
or from the public download site (default)
* Static Kubernetes / rktnetes examples are no longer going to be
maintained by this repo or upgraded to Kubernetes v1.6. This is not
considered a deprecation bc the reference clusters are examples.
* Remove static Kubernetes cluster examples so users don't choose it
* Self-hosted Kubernetes (bootkube) is now the standard recommended
Kubernetes cluster configuration
* Listing pods or nodes as the final step of cluster creation should
not fail the entire build, its mainly for a pretty output
* There is no official definition of when a Kubernetes cluster is
"done" bootstrapping, they can momentarily fail to response in the
first minute or so as components stabalize
* Calculate the required service IP values from the service CIDR
* These inputs were never truly customizable anyway since bootkube
start assumed the 1st, 10th, and 15th offsets for named services
* Use the dghubble/bootkube-terraform terraform module to generate
the exact same assets that `bootkube render` would
* Use terraform to automate the kubeconfig copy and bootkube start
* Removes the reuqirement to download a bootkube binary, render assets,
and manually copy assets to nodes
Fixing the source URL format to confirm to more normative rpmbuild
standards and to allow for proper use of spectool/rpmspectool. This
change now produces a proper archive with the name and version number
used.
* Add an option to try experimental self-hosted etcd which uses
the etcd-operator to deploy an etcd cluster as pods atop Kubernetes
and disables the on-host etcd cluster
* When enabled, configure locksmithd to coordinate reboots through
self-hosted etcd
* Add matchbox_http_endpoint and matchbox_rpc_endpoint as variables
* Remove dghubble ssh public key from default
* Add a terraform.tfvars.example and gitignore terraform.tfvars
* Add container-linux-install profile to install Container Linux
* Add cached-container-linux-install profile to install Container Linux
from cached matchbox assets
Previously, the terraform readme was incomplete by only including
terraform plan and apply commands. Additionally, the readme was
updated to include instructions for updating the profiles module
source.
Fixes#502
* Terraform example typo's the port number in the etcd_endpoints
* Causes worker etcd-gateway to fail so Container Linux updates may
not been coordinated by locksmith
* Add grub.efi to get-tftp-files script. This matches prior
dnsmasq images, but was not part of a repeatable build
* Switch rkt run examples to pull from quay.io
* Remove script using acbuild to create ACIs
* Stop shipping or mentioning bootcmd CLI tool
* bootcmd has not been built out into a user-facing tool
* terraform-provider-matchbox addresses some of the need
* Keep bootcmd implementation as an example matchbox gRPC client
* Show matchbox deployment, service, and TLS secret creation
* Provide an Ingress resource for exposing HTTP and gRPC APIs
* Add note mentioning matchbox can be run publicly if best practices
are followed
* 1.8 changed the behavior of url.Parse so it is no longer
appropriate for parsing values like 0.0.0.0:8080
* Pass the address directly to http.ListenAndServe which gives
reasonable errors when bad values are given
Unless `GOARM` is specified, the Go compiler will choose the compiling system's architecture as a target (similar to GCC's `-march=native`). This commit prevents that by explicitly selecting ARMv6 (e.g. Raspberry Pi 1) as the target.
According to the documentation, ARM64 does not need this, as ARMv8 is implicit and the value of `GOARM` is not considered for this architecture.
See-Also: https://github.com/golang/go/wiki/GoArm
* Move toward a unified container image which is run
either by rkt or docker
* Sadly, image signing only supported by rkt and is not
part of the new standard
* Add Profile 'args' field as a list of kernel args
* Deprecate 'cmdline' field map of kernel args
* Add missing console=tty0 console=ttyS0 kernel args
to all example clusters
* Show `virsh console nodeN` command for development
with local QEMU/KVM nodes
* Focus on real-world, varied network environments
and flexibility, and bare-metal Tectonic
* iPXE provides better hardware introspection than being
limited to MAC
* ISC DHCP and dnsmasq provides production DHCP service
* Drop Pixiecore from the setups we can support or recommend
<!-- READ: Issues are used to receive focused bug reports from users and to track planned future enhancements by the authors. Topics like support, debugging help, advice, and operation are out of scope and should not use issues-->
**Description**
A clear and concise description of what the bug is.
**Steps to Reproduce**
Provide clear steps to reproduce the bug.
- [ ] Relevant error messages if appropriate (concise, not a dump of everything).
**Expected behavior**
A clear and concise description of what you expected to happen.
* Release: Matchbox version or Git SHA (reporting latest is **not** helpful)
**Possible Solution**
<!-- Most bug reports should have some inkling about solutions. Otherwise, your report may be less of a bug and more of a support request (see top).-->
* Update butane from v0.17.0 to v0.18.0 ([#1079](https://github.com/poseidon/matchbox/pull/1079))
* Add support for `fcos` [v1.5.0](https://coreos.github.io/butane/config-fcos-v1_5/) Butane Configs
* Add support for `flatcar` [v1.1.0](https://coreos.github.io/butane/config-flatcar-v1_1/) Butane Configs
* Render Ignition as Ignition spec [v3.4.0](https://coreos.github.io/ignition/configuration-v3_4/)
## v0.10.0
* Remove support for Ignition v0.35.0 (Ignition spec v2.x)
* Remove support for Container Linux Configs (**action required**)
* Container Linux Configs were a YAML format that rendered to Ignition (spec v2.x)
* Flatcar Linux now supports Ignition v2 (spec v3.x)
* Butane is a suitable YAML format that renders Ignition v2 (spec v3.x)
* Upgrade Ignition from v0.35.0 (spec v2.x) to v2.14.0 (spec v3.x)
* Update Go version (v1.20.2) and alpine base image (v3.17.3)
* Add limited support for Matchbox rendering Butane configs ([#997](https://github.com/poseidon/matchbox/pull/997)) ([docs](https://matchbox.psdn.io/ignition/#matchbox-rendering))
* Recommend writing Butane via external tools (**action required**)
* For Terraform, use [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct)
* For a CLI, use [`butane`](https://github.com/coreos/butane)
* Parse Ignition and render forward to Ignition v2 (spec v3.3)
* Ignition is [forward](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61) compatible (e.g. a `v3.1` spec can be rendered as `v3.3` safely)
If you still template Container Linux Configs via Matchbox, [migrate](https://www.flatcar.org/docs/latest/provisioning/config-transpiler/) to Butane by prepending:
```yaml
variant:flatcar
version:1.0.0
```
## v0.9.1
* Add dependabot Go module update automation ([#833](https://github.com/poseidon/matchbox/pull/833))
* Update Go version (v1.18.4) and alpine base image (v3.16.1)
* Move `dnsmasq` container image to its own [repo](https://github.com/poseidon/dnsmasq) ([#840](https://github.com/poseidon/matchbox/pull/840))
* Deprecate rendering Container Linux Configs
* Please migrate to serving CoreOS Ignition directly
* Use tools like [poseidon/ct](https://github.com/poseidon/terraform-provider-ct) or [butane](https://coreos.github.io/butane/getting-started/) to validate and convert a Butane Config (`focs` or `flatcar`) to Ignition (for Matchbox to serve)
### Docs/Examples
* Migrate docs website to GitHub Pages ([#976](https://github.com/poseidon/matchbox/pull/976))
* Update Fedora CoreOS images and configuration ([#972](https://github.com/poseidon/matchbox/pull/972))
* Update Fedora CoreOS initrd karg for UEFI ([#978](https://github.com/poseidon/matchbox/pull/978))
* Update Flatcar Linux examples to use Ignition v3.3.0 ([#980](https://github.com/poseidon/matchbox/pull/980))
## v0.9.0
* Refresh docs and examples for Fedora CoreOS and Flatcar Linux ([#815](https://github.com/poseidon/matchbox/pull/815), [#816](https://github.com/poseidon/matchbox/pull/816))
* Publish Matchbox images from internal infra to Quay (`quay.io/poseidon/matchbox`)
* Update Go version from v1.13.4 to v1.14.9
* Update base image from `alpine:3.10` to `alpine:3.12` ([#784](https://github.com/poseidon/matchbox/pull/784))
* Include `contrib/k8s` in release tarballs ([#788](https://github.com/poseidon/matchbox/pull/788))
* Remove outdated systemd units ([#817](https://github.com/poseidon/matchbox/pull/817))
* Remove RPM spec file (Copr publishing stopped in v0.6)
## v0.8.3
* Publish docs to [https://matchbox.psdn.io](https://matchbox.psdn.io/) ([#769](https://github.com/poseidon/matchbox/pull/769))
* Update Go version from v1.11.7 to v1.13.4 ([#766](https://github.com/poseidon/matchbox/pull/766), [#770](https://github.com/poseidon/matchbox/pull/770))
* Update container image base from `alpine:3.9` to `alpine:3.10` ([#761](https://github.com/poseidon/matchbox/pull/761))
* Include `get-fedora-coreos` convenience script ([#763](https://github.com/poseidon/matchbox/pull/763))
* Remove rkt tutorials and docs ([#765](https://github.com/poseidon/matchbox/pull/765))
## v0.8.1 - v0.8.2
Releases `v0.8.1` and `v0.8.2` were not built cleanly
* Release tags and container images have been removed
* Caused by go get golint (module-aware) mutating `go.mod` on Travis (see [#775](https://github.com/poseidon/matchbox/pull/775), [#777](https://github.com/poseidon/matchbox/pull/777))
## v0.8.0
* Transfer Matchbox repo from coreos to poseidon GitHub Org
* Publish container images at [quay.io/poseidon/matchbox](https://quay.io/repository/poseidon/matchbox)
* Build Matchbox with Go v1.11.7 for images and binaries
* Update container image base from alpine:3.6 to alpine:3.9
* Render Container Linux Configs as Ignition v2.2.0
* Validate raw Ignition configs with the v2.2 spec (warn-only)
* Fix warnings that v2.2 configs are too new
Note: Release signing key [has changed](https://github.com/poseidon/matchbox/blob/v0.8.0/Documentation/deployment.md) with the project move.
### Examples
* Update Kubernetes example clusters to v1.14.1 (Terraform-based)
## v0.7.1 (2018-11-01)
* Add `kernel_args` variable to the terraform bootkube-install cluster definition
* Add `get-flatcar` helper script
* Add optional TLS support to read-only HTTP API
* Build Matchbox with Go 1.11.1 for images and binaries
### Examples
* Upgrade Kubernetes example clusters to v1.10.0 (Terraform-based)
* Upgrade Kubernetes example clusters to v1.8.5
## v0.7.0 (2017-12-12)
* Add gRPC API endpoints for managing generic (experimental) templates
* Update Container Linux config transpiler to v0.5.0
* Update Ignition to v0.19.0, render v2.1.0 Ignition configs
* Drop support for Container Linux versions below 1465.0.0 (breaking)
* Build Matchbox with Go 1.8.5 for images and binaries
* Remove Profile `Cmdline` map (deprecated in v0.5.0), use `Args` slice instead
* Remove pixiecore support (deprecated in v0.5.0)
* Remove `ContextHandler`, `ContextHandlerFunc`, and `NewHandler` from the `matchbox/http` package.
### Examples / Modules
* Upgrade Kubernetes example clusters to v1.8.4
* Kubernetes examples clusters enable etcd TLS
* Deploy the Container Linux Update Operator (CLUO) to coordinate reboots of Container Linux nodes in Kubernetes clusters. See the cluster [addon docs](Documentation/cluster-addons.md).
* Kubernetes examples (terraform and non-terraform) mask locksmithd
* Terraform modules `bootkube` and `profiles` (Kubernetes) mask locksmithd
## v0.6.1 (2017-05-25)
* Improve the installation documentation
* Move examples/etc/matchbox/cert-gen to scripts/tls
* Build Matchbox with Go 1.8.3 for images and binaries
### Examples
* Upgrade self-hosted Kubernetes cluster examples to v1.6.4
* Add NoSchedule taint to self-hosted Kubernetes controllers
* Remove static Kubernetes and rktnetes cluster examples
## v0.6.0 (2017-04-25)
* New [terraform-provider-matchbox](https://github.com/coreos/terraform-provider-matchbox) plugin for Terraform users!
* New hosted [documentation](https://coreos.com/matchbox/docs/latest) on coreos.com
* Add `ProfileDelete`, `GroupDelete`, `IgnitionGet` and `IgnitionDelete` gRPC endpoints
* Build matchbox with Go 1.8 for container images and binaries
* Generate code with gRPC v1.2.1 and matching Go protoc-gen-go plugin
* Update Ignition to v0.14.0 and coreos-cloudinit to v1.13.0
* Update "fuze" docs to the new name [Container Linux Configs](https://coreos.com/os/docs/latest/configuration.html)
* Remove `bootcmd` binary from release tarballs
### Examples
* Upgrade Kubernetes v1.5.5 (static) example clusters
* Upgrade Kubernetes v1.6.1 (self-hosted) example cluster
* Use etcd3 by default in all clusters (remove etcd2 clusters)
* Add Terraform examples for etcd3 and self-hosted Kubernetes 1.6.1
## v0.5.0 (2017-01-23)
* Rename project to CoreOS `matchbox`!
* Add Profile `args` field to list kernel args
* Update [Fuze](https://github.com/coreos/container-linux-config-transpiler) and [Ignition](https://github.com/coreos/ignition) to v0.11.2
* Switch from `golang.org/x/net/context` to `context`
* Deprecate Profile `cmd` field map of kernel args
* Deprecate Pixiecore support
* Drop build support for Go 1.6
#### Rename
* Move repo from github.com/coreos/coreos-baremetal to github.com/coreos/matchbox
* Rename `bootcfg` binary to `matchbox`
* Rename `bootcfg` packages to `matchbox`
* Publish a `quay.io/coreos/matchbox` container image. The `quay.io/coreos/bootcfg` image will no longer be updated.
* Rename environment variable prefix from `BOOTCFG*` to `MATCHBOX*`
* Change config directory to `/etc/matchbox`
* Change default `-data-path` to `/var/lib/matchbox`
* Change default `-assets-path` to `/var/lib/matchbox/assets`
#### Examples
* Upgrade Kubernetes v1.5.1 (static) example clusters
* Upgrade Kubernetes v1.5.1 (self-hosted) example cluster
* Switch Kubernetes (self-hosted) to run flannel as pods
* Combine rktnetes Ignition into Kubernetes static cluster
#### Migration
* binary users should install the `matchbox` binary (see [installation](Documentation/deployment.md))
* rkt/docker users should start using `quay.io/coreos/matchbox` (see [installation](Documentation/deployment.md))
* RPM users should uninstall bootcfg and install matchbox (see [installation](Documentation/deployment.md))
* Move `/etc/bootcfg` configs and certificates to `/etc/matchbox`
* Move `/var/lib/bootcfg` data to `/var/lib/matchbox`
* See the new [contrib/systemd](contrib/systemd) service examples
* Remove the old `bootcfg` user if you created one
## v0.4.2 (2016-12-7)
#### Improvements
* Add RPM packages to Copr
* Fix packaged `contrib/systemd` units
* Update Go version to 1.7.4
#### Examples
* Upgrade Kubernetes v1.4.6 (static manifest) example clusters
* Upgrade Kubernetes v1.4.6 (rktnetes) example clusters
* Upgrade Kubernetes v1.4.6 (self-hosted) example cluster
## v0.4.1 (2016-10-17)
#### Improvements
* Add ARM and ARM64 release architectures (#309)
* Add guide for installing bootcfg on CoreOS (#306)
* Improvements to the bootcfg cert-gen script (#310)
@@ -28,7 +238,7 @@
* Allow Fuze YAML template files for Ignition 2.0.0 (#141)
* Stop requiring Ignition templates to use file extensions (#176)
* Logging Improvements:
* Add structured loggging with Logrus (#254, #268)
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) IRC channel on freenode.org
Please avoid emailing maintainers found in the MAINTAINERS file directly. They
are very busy and read the mailing lists.
## Getting Started
- Fork the repository on GitHub
- Read the [README](README.md) for build and test instructions
- Play with the project, submit bugs, submit patches!
## Contribution Flow
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where you want to base your work (usually master).
- Make commits of logical units.
- Make sure your commit messages are in the proper format (see below).
- Push your changes to a topic branch in your fork of the repository.
- Make sure the tests pass, and add any new tests as appropriate.
- Submit a pull request to the original repository.
Thanks for your contributions!
### Coding Style
CoreOS projects written in Go follow a set of style guidelines that we've documented
[here](https://github.com/coreos/docs/tree/master/golang). Please follow them when
working on your contributions.
### Format of the Commit Message
We follow a rough convention for commit messages that is designed to answer two
questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
```
scripts: add the test-cluster command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
Fixes #38
```
The format can be described more formally as follows:
```
<subsystem>: <what changed>
<BLANK LINE>
<why this change was made>
<BLANK LINE>
<footer>
```
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
By contributing, you agree to the Linux Foundation's Developer Certificate of Origin ([DCO](DCO)). The DCO is a statement that you, the contributor, have the legal right to make your contribution and understand the contribution will be distributed as part of this project.
Serves a static iPXE boot script which gathers client machine attributes and chainloads to the iPXE endpoint. Use DHCP/TFTP to point iPXE clients to this endpoint as the next-server.
Client's booted with the `/ipxe.boot` endpoint will introspect and make a request to `/ipxe` with the `uuid`, `mac`, `hostname`, and `serial` value as query arguments.
## iPXE
Finds the profile for the machine and renders the network boot config (kernel, options, initrd) as an iPXE script.
Finds the profile for the machine and renders the network boot config as a GRUB config. Use DHCP/TFTP to point GRUB clients to this endpoint as the next-server.
Finds the profile matching the machine and renders the network boot config as JSON to implement the [Pixiecore API](https://github.com/danderson/pixiecore/blob/master/README.api.md). Currently, Pixiecore only provides the machine's MAC address for matching.
`bootcfg` is an HTTP and gRPC service that renders signed [Ignition configs](https://coreos.com/ignition/docs/latest/what-is-ignition.html), [cloud-configs](https://coreos.com/os/docs/latest/cloud-config.html), network boot configs, and metadata to machines to create CoreOS clusters. `bootcfg` maintains **Group** definitions which match machines to *profiles* based on labels (e.g. MAC address, UUID, stage, region). A **Profile** is a named set of config templates (e.g. iPXE, GRUB, Ignition config, Cloud-Config, generic configs). The aim is to use CoreOS Linux's early-boot capabilities to provision CoreOS machines.
Network boot endpoints provide iPXE, GRUB, and [Pixiecore](https://github.com/danderson/pixiecore/blob/master/README.api.md) support. `bootcfg` can be deployed as a binary, as an [appc](https://github.com/appc/spec) container with rkt, or as a Docker container.
A `Store` stores machine Groups, Profiles, and associated Ignition configs, cloud-configs, and generic configs. By default, `bootcfg` uses a `FileStore` to search a `-data-path` for these resources.
Prepare `/var/lib/bootcfg` with `groups`, `profile`, `ignition`, `cloud`, and `generic` subdirectories. You may wish to keep these files under version control.
/var/lib/bootcfg
├── cloud
│ ├── cloud.yaml.tmpl
│ └── worker.sh.tmpl
├── ignition
│ └── raw.ign
│ └── etcd.yaml.tmpl
│ └── simple.yaml.tmpl
├── generic
│ └── config.yaml
│ └── setup.cfg
│ └── datacenter-1.tmpl
├── groups
│ └── default.json
│ └── node1.json
│ └── us-central1-a.json
└── profiles
└── etcd.json
└── worker.json
The [examples](../examples) directory is a valid data directory with some pre-defined configs. Note that `examples/groups` contains many possible groups in nested directories for demo purposes (tutorials pick one to mount). Your machine groups should be kept directly inside the `groups` directory as shown above.
### Profiles
Profiles reference an Ignition config, Cloud-Config, and/or generic config by name and define network boot settings.
The `"boot"` settings will be used to render configs to network boot programs such as iPXE, GRUB, or Pixiecore. You may reference remote kernel and initrd assets or [local assets](#assets).
To use Ignition, set the `coreos.config.url` kernel option to reference the `bootcfg` [Ignition endpoint](api.md#ignition-config), which will render the `ignition_id` file. Be sure to add the `coreos.first_boot` option as well.
To use cloud-config, set the `cloud-config-url` kernel option to reference the `bootcfg` [Cloud-Config endpoint](api.md#cloud-config), which will render the `cloud_id` file.
### Groups
Groups define selectors which match zero or more machines. Machine(s) matching a group will boot and provision according to the group's `Profile`.
Create a group definition with a `Profile` to be applied, selectors for matching machines, and any `metadata` needed to render templated configs. For example `/var/lib/bootcfg/groups/node1.json` matches a single machine with MAC address `52:54:00:89:d8:10`.
For example, a request to `/ignition?mac=52:54:00:89:d8:10` would render the Ignition template in the "etcd" `Profile`, with the machine group's metadata. A request to `/ignition` would match the default group (which has no selectors) and render the Ignition in the "etcd-proxy" Profile. Avoid defining multiple default groups as resolution will not be deterministic.
#### Reserved Selectors
Group selectors can use any key/value pairs you find useful. However, several labels have a defined purpose and will be normalized or parsed specially.
*`uuid` - machine UUID
*`mac` - network interface physical address (normalized MAC address)
*`hostname` - hostname reported by a network boot program
*`serial` - serial reported by a network boot program
### Config Templates
Profiles can reference various templated configs. Ignition JSON configs can be generated from [Fuze config](https://github.com/coreos/fuze/blob/master/doc/configuration.md) template files. Cloud-Config templates files can be used to render a script or Cloud-Config. Generic template files can be used to render arbitrary untyped configs (experimental). Each template may contain [Go template](https://golang.org/pkg/text/template/) elements which will be rendered with machine group metadata, selectors, and query params.
For details and examples:
* [Ignition Config](ignition.md)
* [Cloud-Config](cloud-config.md)
#### Variables
Within Ignition/Fuze templates, Cloud-Config templates, or generic templates, you can use group metadata, selectors, or request-scoped query params. For example, a request `/generic?mac=52-54-00-89-d8-10&foo=some-param&bar=b` would match the `node1.json` machine group shown above. If the group's profile ("etcd") referenced a generic template, the following variables could be used.
Note that `.request` is reserved for these purposes so group metadata with data nested under a top level "request" key will be overwritten.
## Assets
`bootcfg` can serve `-assets-path` static assets at `/assets`. This is helpful for reducing bandwidth usage when serving the kernel and initrd to network booted machines. The default assets-path is `/var/lib/bootcfg/assets` or you can pass `-assets-path=""` to disable asset serving.
bootcfg.foo/assets/
└── coreos
└── VERSION
├── coreos_production_pxe.vmlinuz
└── coreos_production_pxe_image.cpio.gz
For example, a `Profile` might refer to a local asset `/assets/coreos/VERSION/coreos_production_pxe.vmlinuz` instead of `http://stable.release.core-os.net/amd64-usr/VERSION/coreos_production_pxe.vmlinuz`.
See the [get-coreos](../scripts/README.md#get-coreos) script to quickly download, verify, and place CoreOS assets.
## Network
`bootcfg` does not implement or exec a DHCP/TFTP server. Read [network setup](network-setup.md) or use the [coreos/dnsmasq](../contrib/dnsmasq) image if you need a quick DHCP, proxyDHCP, TFTP, or DNS setup.
The self-hosted Kubernetes example provisions a 3 node "self-hosted" Kubernetes v1.4.1 cluster. On-host kubelets wait for an apiserver to become reachable, then yield to kubelet pods scheduled via daemonset. [bootkube](https://github.com/kubernetes-incubator/bootkube) is run on any controller to bootstrap a temporary apiserver which schedules control plane components as pods before exiting. An etcd cluster backs Kubernetes and coordinates CoreOS auto-updates (enabled for disk installs).
## Requirements
Ensure that you've gone through the [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with docker](getting-started-docker.md) guide and understand the basics. In particular, you should be able to:
* Use rkt or Docker to start `bootcfg`
* Create a network boot environment with `coreos/dnsmasq`
* Create the example libvirt client VMs
Build and install the [fork of bootkube](https://github.com/dghubble/bootkube), which supports DNS names.
## Examples
The [examples](../examples) statically assign IP addresses to libvirt client VMs created by `scripts/libvirt`. The examples can be used for physical machines if you update the MAC addresses. See [network setup](network-setup.md) and [deployment](deployment.md).
* [bootkube](../examples/groups/bootkube) - iPXE boot a self-hosted Kubernetes cluster
* [bootkube-install](../examples/groups/bootkube-install) - Install a self-hosted Kubernetes cluster
### Assets
Download the CoreOS image assets referenced in the target [profile](../examples/profiles).
Use the `bootkube` tool to render Kubernetes manifests and credentials into an `--asset-dir`. Later, `bootkube` will schedule these manifests during bootstrapping and the credentials will be used to access your cluster.
# If running with docker, use 172.17.0.21 instead of 172.15.0.21
Use rkt or docker to start `bootcfg` and mount the desired example resources. Create a network boot environment and power-on your machines. Revisit [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with Docker](getting-started-docker.md) for help.
Client machines should boot and provision themselves. Local client VMs should network boot CoreOS and become available via SSH in about 1 minute. If you chose `bootkube-install`, notice that machines install CoreOS and then reboot (in libvirt, you must hit "power" again). Time to network boot and provision physical hardware depends on a number of factors (POST duration, boot device iteration, network speed, etc.).
## bootkube
We're ready to use [bootkube](https://github.com/kubernetes-incubator/bootkube) to create a temporary control plane and bootstrap a self-hosted Kubernetes cluster.
Secure copy the `kubeconfig` to `/etc/kubernetes/kubeconfig` on **every** node (i.e. 172.15.0.21-23 for metal0 or 172.17.0.21-23 for docker0).
for node in '172.15.0.21' '172.15.0.22' '172.15.0.23'; do
Secure copy the `bootkube` generated assets to any controller node and run `bootkube-start`.
scp -r assets core@172.15.0.21:/home/core/assets
ssh core@172.15.0.21 'sudo ./bootkube-start'
Watch the temporary control plane logs until the scheduled kubelet takes over in place of the on-host kubelet.
[ 299.241291] bootkube[5]: Pod Status: kube-api-checkpoint Running
[ 299.241618] bootkube[5]: Pod Status: kube-apiserver Running
[ 299.241804] bootkube[5]: Pod Status: kube-scheduler Running
[ 299.241993] bootkube[5]: Pod Status: kube-controller-manager Running
[ 299.311743] bootkube[5]: All self-hosted control plane components successfully started
You may cleanup the `bootkube` assets on the node, but you should keep the copy on your laptop. It contains a `kubeconfig` and may need to be re-used if the last apiserver were to fail and bootstrapping were needed.
## Verify
[Install kubectl](https://coreos.com/kubernetes/docs/latest/configure-kubectl.html) on your laptop. Use the generated kubeconfig to access the Kubernetes cluster. Verify that the cluster is accessible and that the kubelet, apiserver, scheduler, and controller-manager are running as pods.
The gRPC API allows clients with a TLS client certificate and key to make RPC requests to programmatically create or update `bootcfg` resources. The API can be enabled with the `-rpc-address` flag and by providing a TLS server certificate and key with `-cert-file` and `-key-file` and a CA certificate for authenticating clients with `-ca-file`.
Run the binary with TLS credentials from `examples/etc/bootcfg`.
Clients, such as `bootcmd`, verify the server's certificate with a CA bundle passed via `-ca-file` and present a client certificate and key via `-cert-file` and `-key-file` to cal the gRPC API.
This guide walks through deploying the `bootcfg` service on a Linux host (via binary, rkt, or docker) or on a Kubernetes cluster.
## Provisoner
`bootcfg` is a service for network booting and provisioning machines to create CoreOS clusters. `bootcfg` should be installed on a provisioner machine (CoreOS or any Linux distribution) or cluster (Kubernetes) which can serve configs to client machines in a lab or datacenter.
Choose one of the supported installation options:
* [CoreOS (systemd & rkt)](#coreos)
* [General Linux (systemd & binary)](#binary-release)
# gpg: Good signature from "CoreOS Application Signing Key <security@coreos.com>"
```
Untar the release.
```sh
$ tar xzvf coreos-baremetal-v0.4.0-linux-amd64.tar.gz
$ cd coreos-baremetal-v0.4.0-linux-amd64
```
## TLS Credentials
*Skip this unless you need to enable the gRPC API*
The `bootcfg` gRPC API allows client apps (`bootcmd` CLI, Tectonic Installer, etc.) to manage how machines are provisioned. TLS credentials are needed for client authentication and to establish a secure communication channel. Client machines (those PXE booting) read from the HTTP endpoints and do not require this setup.
If your organization manages public key infrastructure and a certificate authority, create a server certificate and key for the `bootcfg` service and a client certificate and key for each client tool.
Otherwise, generate a self-signed `ca.crt`, a server certificate (`server.crt`, `server.key`), and client credentials (`client.crt`, `client.key`) with the `scripts/tls/cert-gen` script. Export the DNS name or IP (discouraged) of the provisioner host.
```sh
$ cd scripts/tls
# DNS or IP Subject Alt Names where bootcfg can be reached
The example systemd unit exposes the `bootcfg` HTTP machine endpoints on port 8080 and the (optional) gRPC API on port 8081 (remove the `-rpc-address` flag if you don't need the gRPC API). Customize the port settings to suit your preferences and be sure to allow your choices within the host's firewall so clients can access the services.
The unit will rkt run the latest tagged `bootcfg` release, signed by the [CoreOS App Signing Key](https://coreos.com/security/app-signing-key/). Trust the public key.
Start the `bootcfg` service and enable it if you'd like it to start on every boot.
```sh
$ sudo systemctl enable bootcfg.service
$ sudo systemctl start bootcfg.service
```
### General Linux
Pre-built binaries are available for general Linux distributions.
#### Binary
Copy the `bootcfg` static binary to an appropriate location on the host.
```sh
$ sudo cp bootcfg /usr/local/bin
```
#### Set Up User/Group
The `bootcfg` service should be run by a non-root user with access to the `bootcfg` data directory (`/var/lib/bootcfg`). Create a `bootcfg` user and group.
The example unit exposes the `bootcfg` HTTP machine endpoints on port 8080 and exposes the (optional) gRPC API on port 8081 (remove the `-rpc-address` flag if you don't need the gRPC API). Customize the port settings to suit your preferences.
#### Firewall
Be sure to allow your port choices on the provisioner's firewall so the clients can access the service. Here are the commands for those using `firewalld`:
`bootcfg` can serve CoreOS images in development or lab environments to reduce bandwidth usage and increase the speed of CoreOS PXE boots and installs to disk.
Download a recent CoreOS [release](https://coreos.com/releases/) with signatures.
For large production environments, use a cache proxy or mirror suitable for your environment to serve CoreOS images.
## Network
Review [network setup](https://github.com/coreos/coreos-baremetal/blob/master/Documentation/network-setup.md) with your network administrator to set up DHCP, TFTP, and DNS services on your network. At a high level, your goals are to:
* Chainload PXE firmwares to iPXE
* Point iPXE client machines to the `bootcfg` iPXE HTTP endpoint `http://bootcfg.example.com:8080/boot.ipxe`
* Ensure `bootcfg.example.com` resolves to your `bootcfg` deployment
CoreOS provides [dnsmasq](https://github.com/coreos/coreos-baremetal/tree/master/contrib/dnsmasq) as `quay.io/coreos/dnsmasq`, if you wish to use rkt or Docker.
## rkt
Run the most recent tagged and signed `bootcfg` [release](https://github.com/coreos/coreos-baremetal/releases) ACI. Trust the [CoreOS App Signing Key](https://coreos.com/security/app-signing-key/) for image signature verification.
The `bootcfg` HTTP server should be exposed on NodePort `tcp:31488` on each node in the cluster. `BOOTCFG_LOG_LEVEL` is set to debug.
```sh
$ kubectl get deployments
$ kubectl get services
$ kubectl get pods
$ kubectl logs POD-NAME
```
The example manifests use Kubernetes `emptyDir` volumes to back the `bootcfg` FileStore (`/var/lib/bootcfg`). This doesn't provide long-term persistent storage so you may wish to mount your machine groups, profiles, and Ignition configs with a [gitRepo](http://kubernetes.io/docs/user-guide/volumes/#gitrepo) and host image assets on a file server.
### Documentation
View the [documentation](https://github.com/coreos/coreos-baremetal#coreos-on-baremetal) for `bootcfg` service docs, tutorials, example clusters and Ignition configs, PXE booting guides, or machine lifecycle guides.
In this tutorial, we'll run `bootcfg` on your Linux machine with Docker to network boot and provision a cluster of QEMU/KVM CoreOS machines locally. You'll be able to create Kubernetes clustes, etcd clusters, and test network setups.
*Note*: To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).
## Requirements
Install the package dependencies and start the Docker daemon.
# Fedora
sudo dnf install docker virt-install virt-manager
sudo systemctl start docker
# Debian/Ubuntu
# check Docker's docs to install Docker 1.8+ on Debian/Ubuntu
Run the latest `bootcfg` Docker image from `quay.io/coreos/bootcfg` with the `etcd-docker` example. The container should receive the IP address 172.17.0.2 on the `docker0` bridge.
Take a look at the [etcd groups](../examples/groups/etcd-docker) to get an idea of how machines are mapped to Profiles. Explore some endpoints port mapped to localhost:8080.
Since the virtual network has no network boot services, use the `dnsmasq` image to create an iPXE network boot environment which runs DHCP, DNS, and TFTP.
sudo docker run --name dnsmasq --cap-add=NET_ADMIN -v $PWD/contrib/dnsmasq/docker0.conf:/etc/dnsmasq.conf:Z quay.io/coreos/dnsmasq -d
In this case, dnsmasq runs a DHCP server allocating IPs to VMs between 172.17.0.43 and 172.17.0.99, resolves `bootcfg.foo` to 172.17.0.2 (the IP where `bootcfg` runs), and points iPXE clients to `http://bootcfg.foo:8080/boot.ipxe`.
## Client VMs
Create QEMU/KVM VMs which have known hardware attributes. The nodes will be attached to the `docker0` bridge, where Docker's containers run.
sudo ./scripts/libvirt create-docker
sudo virt-manager
You can use `virt-manager` to watch the console and reboot VM machines with
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt start
## Verify
The VMs should network boot and provision themselves into a three node etcd cluster, with other nodes behaving as etcd proxies.
The example profile added autologin so you can verify that etcd works between nodes.
systemctl status etcd2
etcdctl set /message hello
etcdctl get /message
fleetctl list-machines
Clean up the VM machines.
sudo docker rm -f dnsmasq
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt destroy
## Going Further
Learn more about [bootcfg](bootcfg.md) or explore the other [example](../examples) clusters. Try the [k8s-docker example](kubernetes.md) to produce a TLS-authenticated Kubernetes cluster you can access locally with `kubectl` ([docs](../examples/README.md#kubernetes)).
In this tutorial, we'll run `bootcfg` on your Linux machine with `rkt` and `CNI` to network boot and provision a cluster of QEMU/KVM CoreOS machines locally. You'll be able to create Kubernetes clustes, etcd clusters, and test network setups.
*Note*: To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).
## Requirements
Install [rkt](https://coreos.com/rkt/docs/latest/distributions.html) 1.8 or higher ([example script](https://github.com/dghubble/phoenix/blob/master/scripts/fedora/sources.sh)) and setup rkt [privilege separation](https://coreos.com/rkt/docs/latest/trying-out-rkt.html).
**Note**: rkt does not yet integrate with SELinux on Fedora. As a workaround, temporarily set enforcement to permissive if you are comfortable (`sudo setenforce Permissive`). Check the rkt [distribution notes](https://github.com/coreos/rkt/blob/master/Documentation/distributions.md) or see the tracking [issue](https://github.com/coreos/rkt/issues/1727).
Clone the [coreos-baremetal](https://github.com/coreos/coreos-baremetal) source which contains the examples and scripts.
After a recent update, you may see a warning that NetworkManager controls the interface. Work-around this using the firewall-config GUI to add `metal0` to the trusted zone.
## Containers
Run the latest `bootcfg` ACI with rkt and the `etcd` example.
If you get an error about the IP assignment, stop old pods and run garbage collection.
sudo rkt gc --grace-period=0
Take a look at the [etcd groups](../examples/groups/etcd) to get an idea of how machines are mapped to Profiles. Explore some endpoints exposed by the service.
Since the virtual network has no network boot services, use the `dnsmasq` ACI to create an iPXE network boot environment which runs DHCP, DNS, and TFTP.
Trust the [CoreOS App Signing Key](https://coreos.com/security/app-signing-key/).
sudo rkt run coreos.com/dnsmasq:v0.3.0 --net=metal0:IP=172.15.0.3 --mount volume=config,target=/etc/dnsmasq.conf --volume config,kind=host,source=$PWD/contrib/dnsmasq/metal0.conf
In this case, dnsmasq runs a DHCP server allocating IPs to VMs between 172.15.0.50 and 172.15.0.99, resolves `bootcfg.foo` to 172.15.0.2 (the IP where `bootcfg` runs), and points iPXE clients to `http://bootcfg.foo:8080/boot.ipxe`.
## Client VMs
Create QEMU/KVM VMs which have known hardware attributes. The nodes will be attached to the `metal0` bridge, where your pods run.
sudo ./scripts/libvirt create
sudo virt-manager
You can use `virt-manager` to watch the console and reboot VM machines with
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt start
## Verify
The VMs should network boot and provision themselves into a three node etcd cluster, with other nodes behaving as etcd proxies.
The example profile added autologin so you can verify that etcd works between nodes.
systemctl status etcd2
etcdctl set /message hello
etcdctl get /message
fleetctl list-machines
Press ^] three times to stop a rkt pod. Clean up the VM machines.
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt destroy
## Going Further
Learn more about [bootcfg](bootcfg.md) or explore the other [example](../examples) clusters. Try the [k8s example](kubernetes.md) to produce a TLS-authenticated Kubernetes cluster you can access locally with `kubectl` ([docs](../examples/README.md#kubernetes)).
Create UEFI VM nodes which have known hardware attributes.
sudo ./scripts/libvirt create-uefi
## Docker
If you use Docker, run `bootcfg` according to [bootcfg with Docker](getting-started-docker.md), but mount the [grub](../examples/groups/grub) group example. Then start the `coreos/dnsmasq` Docker image, which bundles a `grub.efi`.
Ignition is a system for declaratively provisioning disks during the initramfs, before systemd starts. It runs only on the first boot and handles partitioning disks, formatting partitions, writing files (regular files, systemd units, networkd units, etc.), and configuring users. See the Ignition [docs](https://coreos.com/ignition/docs/latest/) for details.
## Fuze Configs
Ignition 2.0.0+ configs are versioned, *machine-friendly* JSON documents (which contain encoded file contents). Operators should write and maintain configs in a *human-friendly* format, such as CoreOS [fuze](https://github.com/coreos/fuze) configs. As of `bootcfg` v0.4.0, Fuze configs are the primary way to use CoreOS Ignition.
The [Fuze schema](https://github.com/coreos/fuze/blob/master/doc/configuration.md) formalizes and improves upon the YAML to Ignition JSON transform. Fuze provides better support for Ignition 2.0.0+, handles file content encoding, patches Ignition bugs, performs better validations, and lets services (like `bootcfg`) negotiate the Ignition version required by a CoreOS client.
## Adding Fuze Configs
Fuze template files can be added in the `/var/lib/bootcfg/ignition` directory or in an `ignition` subdirectory of a custom `-data-path`. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be evaluated with group metadata, selectors, and query params.
/var/lib/bootcfg
├── cloud
├── ignition
│ └── k8s-controller.yaml
│ └── etcd.yaml
│ └── k8s-worker.yaml
│ └── raw.ign
└── profiles
### Reference
Reference an Fuze config in a [Profile](bootcfg.md#profiles) with `ignition_id`. When PXE booting, use the kernel option `coreos.first_boot=1` and `coreos.config.url` to point to the `bootcfg` [Ignition endpoint](api.md#ignition-config).
### Migration from v0.3.0
In v0.4.0, `bootcfg` switched to using the CoreOS [fuze](https://github.com/coreos/fuze) library, which formalizes and improves upon the YAML to Ignition JSON transform. Fuze provides better support for Ignition 2.0.0+, handles file content encoding, patches Ignition bugs, and performs better validations.
Upgrade your Ignition YAML templates to match the [Fuze config schema](https://github.com/coreos/fuze/blob/master/doc/configuration.md). Typically, you'll need to do the following:
* Remove `ignition_version: 1`, Fuze configs are version-less
* Update `filesystems` section and set the `name`
* Update `files` section to use `inline` as shown below
* Replace `uid` and `gid` with `user` and `group` objects as shown above
Maintain readable inline file contents in Fuze:
```
...
files:
- path: /etc/foo.conf
filesystem: root
contents:
inline: |
foo bar
```
Support for the older Ignition v1 format has been dropped, so CoreOS machines must be **1010.1.0 or newer**. Read the upstream Ignition v1 to 2.0.0 [migration guide](https://coreos.com/ignition/docs/latest/migrating-configs.html) to understand the reasons behind schema changes.
## Examples
Here is an example Fuze template. This template will be rendered into a Fuze config (YAML), using group metadata, selectors, and query params as template variables. Finally, the Fuze config is served to client machines as Ignition JSON.
ignition/format-disk.yaml.tmpl:
---
storage:
disks:
- device: /dev/sda
wipe_table: true
partitions:
- label: ROOT
filesystems:
- name: root
mount:
device: "/dev/sda1"
format: "ext4"
create:
force: true
options:
- "-LROOT"
files:
- filesystem: root
path: /home/core/foo
mode: 0644
user:
id: 500
group:
id: 500
contents:
inline: |
{{.example_contents}}
{{ if index . "ssh_authorized_keys" }}
passwd:
users:
- name: core
ssh_authorized_keys:
{{ range $element := .ssh_authorized_keys }}
- {{$element}}
{{end}}
{{end}}
The Ignition config response (formatted) to a query `/ignition?label=value` for a CoreOS instance supporting Ignition 2.0.0 would be:
{
"ignition": {
"version": "2.0.0",
"config": {}
},
"storage": {
"disks": [
{
"device": "/dev/sda",
"wipeTable": true,
"partitions": [
{
"label": "ROOT",
"number": 0,
"size": 0,
"start": 0
}
]
}
],
"filesystems": [
{
"name": "root",
"mount": {
"device": "/dev/sda1",
"format": "ext4",
"create": {
"force": true,
"options": [
"-LROOT"
]
}
}
}
],
"files": [
{
"filesystem": "root",
"path": "/home/core/foo",
"contents": {
"source": "data:,Example%20file%20contents%0A",
"verification": {}
},
"mode": 420,
"user": {
"id": 500
},
"group": {
"id": 500
}
}
]
},
"systemd": {},
"networkd": {},
"passwd": {}
}
See [examples/ignition](../examples/ignition) for numerous Fuze template examples.
### Raw Ignition
If you prefer to design your own templating solution, raw Ignition files (suffixed with `.ign` or `.ignition`) are served directly.
The Kubernetes example provisions a 3 node Kubernetes v1.4.1 cluster with one controller, two workers, and TLS authentication. An etcd cluster backs Kubernetes and coordinates CoreOS auto-updates (enabled for disk installs).
## Requirements
Ensure that you've gone through the [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with docker](getting-started-docker.md) guide and understand the basics. In particular, you should be able to:
* Use rkt or Docker to start `bootcfg`
* Create a network boot environment with `coreos/dnsmasq`
* Create the example libvirt client VMs
## Examples
The [examples](../examples) statically assign IP addresses to libvirt client VMs created by `scripts/libvirt`. VMs are setup on the `metal0` CNI bridge for rkt or the `docker0` bridge for Docker. The examples can be used for physical machines if you update the MAC addresses. See [network setup](network-setup.md) and [deployment](deployment.md).
* [k8s](../examples/groups/k8s) - iPXE boot a Kubernetes cluster
* [k8s-install](../examples/groups/k8s-install) - Install a Kubernetes cluster to disk
**Note**: TLS assets are served to any machines which request them, which requires a trusted network. Alternately, provisioning may be tweaked to require TLS assets be securely copied to each host. Read about our longer term security plans at [Distributed Trusted Computing](https://coreos.com/blog/coreos-trusted-computing.html).
## Containers
Use rkt or docker to start `bootcfg` and mount the desired example resources. Create a network boot environment and power-on your machines. Revisit [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with Docker](getting-started-docker.md) for help.
Client machines should boot and provision themselves. Local client VMs should network boot CoreOS in about a 1 minute and the Kubernetes API should be available after 3-4 minutes (each node downloads a ~160MB Hyperkube). If you chose `k8s-install`, notice that machines install CoreOS and then reboot (in libvirt, you must hit "power" again). Time to network boot and provision Kubernetes clusters on physical hardware depends on a number of factors (POST duration, boot device iteration, network speed, etc.).
## Verify
[Install kubectl](https://coreos.com/kubernetes/docs/latest/configure-kubectl.html) on your laptop. Use the generated kubeconfig to access the Kubernetes cluster created on rkt `metal0` or `docker0`.
Sign up for [Tectonic Starter](https://tectonic.com/starter/) for free and deploy the [Tectonic Console](https://tectonic.com/enterprise/docs/latest/deployer/tectonic_console.html) with a few `kubectl` commands!
Physical machines [network boot](network-booting.md) in an network boot environment with DHCP/TFTP/DNS services or with [coreos/dnsmasq](../contrib/dnsmasq).
`bootcfg` serves iPXE, GRUB, or Pixiecore boot configs via HTTP to machines based on Group selectors (e.g. UUID, MAC, region, etc.) and machine Profiles. Kernel and initrd images are fetched and booted with Ignition to install CoreOS. The "first boot" Ignition config if fetched and CoreOS is installed.
CoreOS boots ("first boot" from disk) and runs Ignition to provision its disk with systemd units, files, keys, and more to become a cluster node. Systemd units may fetch metadata from a remote source if needed.
Coordinated auto-updates are enabled. Systems like [fleet](https://coreos.com/docs/#fleet) or [Kubernetes](http://kubernetes.io/docs/) coordinate container services. IPMI, vendor utilities, or first-boot are used to re-provision machines into new roles.
This guide shows how to create a DHCP/TFTP/DNS network boot environment to work with `bootcfg` to boot and provision PXE, iPXE, or GRUB2 client machines.
`bootcfg` serves iPXE scripts or GRUB configs over HTTP to serve as the entrypoint for CoreOS cluster bring-up. It does not implement or exec a DHCP, TFTP, or DNS server. Instead, you can configure your own network services to point to `bootcfg` or use the convenient [coreos/dnsmasq](../contrib/dnsmasq) container image (used in libvirt demos).
*Note*: These are just suggestions. Your network administrator or system administrator should choose the right network setup for your company.
## Requirements
Client hardware must have a network interface which supports PXE or iPXE.
## Goals
* Add a DNS name which resolves to a `bootcfg` deploy.
* Chainload PXE firmware to iPXE or GRUB2
* Point iPXE clients to `http://bootcfg.foo:port/boot.ipxe`
* Point GRUB clients to `http://bootcfg.foo:port/grub`
## Setup
Many companies already have DHCP/TFTP configured to "PXE-boot" PXE/iPXE clients. In this case, machines (or a subset of machines) can be made to chainload from `chain http://bootcfg.foo:port/boot.ipxe`. Older PXE clients can be made to chainload into iPXE or GRUB to be able to fetch subsequent configs via HTTP.
On simpler networks, such as what a developer might have at home, a relatively inflexible DHCP server may be in place, with no TFTP server. In this case, a proxy DHCP server can be run alongside a non-PXE capable DHCP server.
This diagram can point you to the **right section(s)** of this document.
The setup of DHCP, TFTP, and DNS services on a network varies greatly. If you wish to use rkt or Docker to quickly run DHCP, proxyDHCP TFTP, or DNS services, use [coreos/dnsmasq](#coreos/dnsmasq).
## DNS
Add a DNS entry (e.g. `bootcfg.foo`, `provisoner.mycompany-internal`) that resolves to a deployment of the CoreOS `bootcfg` service from machines you intend to boot and provision.
dig bootcfg.foo
If you deployed `bootcfg` to a known IP address (e.g. dedicated host, load balanced endpoint, Kubernetes NodePort) and use `dnsmasq`, a domain name to IPv4/IPv6 address mapping could be added to the `/etc/dnsmasq.conf`.
# dnsmasq.conf
address=/bootcfg.foo/172.15.0.2
## iPXE
Servers with DHCP/TFTP/ services which already network boot iPXE clients can use the `chain` command to make clients download and execute the iPXE boot script from `bootcfg`.
# /var/www/html/ipxe/default.ipxe
chain http://bootcfg.foo:8080/boot.ipxe
You can chainload from a menu entry or use other [iPXE commands](http://ipxe.org/cmd) if you have needs beyond just delegating to the iPXE script served by `bootcfg`.
## GRUB
Needs docs.
### Configuring DHCP
Configure your DHCP server to supply options to older PXE client firmware to specify the location of an iPXE or GRUB network boot program on your TFTP server. Send clients to the `bootcfg` iPXE script or GRUB config endpoints.
Here is an example `/etc/dnsmasq.conf`:
```ini
dhcp-range=192.168.1.1,192.168.1.254,30m
enable-tftp
tftp-root=/var/lib/tftpboot
# if request comes from older PXE ROM, chainload to iPXE (via TFTP)
dhcp-boot=tag:!ipxe,undionly.kpxe
# if request comes from iPXE user class, set tag "ipxe"
dhcp-userclass=set:ipxe,iPXE
# point ipxe tagged requests to the bootcfg iPXE boot script (via HTTP)
Alternately, a DHCP proxy server can be run alongside an existing non-PXE DHCP server. The proxy DHCP server provides only the next server and boot filename Options, leaving IP allocation to the DHCP server. Clients listen for both DHCP offers and merge the responses as though they had come from one PXE-enabled DHCP server.
Example `/etc/dnsmasq.conf`:
```ini
dhcp-range=192.168.1.1,proxy,255.255.255.0
enable-tftp
tftp-root=/var/lib/tftpboot
# if request comes from older PXE ROM, chainload to iPXE (via TFTP)
pxe-service=tag:#ipxe,x86PC,"PXE chainload to iPXE",undionly.kpxe
# if request comes from iPXE user class, set tag "ipxe"
dhcp-userclass=set:ipxe,iPXE
# point ipxe tagged requests to the bootcfg iPXE boot script (via HTTP)
sudo rkt run coreos.com/dnsmasq:v0.3.0 --net=host -- -d -q --dhcp-range=192.168.1.1,proxy,255.255.255.0 --enable-tftp --tftp-root=/var/lib/tftpboot --dhcp-userclass=set:ipxe,iPXE --pxe-service=tag:#ipxe,x86PC,"PXE chainload to iPXE",undionly.kpxe --pxe-service=tag:ipxe,x86PC,"iPXE",http://bootcfg.foo:8080/boot.ipxe --log-queries --log-dhcp
```
With Docker:
```sh
sudo docker run --net=host --rm --cap-add=NET_ADMIN quay.io/coreos/dnsmasq -d -q --dhcp-range=192.168.1.1,proxy,255.255.255.0 --enable-tftp --tftp-root=/var/lib/tftpboot --dhcp-userclass=set:ipxe,iPXE --pxe-service=tag:#ipxe,x86PC,"PXE chainload to iPXE",undionly.kpxe --pxe-service=tag:ipxe,x86PC,"iPXE",http://bootcfg.foo:8080/boot.ipxe --log-queries --log-dhcp
```
### Configurable TFTP
If your DHCP server is configured to PXE boot clients, but you don't have control over this configuration, you can modify the pxelinux.cfg's served to PXE clients.
Add ipxe.lkrn to `/var/lib/tftpboot` (see [iPXE docs](http://ipxe.org/embed)).
## coreos/dnsmasq
On networks without network services, the `coreos.com/dnsmasq:v0.3.0` rkt ACI or `coreos/dnsmasq:latest` Docker image can setup an appropriate environment quickly. The images bundle `undionly.kpxe` and `grub.efi` for convenience. Here are some examples which run a DHCP/TFTP/DNS server on your host's network:
The `bootcfg` OpenPGP signature endpoints serve detached binary and ASCII armored signatures of rendered configs, if enabled. Each config endpoint has corresponding signature endpoints, typically suffixed with `.sig` or `.asc`.
To enable OpenPGP signing, provide the path to a secret keyring containing a single signing key with `-key-ring-path` or by setting `BOOTCFG_KEY_RING_PATH`. If a passphrase is required, set it via the `BOOTCFG_PASSPHRASE` environment variable.
Here are example signature endpoints without their query parameters.
In production, mount your signing keyring and source the passphrase from a [Kubernetes secret](http://kubernetes.io/v1.1/docs/user-guide/secrets.html). Use a signing subkey exported to a keyring by itself, which can be revoked by a primary key, if needed.
To try it locally, you may use the test fixture keyring. **Warning: The test fixture keyring is for examples only.**
## Verify
Verify a signature response and config response from the command line using the public key. Notice that most configs have a trailing newline.
**Warning: The test fixture keyring is for examples only.**
Create a signing key or subkey according to your requirements and security policies. Here are some basic [guides](https://coreos.com/rkt/docs/latest/signing-and-verification-guide.html).
The `rktnetes` example provisions a 3 node Kubernetes v1.4.1 cluster with [rkt](https://github.com/coreos/rkt) as the container runtime. The cluster has one controller, two workers, and TLS authentication. An etcd cluster backs Kubernetes and coordinates CoreOS auto-updates (enabled for disk installs).
## Requirements
Ensure that you've gone through the [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with docker](getting-started-docker.md) guide and understand the basics. In particular, you should be able to:
* Use rkt or Docker to start `bootcfg`
* Create a network boot environment with `coreos/dnsmasq`
* Create the example libvirt client VMs
## Examples
The [examples](../examples) statically assign IP addresses to libvirt client VMs created by `scripts/libvirt`. VMs are setup on the `metal0` CNI bridge for rkt or the `docker0` bridge for Docker. The examples can be used for physical machines if you update the MAC addresses. See [network setup](network-setup.md) and [deployment](deployment.md).
* [rktnetes](../examples/groups/rktnetes) - iPXE boot a Kubernetes cluster
* [rktnetes-install](../examples/groups/rktnetes-install) - Install a Kubernetes cluster to disk
**Note**: TLS assets are served to any machines which request them, which requires a trusted network. Alternately, provisioning may be tweaked to require TLS assets be securely copied to each host. Read about our longer term security plans at [Distributed Trusted Computing](https://coreos.com/blog/coreos-trusted-computing.html).
## Containers
Use rkt or docker to start `bootcfg` and mount the desired example resources. Create a network boot environment and power-on your machines. Revisit [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with Docker](getting-started-docker.md) for help.
Client machines should boot and provision themselves. Local client VMs should network boot CoreOS in about a 1 minute and the Kubernetes API should be available after 3-4 minutes (each node downloads a ~160MB Hyperkube). If you chose `rktnetes-install`, notice that machines install CoreOS and then reboot (in libvirt, you must hit "power" again). Time to network boot and provision Kubernetes clusters on physical hardware depends on a number of factors (POST duration, boot device iteration, network speed, etc.).
## Verify
[Install kubectl](https://coreos.com/kubernetes/docs/latest/configure-kubectl.html) on your laptop. Use the generated kubeconfig to access the Kubernetes cluster created on rkt `metal0` or `docker0`.
$ cd /path/to/coreos-baremetal
$ kubectl --kubeconfig=examples/assets/tls/kubeconfig get nodes
NAME STATUS AGE
node1.example.com Ready 3m
node2.example.com Ready 3m
node3.example.com Ready 3m
Get all pods.
$ kubectl --kubeconfig=examples/assets/tls/kubeconfig get pods --all-namespaces
The Torus example provisions a 3 node CoreOS cluster, with `etcd3` and Torus, to demonstrate a stand-alone storage cluster. Each of the 3 nodes runs a Torus instance which makes 1GiB of space available (configured per node by "torus_storage_size" in machine group metadata).
## Requirements
Ensure that you've gone through the [bootcfg with rkt](getting-started-rkt.md) guide and understand the basics. In particular, you should be able to:
* Use rkt or Docker to start `bootcfg`
* Create a network boot environment with `coreos/dnsmasq`
* Create the example libvirt client VMs
* Install the Torus [binaries](https://github.com/coreos/torus/releases)
## Examples
The [examples](../examples) statically assign IP addresses to libvirt client VMs created by `scripts/libvirt`. The examples can be used for physical machines if you update the MAC addresses. See [network setup](network-setup.md) and [deployment](deployment.md).
* [torus](../examples/groups/torus) - iPXE boot a Torus cluster
## Assets
Download the CoreOS image assets referenced in the target [profile](../examples/profiles).
Create a network boot environment and power-on your machines. Revisit [bootcfg with rkt](getting-started-rkt.md) or [bootcfg with Docker](getting-started-docker.md) for help. Client machines should network boot and provision themselves.
## Verify
Install the Torus [binaries](https://github.com/coreos/torus/releases) on your laptop. Torus uses etcd3 for coordination and metadata storage, so any etcd node in the cluster can be queried with `torusctl`.
./torusctl --etcd 172.15.0.21:2379 list-peers
Run `list-peers` to report the status of data nodes in the Torus cluster.
Torus has already initialized its metadata within etcd3 to format the cluster and added all peers to the pool. Each node provides 1 GiB of storage and has `MEMBER` status `OK`.
### Volume Creation
Create a new replicated, virtual block device or `volume` on Torus.
Running DHCP or proxyDHCP with `coreos/dnsmasq` on a host requires that the Firewall allow DHCP and TFTP (for chainloading) services to run.
## Port Collision
Running DHCP or proxyDHCP can cause port already in use collisions depending on what's running. Fedora runs bootp listening on udp/67 for example. Find the service using the port.
sudo lsof -i :67
Evaluate whether you can configure the existing service or whether you'd like to stop it and test with `coreos/dnsmasq`.
## No boot filename received
PXE client firmware did not receive a DHCP Offer with PXE-Options after several attempts. If you're using the `coreos/dnsmasq` image with `-d`, each request should log to stdout. Using the wrong `-i` interface is the most common reason DHCP requests are not received. Otherwise, wireshark can be useful for investigating.
`matchbox` is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Ignition config.
[](https://travis-ci.org/coreos/coreos-baremetal) [](https://godoc.org/github.com/coreos/coreos-baremetal) [](https://quay.io/repository/coreos/bootcfg) [](https://botbot.me/freenode/coreos)
## Features
Guides and a service for network booting and provisioning CoreOS clusters on virtual or physical hardware.
* Chainload via iPXE and match hardware labels
* Provision Fedora CoreOS or Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Authenticated gRPC API for clients (e.g. Terraform)
`bootcfg` is an HTTP and gRPC service that renders signed [Ignition configs](https://coreos.com/ignition/docs/latest/what-is-ignition.html), [cloud-configs](https://coreos.com/os/docs/latest/cloud-config.html), network boot configs, and metadata to machines to create CoreOS clusters. Groups match machines based on labels (e.g. MAC, UUID, stage, region) and use named Profiles for provisioning. Network boot endpoints provide PXE, iPXE, and GRUB. `bootcfg` can be deployed as a binary, as an [appc](https://github.com/appc/spec) container with [rkt](https://coreos.com/rkt/docs/latest/), or as a Docker container.
Matchbox can be installed from a binary or a container image.
*Install Matchbox as a [binary](docs/deployment.md#matchbox-binary), as a [container image](docs/deployment.md#container-image), or on [Kubernetes](docs/deployment.md#kubernetes)
*Setup a PXE-enabled [network](docs/network-setup.md)
### Examples
## Tutorials
The [examples](examples) network boot and provision CoreOS clusters. Network boot [QEMU/KVM](scripts/README.md#libvirt) VMs to try the examples on your Linux laptop.
Start provisioning machines with Fedora CoreOS or Flatcar Linux.
[](https://quay.io/repository/coreos/dnsmasq)
Moved to [dnsmasq](https://github.com/poseidon/dnsmasq).
`dnsmasq` provides an App Container Image (ACI) or Docker image for running DHCP, proxy DHCP, DNS, and/or TFTP with [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) in a container/pod. Use it to test different network setups with clusters of network bootable machines.
The image bundles `undionly.kpxe` which chainloads PXE clients to iPXE and `grub.efi` (experimental) which chainloads UEFI architectures to GRUB2.
docker run --cap-add NET_ADMIN quay.io/coreos/dnsmasq
## Configuration Flags
Configuration arguments can be provided at the command line. Check the dnsmasq [man pages](http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html) for a complete list, but here are some important flags.
| flag | description | example |
|----------|-------------|---------|
| --dhcp-range | Enable DHCP, lease given range | `172.15,0.50,172.15.0.99`, `192.168.1.1,proxy,255.255.255.0` |
| --dhcp-boot | DHCP next server option | `http://bootcfg.foo:8080/boot.ipxe` |
| --enable-tftp | Enable serving from tftp-root over TFTP | NA |
| --address | IP address for a domain name | /bootcfg.foo/172.15.0.2 |
## ACI
Build a `dnsmasq` ACI with the build script which uses [acbuild](https://github.com/appc/acbuild).
cd contrib/dnsmasq
./get-tftp-files
sudo ./build-aci
Run `dnsmasq.aci` with rkt to run DHCP/proxyDHCP/TFTP/DNS services.
Serves a static iPXE boot script which gathers client machine attributes and chainloads to the iPXE endpoint. Use DHCP/TFTP to point iPXE clients to this endpoint as the next-server.
```
GET http://matchbox.foo/boot.ipxe
GET http://matchbox.foo/boot.ipxe.0 // for dnsmasq
Client's booted with the `/ipxe.boot` endpoint will introspect and make a request to `/ipxe` with the `uuid`, `mac`, `hostname`, and `serial` value as query arguments.
## iPXE
Finds the profile for the machine and renders the network boot config (kernel, options, initrd) as an iPXE script.
Finds the profile for the machine and renders the network boot config as a GRUB config. Use DHCP/TFTP to point GRUB clients to this endpoint as the next-server.
**Note:** We recommend migrating to [Ignition](ignition.md) for hardware provisioning.
!!! warning
Migrate to [Ignition configs](ignition.md). Cloud-Config support will be removed in the future.
CoreOS Cloud-Config is a system for configuring machines with a Cloud-Config file or executable script from user-data. Cloud-Config runs in userspace on each boot and implements a subset of the [cloud-init spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data). See the cloud-config [docs](https://coreos.com/os/docs/latest/cloud-config.html) for details.
Cloud-Config template files can be added in `/var/lib/bootcfg/cloud` or in a `cloud` subdirectory of a custom `-data-path`. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be evaluated with group metadata, selectors, and query params.
Cloud-Config template files can be added in `/var/lib/matchbox/cloud` or in a `cloud` subdirectory of a custom `-data-path`. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be evaluated with group metadata, selectors, and query params.
/var/lib/bootcfg
├── cloud
│ ├── cloud.yaml
│ └── script.sh
├── ignition
└── profiles
```
/var/lib/matchbox
├── cloud
│ ├── cloud.yaml
│ └── script.sh
├── ignition
└── profiles
```
## Reference
Reference a Cloud-Config in a [Profile](bootcfg.md#profiles) with `cloud_id`. When PXE booting, use the kernel option `cloud-config-url` to point to `bootcfg` [cloud-config endpoint](api.md#cloud-config).
Reference a Cloud-Config in a [Profile](matchbox.md#profiles) with `cloud_id`. When PXE booting, use the kernel option `cloud-config-url` to point to `matchbox` [cloud-config endpoint](api-http.md#cloud-config).
## Examples
Here is an example Cloud-Config which starts some units and writes a file.
#cloud-config
coreos:
units:
- name: etcd2.service
command: start
- name: fleet.service
command: start
write_files:
- path: "/home/core/welcome"
owner: "core"
permissions: "0644"
content: |
{{.greeting}}
```yaml
#cloud-config
coreos:
units:
- name: etcd2.service
command: start
write_files:
- path: "/home/core/welcome"
owner: "core"
permissions: "0644"
content: |
{{.greeting}}
```
The Cloud-Config [Validator](https://coreos.com/validate/) is also useful for checking your Cloud-Config files for errors.
The gRPC API allows clients with a TLS client certificate and key to make RPC requests to programmatically create or update `matchbox` resources. The API can be enabled with the `-rpc-address` flag and by providing a TLS server certificate and key with `-cert-file` and `-key-file` and a CA certificate for authenticating clients with `-ca-file`.
Run the binary with TLS credentials from `examples/etc/matchbox`.
Clients, such as `bootcmd`, verify the server's certificate with a CA bundle passed via `-ca-file` and present a client certificate and key via `-cert-file` and `-key-file` to cal the gRPC API.
This guide walks through deploying the `matchbox` service on a Linux host (as a binary or container image) or on a Kubernetes cluster.
## Provisoner
Matchbox is a service for network booting and provisioning machines to create Fedora CoreOS or Flatcar Linux clusters. Matchbox may installed on a host server or Kubernetes cluster that can serve configs to client machines in a lab or datacenter.
Choose one of the supported installation options:
* [Matchbox binary](#matchbox-binary)
* [Container image](#container-image)
* [Kubernetes manifests](#kubernetes)
## Download
Download the latest Matchbox [release](https://github.com/poseidon/matchbox/releases).
Verify the release has been signed by Dalton Hubble's GPG [Key](https://keyserver.ubuntu.com/pks/lookup?search=0x8F515AD1602065C8&op=vindex)'s signing subkey.
gpg: Good signature from "Dalton Hubble <dghubble@gmail.com>"
```
Untar the release.
```sh
$ tar xzvf matchbox-v0.10.0-linux-amd64.tar.gz
$ cd matchbox-v0.10.0-linux-amd64
```
## Install
Run Matchbox as a binary, a container image, or on Kubernetes.
### Matchbox Binary
Pre-built binaries are available for generic Linux distributions. Copy the `matchbox` static binary to an appropriate location on the host.
```sh
$ sudo cp matchbox /usr/local/bin
```
#### Set up User/Group
The `matchbox` service should be run by a non-root user with access to the `matchbox` data directory (`/var/lib/matchbox`). Create a `matchbox` user and group.
Customize Matchbox by editing the systemd unit or adding a systemd dropin. Find the complete set of `matchbox` flags and environment variables at [config](config.md).
```sh
$ sudo systemctl edit matchbox
```
By default, the read-only HTTP machine endpoint will be exposed on port **8080**.
Create machine profiles, groups, or Ignition configs by adding files to `/var/lib/matchbox`.
### Kubernetes
Install Matchbox on a Kubernetes cluster with the example manifests.
```sh
$ kubectl apply -R -f contrib/k8s
$ kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
matchbox 10.3.0.145 <none> 8080/TCP,8081/TCP 46m
```
Example manifests in [contrib/k8s](../contrib/k8s) enable the gRPC API to allow client apps to update matchbox objects. Generate TLS server certificates for `matchbox-rpc.example.com` [as shown](#generate-tls-certificates) and create a Kubernetes secret. Alternately, edit the example manifests if you don't need the gRPC API enabled.
Create an Ingress resource to expose the HTTP read-only and gRPC API endpoints. The Ingress example requires the cluster to have a functioning [Nginx Ingress Controller](https://github.com/kubernetes/ingress).
The Matchbox gRPC API allows clients (terraform-provider-matchbox) to create and update Matchbox resources. TLS credentials are needed for client authentication and to establish a secure communication channel. Client machines (those PXE booting) read from the HTTP endpoints and do not require this setup.
The `cert-gen` helper script generates a self-signed CA, server certificate, and client certificate. **Prefer your organization's PKI, if possible**
Navigate to the `scripts/tls` directory.
```sh
$ cd scripts/tls
```
Export `SAN` to set the Subject Alt Names which should be used in certificates. Provide the fully qualified domain name or IP (discouraged) where Matchbox will be installed.
For large production environments, use a cache proxy or mirror suitable for your environment to serve images.
## Network
Review [network setup](https://github.com/poseidon/matchbox/blob/master/docs/network-setup.md) with your network administrator to set up DHCP, TFTP, and DNS services on your network. At a high level, your goals are to:
* Chainload PXE firmwares to iPXE
* Point iPXE client machines to the `matchbox` iPXE HTTP endpoint `http://matchbox.example.com:8080/boot.ipxe`
* Ensure `matchbox.example.com` resolves to your `matchbox` deployment
Poseidon provides [dnsmasq](https://github.com/poseidon/matchbox/tree/master/contrib/dnsmasq) as `quay.io/poseidon/dnsmasq`.
# TLS
Matchbox can serve the read-only HTTP API with TLS.
| Name | Type | Description |
|----------------|--------|-------------|
| -web-ssl | bool | true/false |
| -web-cert-file | string | Path to the server TLS certificate file |
| -web-key-file | string | Path to the server TLS key file |
However, it is more common to use an Ingress Controller (Kubernetes) to terminate TLS.
### Operational notes
* Secrets: Matchbox **can** be run as a public facing service. However, you **must** follow best practices and avoid writing secret material into machine user-data. Instead, load secret materials from an internal secret store.
* Storage: Example manifests use Kubernetes `emptyDir` volumes to store `matchbox` data. Swap those out for a Kubernetes persistent volume if available.
In this tutorial, we'll run `matchbox` on a Linux machine with Docker to network boot and provision local QEMU/KVM machines as Fedora CoreOS or Flatcar Linux machines. You'll be able to test network setups and Ignition provisioning.
!!! note
To provision physical machines, see [network setup](network-setup.md) and [deployment](deployment.md).
## Requirements
Install the package dependencies and start the Docker daemon.
For development convenience, add `/etc/hosts` entries for nodes so they may be referenced by name.
```sh
# /etc/hosts
...
172.17.0.21 node1.example.com
172.17.0.22 node2.example.com
172.17.0.23 node3.example.com
```
## Containers
Run the `matchbox` and `dnsmasq` services on the `docker0` bridge. `dnsmasq` will run DHCP, DNS and TFTP services to create a suitable network boot environment. `matchbox` will serve configs to machines as they PXE boot.
The `devnet` convenience script can start these services and accepts the name of any example in [examples](https://github.com/poseidon/matchbox/tree/master/examples).
```sh
$ sudo ./scripts/devnet create fedora-coreos
```
Inspect the logs.
```
$ sudo ./scripts/devnet status
```
Inspect the examples and Matchbox endpoints to see how machines (e.g. node1 with MAC `52:54:00:a1:9c:ae`) are mapped to Profiles, and therefore iPXE and Ignition configs.
In this tutorial, we'll use `matchbox` with Terraform to provision Fedora CoreOS or Flatcar Linux machines.
We'll install the `matchbox` service, setup a PXE network boot environment, and use Terraform configs to declare infrastructure and apply resources on `matchbox`.
## matchbox
Install `matchbox` on a host server or Kubernetes cluster. Generate TLS credentials and enable the gRPC API as directed. Save the `ca.crt`, `client.crt`, and `client.key` on your local machine (e.g. `~/.matchbox`).
* Installing on a [Linux distro](deployment.md)
* Installing on [Kubernetes](deployment.md#kubernetes)
* Running with [docker](deployment.md#docker)
Verify the matchbox read-only HTTP endpoints are accessible.
```sh
$ curl http://matchbox.example.com:8080
matchbox
```
Verify your TLS client certificate and key can be used to access the gRPC API.
Select from the Terraform [examples](https://github.com/poseidon/matchbox/tree/master/examples/terraform). For example,
*`fedora-coreos-install` - PXE boot, install Fedora CoreOS to disk, reboot, and machines come up with your SSH authorized key set
*`flatcar-install` - PXE boot, install Flatcar Linux to disk, reboot, and machines come up with your SSH authorized key set
These aren't exactly full clusters, but they show declarations and network provisioning.
```sh
$ cd fedora-coreos-install # or flatcar-install
```
!!! note
Fedora CoreOS images are only served via HTTPS, so your iPXE firmware must be compiled to support HTTPS downloads.
Let's review the terraform config and learn a bit about Matchbox.
### Provider
Matchbox is configured as a provider platform for bare-metal resources.
```tf
// Configure the matchbox provider
provider"matchbox"{
endpoint=var.matchbox_rpc_endpoint
client_cert=file("~/.matchbox/client.crt")
client_key=file("~/.matchbox/client.key")
ca=file("~/.matchbox/ca.crt")
}
terraform{
required_providers{
ct={
source="poseidon/ct"
version="0.10.0"
}
matchbox={
source="poseidon/matchbox"
version="0.5.0"
}
}
}
```
### Profiles
Machine profiles specify the kernel, initrd, kernel args, Ignition Config, or other configs (e.g. templated Butane Config, generic) used to network boot and provision a bare-metal machine. The profile below would PXE boot machines using a Fedora CoreOS kernel and initrd (see [assets](api-http.md#assets) to learn about caching for speed), perform a disk install, reboot (first boot from disk), and use a [Fedora CoreOS Config](https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md) to generate an Ignition config to provision.
"--name main https://builds.coreos.fedoraproject.org/prod/streams/${var.os_stream}/builds/${var.os_version}/x86_64/fedora-coreos-${var.os_version}-live-initramfs.x86_64.img"
Matcher groups match machines based on labels like MAC, UUID, etc. to different profiles and templates in machine-specific values. The group below does not have a `selector` block, so any machines which network boot from Matchbox will match this group and be provisioned using the `fedora-coreos-install` profile. Machines are matched to the most specific matching group.
Some Terraform [variables](https://www.terraform.io/docs/configuration/variables.html) are used in the examples. A quick way to set their value is by creating a `terraform.tfvars` file.
Matchbox can integrate with many on-premise network setups. It does not seek to be the DHCP server, TFTP server, or DNS server for the network. Instead, matchbox serves iPXE scripts as the entrypoint for provisioning network booted machines. PXE clients are supported by chainloading iPXE firmware.
In the simplest case, an iPXE-enabled network can chain to Matchbox,
```
# /var/www/html/ipxe/default.ipxe
chain http://matchbox.foo:8080/boot.ipxe
```
Read [network-setup.md](network-setup.md) for the complete range of options. Network admins have a great amount of flexibility:
* May keep using existing DHCP, TFTP, and DNS services
* May configure subnets, architectures, or specific machines to delegate to matchbox
* May place matchbox behind a menu entry (timeout and default to matchbox)
If you've never setup a PXE-enabled network before or you're trying to setup a home lab, checkout the [quay.io/poseidon/dnsmasq](https://quay.io/repository/poseidon/dnsmasq) container image [copy-paste examples](https://github.com/poseidon/matchbox/blob/master/docs/network-setup.md#poseidondnsmasq) and see the section about [proxy-DHCP](https://github.com/poseidon/matchbox/blob/master/docs/network-setup.md#proxy-dhcp).
## Boot
Its time to network boot your machines. Use the BMC's remote management capabilities (may be vendor-specific) to set the boot device (on the next boot only) to PXE and power on each machine.
```sh
$ ipmitool -H node1.example.com -U USER -P PASS power off
$ ipmitool -H node1.example.com -U USER -P PASS power on
```
Each machine should chainload iPXE, delegate to Matchbox, receive its iPXE config (or other supported configs) and begin the provisioning process. The examples assume machines are configured to boot from disk first and PXE only when requested, but you can write profiles for different cases.
Once the install completes and the machine reboots, you can SSH.
```ssh
$ ssh core@node1.example.com
```
To re-provision the machine for another purpose, run `terraform apply` and PXE boot machines again.
## Going Further
Matchbox can be used to provision multi-node Fedora CoreOS or Flatcar Linux clusters at one or many on-premise sites if deployed in an HA way. Machines can be matched individually by MAC address, UUID, region, or other labels you choose. Installs can be made much faster by caching images in the built-in HTTP [assets](api-http.md#assets) server.
[Ignition](https://github.com/coreos/ignition) can be used to partition disks, create file systems, write systemd units, write networkd configs or regular files, and create users. Nodes can be network provisioned into a complete cluster system that meets your needs. For example, see [Typhoon](https://typhoon.psdn.io/fedora-coreos/bare-metal/).
For local development, install the dependencies for libvirt with UEFI.
* [UEFI with QEMU](https://fedoraproject.org/wiki/Using_UEFI_with_QEMU)
Ensure that you've gone through the [matchbox with docker](getting-started-docker.md) and [matchbox](matchbox.md) guides and understand the basics.
## Containers
Run `matchbox` according to [matchbox with Docker](getting-started-docker.md), but mount the [grub](../examples/groups/grub) group example. Then start the `poseidon/dnsmasq` Docker image, which bundles a `grub.efi`.
[Ignition](https://coreos.github.io/ignition/) configs define how disks should be provisioned (on network boot and first-boot from disk) to partition disks, write files (regular files, systemd units, networkd units, etc.), and configure users. Ignition is used by:
* Fedora CoreOS
* RHEL CoreOS
* Flatcar Linux
See the Ignition Config v3.x [specs](https://coreos.github.io/ignition/specs/) for details.
## Usage
Ignition configs can be added to the `/var/lib/matchbox/ignition` directory or in an `ignition` subdirectory of a custom `-data-path`. Ignition configs must end in `.ign` or `ignition`.
```
/var/lib/matchbox
├── ignition
│ └── k8s-controller.ign
│ └── k8s-worker.ign
└── profiles
```
Matchbox Profiles can set an Ignition config for provisioning machines. Specify the Ignition config in a [Profile](matchbox.md#profiles) with `ignition_id`.
```json
{
"id":"worker",
"name":"My Profile",
"boot":{
...
},
"ignition_id":"my-ignition.ign"
}
```
When PXE booting, set kernel arguments depending on the OS (e.g. `ignition.firstboot` on FCOS, `flatcar.first_boot=yes` on Flatcar).
Point the `ignition.config.url` or `flatcar.config.url` to point to the `matchbox` [Ignition endpoint](api-http.md#ignition-config).
Matchbox parses Ignition configs (e.g. `.ign` or `.ignition`) at spec v3.3 or below and renders to the current supported version (v3.3). This relies on Ignition's [forward compatibility](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61).
## Writing Configs
Ignition configs can be prepared externally and loaded via the gRPC API, rather than writing Ignition by hand.
### Terraform
Terraform can be used to prepare Ignition configs, while providing integrations with external systems and rich templating. Using tools like [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct), you can write Butane config (an easier YAML format), validate configs, and load Ignition into Matchbox ([examples](https://github.com/poseidon/matchbox/tree/main/examples/terraform)).
Define a Butane config for Fedora CoreOS or Flatcar Linux:
```yaml
variant:fcos
version:1.5.0
passwd:
users:
- name:core
ssh_authorized_keys:
- ssh-key foo
```
```yaml
variant:flatcar
version:1.1.0
passwd:
users:
- name:core
ssh_authorized_keys:
- ssh-key foo
```
Define a `ct_config` data source with strict validation. Optionally use Terraform [templating](https://github.com/poseidon/terraform-provider-ct).
```tf
data"ct_config""worker"{
content=file("worker.yaml")
strict=true
pretty_print=false
snippets=[
file("units.yaml"),
file("storage.yaml"),
]
}
```
Then render the Butane config to Ignition and use it in a Matchbox Profile.
See the Terraform [examples](https://github.com/poseidon/matchbox/tree/main/examples#terraform-examples) for details.
### Butane
The [Butane](https://coreos.github.io/butane/) command line tool can be used to convert Butane configs (an easier YAML format) to Ignition. Then you can use the Matchbox gRPC API to upload the rendered Ignition to Matchbox for serving to machines on boot.
See [examples/ignition](../examples/ignition) for Butane config examples.
### Matchbox Rendering
While Matchbox recommends preparing Ignition configs externally (e.g. using Terraform's rich templating), Matchbox does still support limited templating and translation features with a builtin Butane converter.
Specify a Butane config in a [Profile](matchbox.md#profiles) with `ignition_id` (file must not end in `.ign` or `.ignition`).
```json
{
"id":"worker",
"name":"My Profile",
"boot":{
...
},
"ignition_id":"butane.yaml"
}
```
Here is an example Butane config with Matchbox template elements. Template files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be interpreted using group metadata, selectors, and query params.
```yaml
variant:flatcar
version:1.1.0
storage:
files:
- path:/var/home/core/foo
mode:0644
contents:
inline:|
{{.example_contents}}
{{if index . "ssh_authorized_keys" }}
passwd:
users:
- name:core
ssh_authorized_keys:
{{range $element := .ssh_authorized_keys }}
- {{$element}}
{{end}}
{{end}}
```
Matchbox will use the Butane library to config to the current supported Ignition version. This relies on Ignition's [forward compatibility](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61).
Matchbox is a service that matches bare-metal machines to profiles that PXE boot and provision clusters. Machines are matched by labels like MAC or UUID during PXE and profiles specify a kernel/initrd, iPXE config, and Ignition config.
## Features
* Chainload via iPXE and match hardware labels
* Provision Fedora CoreOS or Flatcar Linux (powered by [Ignition](https://github.com/coreos/ignition))
* Authenticated gRPC API for clients (e.g. Terraform)
## Installation
Matchbox can be installed from a binary or a container image.
* Install Matchbox as a [binary](deployment.md#matchbox-binary), as a [container image](deployment.md#container-image), or on [Kubernetes](deployment.md#kubernetes)
* Setup a PXE-enabled [network](network-setup.md)
## Tutorials
Start provisioning machines with Fedora CoreOS or Flatcar Linux.
* [Terraform Usage](getting-started.md)
* Fedora CoreOS (live PXE or PXE install to disk)
* Flatcar Linux (live PXE or PXE install to disk)
* [Local QEMU/KVM](getting-started-docker.md)
* Fedora CoreOS (live PXE or PXE install to disk)
* Flatcar Linux (live PXE or PXE install to disk)
## Related
* [dnsmasq](https://github.com/poseidon/matchbox/tree/master/contrib/dnsmasq) - container image to run DHCP, TFTP, and DNS services
* [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox) - Terraform provider plugin for Matchbox
* [Typhoon](https://typhoon.psdn.io/) - minimal and free Kubernetes distribution, supporting bare-metal
Physical machines [network boot](network-booting.md) in an network boot environment with DHCP/TFTP/DNS services or with [poseidon/dnsmasq](../contrib/dnsmasq).
`matchbox` serves iPXE or GRUB configs via HTTP to machines based on Group selectors (e.g. UUID, MAC, region, etc.) and machine Profiles. Kernel and initrd images are fetched and booted with Ignition to install CoreOS Container Linux. The "first boot" Ignition config if fetched and Container Linux is installed.
Container Linux boots ("first boot" from disk) and runs Ignition to provision its disk with systemd units, files, keys, and more to become a cluster node. Systemd units may fetch metadata from a remote source if needed.
Coordinated auto-updates are enabled. Systems like [Kubernetes](http://kubernetes.io/docs/) coordinate container services. IPMI, vendor utilities, or first-boot are used to re-provision machines into new roles.
## Machine lifecycle
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
