* We now use GitHub Actions instead of a private Drone which is great
for OSS build transparency, but one drawback to that change is that
we no longer auto-deploy matchbox to internal Poseidon Labs infra
* We don't want public workflows for deploys to our private infra
Some commands were added in the ipxeBootstrap to enable architecture detection based on the architecture the iPXE version used was built for.
This is done through the use of an additional flag in the chain which is the ipxeBootstrap using the buildarch parameter.
* Add a publish workflow that uses the mkdocs-pages re-usable GitHub
Workflow when the `release-docs` branch is updated
* Build the docs site with mkdocs and publish to GitHub Pages
* Migrate from the internal Drone server using a GitHub Workflow
to perform the multi-arch container image build
* Use self-hosted GitHub runners on ARM64 to perform the ARM64
build step faster that QEMU/KVM emulation
* Mandate approval for all workflow runs from outside contributors
since the builds use push credentials and partially run internally
* Regenerate example Ignition using fcct v0.18.0 to produce Ignition
spec v3.4.0 as a followup to https://github.com/poseidon/matchbox/pull/1079
* Fix one profile example where a double escape isn't needed
* Emphasize that Container Linux Configs have been dropped and
that Butane is the way forward to use modern Ignition v2
* Include some links to Flatcar Linux docs
* Update Makefile and docs for v0.10.0 release process
* Update github.com/coreos/go-semver from v0.3.0 to 167f5da to
prevent it from using gopkg.in/yaml.v2 (CVE-2019-11254)
```
go mod why gopkg.in/yaml.v2
(main module does not need package gopkg.in/yaml.v2)
```
* Recommend preparing Ignition configs external to Matchbox (e.g.
with Terraform poseidon/terraform-provider-ct)
* Document that Matchbox Rendering is discouraged
* Add Butane Config support as a replacement for dropping Container
Linux Config support. Perform Matchbox Go template evaluation,
translation to Ignition, and parsing to a forward compatible version
* Both Flatcar Linux and Fedora CoreOS now support Ignition v2
(spec v3.x) for machine consumption
* Drop support for Ignition v0.35.0 (Ignition spec v2.x)
* Drop support for Container Linux Configs (the YAML format that
rendered to Ignition v2.x JSON for Flatcar Linux)
* Deprecate rendering Container Linux Configs
* Use tools like [poseidon/ct](https://github.com/poseidon/terraform-provider-ct) or
[butane](https://coreos.github.io/butane/getting-started/) to validate and convert a
Butane Config (`focs` or `flatcar`) to Ignition (for Matchbox to serve)
* Please migrate to serving CoreOS Ignition directly, the Container
Linux related HTTP and gRPC endpoints will be removed in future
* Discontinue using Matchbox's Container Linux Config features
* Flatcar Linux OS now supports Ignition v2.13+ which means it
can accept Ignition v3.x spec's, like Fedora CoreOS. Matchbox
supports this by serving Ignition documents directly
* Users of the poseidon/matchbox Terraform provider can pass
a `matchbox_profile` `raw_ignition` contents with the desired
Ignition v3.3 spec
* Users of the poseidon/ct Terraform provider can write Butane
Config YAML, perform templating, and render an Ignition document
using either the fcos or the flatcar variant
* Default branch was renamed from master to main
* Internal build and publishing workflows were unaffected,
the GitHub Action is just for public contribution vetting
* Add initrd=main karg directive for UEFI (ignored by BIOS)
* Update Butane config version to v1.4.0 (generates Ignition v3.3.0)
in `fedora-coreos` and `fedora-coreos-install` examples
* Update virt-install flag --os-variant
* Removed virt-install deprecated flag --os-type
* Remove virt-install QEMU/KVM event preserve since it apparently
wasn't implemented anyway and QEMU/KVM now warns about it
* Remove serial consol kernel argument from examples, but still
mention it in docs
Rel:
* https://github.com/coreos/fedora-coreos-docs/pull/282
* https://www.spinics.net/linux/fedora/libvir/msg222078.html
We still support passing the rootfs image as an appended initrd, but we
recommend using coreos.live.rootfs_url instead: it generally boots faster
and requires less RAM.
When coreos-installer is running from the live image, it no longer needs
a separate install image, since by default it installs from content
embedded in the live system.
* Looking at enabling some update automation, the clarity of
using non-vendored Go modules/checksums outweighs the risk
of a module disappearing (mitigated by proxies)
* Update Fedora CoreOS live PXE and disk install examples to
Fedora 33
* Increase libvirt VM memory from 2GB to 3GB to support live
PXE example, which is mostly just for laptop examples/demos.
Reduce the VM count from 3 to 2 to compensate.
* Change `fedora-coreos.ign` to suggest using an ed25519 SSH
key since Fedora CoreOS 33 disables RSA SHA1 (256 is still ok
but most people won't know which they have)
* Update Terraform examples to use Terraform v0.13
* Refresh examples to show Fedora CoreOS and Flatcar Linux
* Remove the etcd3 example, not worth it to maintain
* Refactor examples to boot provision minimal hosts
with Fedora CoreOS or Flatcar Linux
* Remove the etcd3 cluster example or other specific
kinds of hosts
* Update script get-fedora-coreos
* Remove script get-coreos
* Matchbox is published as a binary or container image,
not planning to resume RPM/Copr publishing since its a
pain
* Publishing to Copr repo stopped in v0.6 (3 years ago)
* Use our own infra to perform image builds and push images. Provides
future options for publishing images to multiple image registries and
for multiple architectures, while keeping push permissions in-house
* Remove Travis ability to push to Quay
* Consider splitting repo and also enabling Quay automated builds
* Use our own infra to perform image builds and push images.
Provides future options for publishing images to multiple
image registries and for multiple architectures, while keeping
push permissions in-house
* Remove Travis ability to push to Quay
* Use Travis only for Go tests of Pull Requests
* Use `quay.io/dghubble/protoc` as the standard codegen
environment across projects. Pin a version (v3.10.1)
* Generated code is now automatically ignored by golint
based on the comment header, which will simplify linting
* Upgrade protobuf from 2bba0603135d to v1.3.2
* Upgrade grpc from v1.2.1 to v1.25.1
* Remove `get-protoc` and `codegen` scripts
* Travis creates a Go v1.13.4 environment, checks out the source
repo, and _then_ executes the `install` block to install `golint`
* With module-aware Go, this means installing `golint` caused a
diff, before the Matchbox `make` target was invoked. Builds were
correctly identified as "dirty" as a result
* Release tags v0.8.1 and v0.8.2 have been removed to avoid any
confusion. Container images with binaries considered dirty have
been removed as well
* rkt achieved its mission creating the OCI standard,
pluggable runtimes, and prevented Docker world
domination. We can now proceed with disarmament
* Few, if any, developers still use rkt locally
* Matchbox examples should be simple and educational to
show how to PXE provision machines into clusters. Today,
these goals are achieved well enough by the 3-node etcd
cluster example
* Several years ago, I put together examples PXE booting
Kubernetes clusters with Matchbox. That was before we wrote
Tectonic or Kubernetes was as popular as it is. Today, a
Kubernetes distro is a project in its own right. It no
longer makes sense to maintain (duplicate) a Kubernetes
distro as "an example" inside Matchbox.
* Matchbox is now used for Kubernetes cluster provisioning in
more organizations than ever. It backs the poseidon/Typhoon and
kinvolk/Locomotive distros. These both serve as great external
examples of using Matchbox to provision Kubernetes clusters
Attention: If you relied on Matchbox Kubernetes docs, you can
find a similar guide at https://typhoon.psdn.io/cl/bare-metal/
(same author). https://github.com/poseidon/typhoon/
* Remove files that are unused or refer to CoreOS policies
or points of contact that no longer apply
* Specifically, CoreOS (and by extension Red Hat) no longer
manages this project, events, etc.
* Update base image from alpine:3.6 to alpine:3.9
* Automatically publish dnsmasq image on merge to a branch
named "dnsmasq". Similar to how matchbox is automatically
published from master. Set the version based on git SHA
* Matchbox has moved to a new home in Poseidon
* Update Makefile so container image name uses
poseidon instead of coreos
* Publish container images to quay.io/poseidon/matchbox
* Render Container Linux Configs referenced in Profiles
as Ignition v2.2.0 documents.
* Recall, configs suffixed with .ign/.ignition will be
served as-is, as raw Ignition. Parse warnings will be
shown
* Serve Ignition configs (ending in .ign/.ignition) with v2.1
or v2.2 formats (previously, configs above v2.1 produced warnings
that the config was too new)
* Use Go 1.11 modules with `GO111MODULE=on`
* Change `make vendor` target to call `go mod vendor`
* Enforce builds and tests use the vendor directory by setting
the `-mod=vendor` flag (notice, travis does not fetch)
* Remove glide requirement and glide files
* Squid proxy docs were added as a draft in 2017 to show a
containerized setup for caching images for network boot
environments
* These docs never matured to a point of viability, I don't
make use of squid, and they're unmaintained. Users would be
better served consulting the squid project
* Local QEMU/KVM tutorials use either docker or rkt/acbuild,
but they're dated. rkt and acbuild are no longer in mainstream
use since they successfully drove OCI standardization
* Remove the rkt examples, as they are just more difficult for
newcomers to use and seldom maintained at this point
* Retain Docker examples, although podman will likely supplant
all docker usage in future
* Upstream author changed case of his/her username (breaking)
* Fix to lowercase name so clients using Go modules can avoid
case-sensitive conflicts. Go modules do not handle this real-world
case well
* Define the required versions of terraform-provider-matchbox
and terraform-provider-ct, with install instructions
* Refresh the tutorial for using Matchbox to PXE boot local
QEMU/KVM machines (using Terraform as a client)
* Add ipxe.efi to dnsmasq image's /var/lib/tftpboot directory
* Add initrd kernel argument respected only by UEFI
https://github.com/coreos/bugs/issues/1239
* Improve network-setup docs and scripts to cover UEFI clients
and to support launching UEFI QEMU/KVM clusters locally
* Reduce references to grub.efi flow, its not a happy path
* Upstream fixes to bump all control plane components to v1.7.5
* Stop including etcd-network-checkpointer with on-host etcd
* Remove experimental_self_hosted_etcd support
* Starting in Go 1.7, the standard library http.Request includes
a Context for passing request-scoped values between chained handlers
* Delete the ContextHandler (breaking, should not have been
exported to begin with)
* Mount host /opt/cni/bin in Kubelet to use host's CNI plugins
* Switch /var/run/kubelet-pod.uuid to /var/cache/kubelet-pod.uuid
to persist between reboots and cleanup old Kubelet pods
* Organize Kubelet flags in alphabetical order
In virt-install v1.4.2, the meaning of `--pxe` changed from "allow pxe
boot" to "always pxe boot." This breaks matchbox, since we expect hosts
to pxe-boot only with empty hds. On hosts with v1.4.2, the VMs loop,
re-installing CL over and over.
The flag isn't necessary anyways, since we pass `--boot=hd,network`,
which enables pxe-booting.
* Provide better errors to clients that forget to specify the
port or include a protocol scheme by mistake
* grpc-go uses net.SplitHostPort to validate server listener
addresses are 'host:port', but doesn't validate Dial targets
etcd examples set ETCDCTL_API=3 but are using v2 etcdctl commands. This
works on CL by accident because it ships with 2.3 so etcdctl doesn't
recognize the API env var.
* Users should deploy the Container Linux Update Operator to coordinate
reboots of Container Linux nodes in a Kubernetes cluster
* Write cluster addon docs to describe CLUO
* Terraform modules `bootkube` and `profiles` (Kubernetes) disable
locksmithd
* Use the container linux update operator to coordinate reboots
* Stop using locksmithd for reboot coordination
* etcd TLS assets now only need to be distributed to controller
nodes which are etcd peers
`After=network-online.target` *should* mean this isn't needed in most
cases, but per
https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/, the
definition of "network-online" is a little shaky.
Regardless, being a little more resilient to network flakes and races is
a good thing. The count of `10` was arbitrarily chosen.
* Ensure that Matchbox (open-source) and Tectonic (enterprise)
are kept separate, Tectonic has its own docs
* Matchbox is agnostic to Kubernetes distribution
* Fixes a regression introduced in 6f02107 which upgraded to
Kubernetes v1.6.6 and added self-hosted etcd with TLS
* Both on-host and self-hosted etcd now require clients to use
TLS client certs so locksmithd
* Upgrade to bootkube v0.4.5
* Enable TLS for experimental self-hosted etcd
* Upstream manifest generation changes modify the flannel
Daemonset, switch several control plane components to run
as non-root, and add an explicit UpdateStrategy to the
control plane components
* Add create, status, and destroy subcommands that use docker as
the container runtime for testing local QEMU/KVM clusters. Before,
only rkt could be used.
* Update local QEMU/KVM tutorial documentation
gRPC API verification step has invalid paths to client.crt and client.key; these are created in ~/matchbox-v0.6.1-linux-amd64/scripts/tls (depending on where the matchbox installer is extracted).
Matchbox added generic template support to enable experimenting with
rendering different kinds of templates, beyond Container Linux configs
and cloud-configs. We'd like to add a gRPC endpoint for generic
templates, as is done for other configs to support gRPC clients.
* Module "profiles" provides container-linux-install and
cached-container-linux-install Profiles
* Module bootkube accepts cached_install variable to determine
whether the cluster should install Container Linux from cache
or from the public download site (default)
* Static Kubernetes / rktnetes examples are no longer going to be
maintained by this repo or upgraded to Kubernetes v1.6. This is not
considered a deprecation bc the reference clusters are examples.
* Remove static Kubernetes cluster examples so users don't choose it
* Self-hosted Kubernetes (bootkube) is now the standard recommended
Kubernetes cluster configuration
* Listing pods or nodes as the final step of cluster creation should
not fail the entire build, its mainly for a pretty output
* There is no official definition of when a Kubernetes cluster is
"done" bootstrapping, they can momentarily fail to response in the
first minute or so as components stabalize
* Calculate the required service IP values from the service CIDR
* These inputs were never truly customizable anyway since bootkube
start assumed the 1st, 10th, and 15th offsets for named services
* Use the dghubble/bootkube-terraform terraform module to generate
the exact same assets that `bootkube render` would
* Use terraform to automate the kubeconfig copy and bootkube start
* Removes the reuqirement to download a bootkube binary, render assets,
and manually copy assets to nodes
Fixing the source URL format to confirm to more normative rpmbuild
standards and to allow for proper use of spectool/rpmspectool. This
change now produces a proper archive with the name and version number
used.
* Add an option to try experimental self-hosted etcd which uses
the etcd-operator to deploy an etcd cluster as pods atop Kubernetes
and disables the on-host etcd cluster
* When enabled, configure locksmithd to coordinate reboots through
self-hosted etcd
* Add matchbox_http_endpoint and matchbox_rpc_endpoint as variables
* Remove dghubble ssh public key from default
* Add a terraform.tfvars.example and gitignore terraform.tfvars
* Add container-linux-install profile to install Container Linux
* Add cached-container-linux-install profile to install Container Linux
from cached matchbox assets
Previously, the terraform readme was incomplete by only including
terraform plan and apply commands. Additionally, the readme was
updated to include instructions for updating the profiles module
source.
Fixes#502
* Terraform example typo's the port number in the etcd_endpoints
* Causes worker etcd-gateway to fail so Container Linux updates may
not been coordinated by locksmith
* Add grub.efi to get-tftp-files script. This matches prior
dnsmasq images, but was not part of a repeatable build
* Switch rkt run examples to pull from quay.io
* Remove script using acbuild to create ACIs
* Stop shipping or mentioning bootcmd CLI tool
* bootcmd has not been built out into a user-facing tool
* terraform-provider-matchbox addresses some of the need
* Keep bootcmd implementation as an example matchbox gRPC client
* Show matchbox deployment, service, and TLS secret creation
* Provide an Ingress resource for exposing HTTP and gRPC APIs
* Add note mentioning matchbox can be run publicly if best practices
are followed
* 1.8 changed the behavior of url.Parse so it is no longer
appropriate for parsing values like 0.0.0.0:8080
* Pass the address directly to http.ListenAndServe which gives
reasonable errors when bad values are given
Unless `GOARM` is specified, the Go compiler will choose the compiling system's architecture as a target (similar to GCC's `-march=native`). This commit prevents that by explicitly selecting ARMv6 (e.g. Raspberry Pi 1) as the target.
According to the documentation, ARM64 does not need this, as ARMv8 is implicit and the value of `GOARM` is not considered for this architecture.
See-Also: https://github.com/golang/go/wiki/GoArm
* Move toward a unified container image which is run
either by rkt or docker
* Sadly, image signing only supported by rkt and is not
part of the new standard
* Add Profile 'args' field as a list of kernel args
* Deprecate 'cmdline' field map of kernel args
* Add missing console=tty0 console=ttyS0 kernel args
to all example clusters
* Show `virsh console nodeN` command for development
with local QEMU/KVM nodes
* Focus on real-world, varied network environments
and flexibility, and bare-metal Tectonic
* iPXE provides better hardware introspection than being
limited to MAC
* ISC DHCP and dnsmasq provides production DHCP service
* Drop Pixiecore from the setups we can support or recommend
Before:
$ sudo ./scripts/devnet
./scripts/devnet: line 20: $1: unbound variable
After:
$ sudo ./scripts/devnet
USAGE: devnet <command>
Commands:
create create bootcfg and PXE services on the bridge
destroy destroy the services on the bridge
* Use the same k8s-controller and k8s-worker profiles
whether booting a live cluster or installing to disk
* Exta root=/dev/sda kernel arg during install is fine
* simple example just network boots CoreOS machines
* simple-install example just network boots and installs CoreOS
* Simple examples don't do much provisioning, except adding pubkeys
* Add `scripts/libvirt create` subcommand for rkt setups
* Add --os-variant=generic to remove nag messages to specify
* Rename places QEMU/KVM VMs were called libvirt VMs
Distributions like Debian 8 ship a `gpg` (1.4.x) and a `gpg2` (2.1.x) binary,
which both use the same config files, and thus cannot be used at the same
time, due to incompatible options. Thus we allow the user to specify which
gpg binary they want to use.
Previously this matched very specific HTTP status codes only, while now it
matches any success or redirection status code. It also works for "HTTP/2"
answers in addition to "HTTP/2.0".
Fixes: #331
* Self-hosted Kuberntes api-server comes up without a hostname
override and detects the hostname it should use from `uname -n`
* kube-apiserver name must correspond to routable kubelet
hostname-override
* Provision /etc/hostname with the FQDN so `uname -n` can be
used by Kubernetes (until NODE_NAME is avail. in k8s 1.4)
* Print helpful message if SAN is unset
* Don't prompt to sign certs, false illusion of choice. Users
running cert-gen need self-signed certs.
* Remove intermediate cert signing requests
* Decrease the scariness of the self-signed warnings
* Add changelog entries since v0.4.0
* Link to Tectonic Installer post
* Don't highlight pixiecore on README. Endpoints support
it, but its not encouraged.
* Get/build protoc and protoc-gen-go binaries under tools
so they do not interfere with any user installations
* Ensure protoc and protoc-gen-go versions are pinned
* Use DNS names to refer to nodes in etcd examples to mirror
production
* Add dnsmasq.conf files for metal0 (rkt) and docker0 examples
which include static MAC->IP and Name->IP mappings
* Remove the etcd-docker example cluster, no longer needed
* Clusters which install to disk auto-update so this bump just
changes the "starting" version. Deployed alpha clusters should
already be using 1109.1.0.
* Serve /metadata including group metadata, selectors, and
query variables in KEY=value "env file" format lines
* Recurse into nested maps (e.g. OUTER_INNER=val)
* Add query parameters to template variables, referenced by
{{.request.query.param}}
* Render Ignition/Fuze, cloud-config, and generic templates and
metadata with collected variables
* Replace template variable {{.query}} with {{.request.raw_query}}
(breaking)
* Simplify common http request handling chains
* Separate github.com/dghubble/ctxh source so we can move
it to a separate CoreOS package or vendor upstream
* Remove unused requireGET
* Units which fail due to unmet dependencies are not restarted.
Upstream issue: github.com/systemd/systemd/issues/1312
* Avoid depending on flanneld. Instead, check and fail if unmet
to force a restart (retry)
* Remove unneeded k8s-assets dependency on kubelet. The k8s-assets
script itself just checks for the Kubernetes API to be up anyway
* Kubelet must be able to resolve DNS names which are
known to the host
* Fixes a bug in which only IP's could be used for
k8s_controller_endpoint metadata
* Upstream systemd issue: github.com/systemd/systemd/issues/1312
* Units which fail due to unmet dependencies are not restarted
as they would be upon failure. Kubelet should keep trying until
flanneld is active.
* Apply the recommended work-around by adding an ExecStart
which checks for a Wanted dependency and purposefully fails
when unmet in order to force a restart to keep trying.
* When raw Ignition (.ign/.ignition) content is provided, Parse
the contents with ignition, but return the contents as JSON
regardless of the result to respect user pass-through expectations
* Warn on parse errors which can occur if a newer Ignition format
is used than the version of Ignition that bootcfg was compiled
with
* By default, templates for Ignition are rendered and
parsed as github.com/coreos/fuze YAML configs, which
formalize the transform from YAML to Ignition JSON
* Ignition files (.ign/.ignition) should be validated and
served directly, without template rendering
* Remove support for Ignition v1
* This change is breaking! Users need to tweak any YAML
Ignition templates to the very similar Fuze YAML format
* Upgrade the self-hosted Kubernetes cluster example to
use bootkube v0.1.1 which supports apiserver checkpointing
in order to tolerate apiserver downtime (e.g. controller
reboot)
* Bumps Kubernetes version to v1.3.0-beta.2
* Re-enable CoreOS auto-updates
* Revert NetBoot id in storage.proto for compatibility
* Add URL.RawQuery to data for generic templates
* Add generic directory to scripts/setup-data-dir
* Add documentation for generic templates.
Profiles now have GenericId which can be used to render a "generic" go template file (neither ignition nor cloud-init) using group metadata and selectors. The storage interface and it's implementations use GetGeneric to retrieve this template.
Partially addresses #224
* scp kubeconfig to hosts rather than insecurely distributing
credentials within Ignition configs. This is also easier than
copy-pasting k8s secrets into machine metadata (slow).
* Self-hosted Kubernetes machine configurations can be versioned
without containing Kubernetes credentials
* Use path-based activiation for the host kubelet
* Update from Kubernetes v1.2.2 to v1.3.0-alpha.5_coreos.0.
* Update host kubelet flags accordingly
* gRPC API server requires a CA certificate to verify
and authenticate clients (passed via -ca-file)
* gRPC clients must authenticate with a client certificate
and key (passed via -cert-file and -key-file)
* A CA certificate (e.g. ca.crt) should be used to sign a server
certificate (server.crt for a private key server.key)
* gRPC API server requires a server certificate and key to be enabled
(passed via -cert-file and -key-file)
* gRPC client bootcmd tool must verify the server identity using a
known CA (passed to bootcmd via -ca-file)
* This provides transport security, it does not provide client
authentication
* Examples which install CoreOS to disk should reference
a CoreOS install image from bootcfg assets to speed up
installs and improve the offline use-case
* Update scripts/get-coreos to download and verify the
CoreOS install image
* Update CoreOS version to get rkt 1.2.1->1.6.0 improvements
for running the kubelet wrapper in Kubernetes clusters
* Update CoreOS version for other clusters so users only need
to download one cached version of assets to run examples
* Rendering an Ignition config or cloud-config template
with machine group metadata will error and log if a metadata
value is missing.
* Previously, the default missing value was "no value"
* Match machines by MAC address in examples
* Re-use the MAC address to assign static networkd configs
where needed.
* UUID is useful to uniquely identify a machine (unlike MAC)
but many users had difficulty finding the network device name
to use in static networkd configs. Selecting by MAC reduces
the potential for user error here.
* bootkube clusters work with bootkube 5c1efe3ac61e270
* Copy bootkube generated assets and executable to any master
* Change hosts to use kubelet client certificate instead of token
* Update bootkube deployment documentation
The docs had the user skip image verification for an image pulled from
Quay, claiming that since Quay hosts docker images the image is
unsigned. This is true when fetching the docker file with "docker://",
but when an ACI is pulled from Quay (which it converts on the fly), the
ACI will be properly signed.
* Stop showing version hash at / path. If bootcfg were deployed
at a public endpoint, the version could be used by attackers to
find out-of-date deployments.
* Users try to use the latest reference example clusters with
a bootcfg v0.3.0 deployment, which did not yet support Ignition
2.0.0. Wait until closer to v0.4.0 to bump Ignition configs.
* Allow Ignition 2.0.0 JSON documents to be served
* Stop requiring Ignition templates to use file extensions to
specify their rendered format for parsing
* Profiles should use only the user-provided DNS endpoint.
Adding 8.8.8.8 can cause inconsistent resolution and in some
setups, 8.8.8.8 is not routable.
* Setting k8s_version via metadata gives the impression
it can be bumped and the cluster will operate with the
desired version. In reality, the k8s profiles change in
minor but important ways which we validate between k8s
releases. It should be part of the k8s profile Ignition.
* bootcfg/version package provides the linker's GIT SHA based
version to bootcfg components
* Display the bootcfg version via the HTTP server / path for
convenience and for validating a deployed version easily
* Log requests to the / rooted subtree paths to surface machines
which are making requests containing typos and mistakes that do
not match HTTP API endpoints
* In the k8s-install example, the k8s-certs@.service checks for
the wrong file and always attempts to curl TLS assets from bootcfg.
After restarts or auto-updates, if bootcfg is not running, the certs
are present on disk so the kubelet and k8s cluster operate normally,
while k8s-certs services fail (mostly harmless).
* Fixes k8s-certs@service failures after restarts when bootcfg is
unavailable. Provisioned nodes should not have a hard dependency on
bootcfg service.
* Ignition and Cloud config templates can reference selector
key/value pairs by lowercase key name
* Metadata endpoint will provide metadata and selector key/val
pairs for the matching machine group
* All Groups retrieved from FileStore's GroupGet (via ParseGroup)
should have normalized selectors since end-users may write Group
definitions with MAC addresses which don't match Go's format
* Change assets download location to examples/assets
* FileStore should default to /var/lib/bootcfg/{profiles,groups,
ignition,cloud}
* Assets location should default to /var/lib/bootcfg/assets
* Fix -rpcAddress flag to be -rpc-address
* Split config.yaml files into JSON groups files
* Update examples to use folders of mountable groups files
* Stop reading groups from a -config config.yaml file
* Add RichGroup JSON <-> Protobuf Serialize Group
* Defining protobuf messages with protobuf rpc services resulted
in generated code which depended upon the gRPC package. This meant
the core server lib depended on gPRC and thus, so did the HTTP
server.
* Move rpc service protobuf definitions under bootcfg/rpc, so the
bootcfg/server package no longer depends on grpc.
* Simplify server lib responses so HTTP server components don't
have to unwrap protobuf Response types intended for gRPC uses
* Wrap server lib responses in protobuf Response types in the
rpc package
* The etcd clusters use static address assignment so dynamic metadata
isn't needed for this case. Custom metadata is useful for fetching
custom data on each boot, to configure units.
* Add gRPC SelectGroup and SelectProfile service endpoints
* HTTP and gRPC servers should both allow labels to be resolved to the
matching Group or Profile. Move logic to bootcfg core server.
* Add gRPC wrapper selectServer to avoid cluttering core server with
response types which are currently only useful for gRPC
* FileStore should manage resources found in type named directories
since this is similar to a traditional table-DB layout and to how
an EtcdStore would work
* http.Dir securely restricts accesses to a filesystem directory to
support http FileServe, which is why this type was used previously
* Dir copies the sanatization behavior of http.Dir verbatim, but
provides ioutil-like write and directory access within the directory
* Why: Safe FileStore writes need to be possible and also, storage
should not have a dependency on net/http just for its http.Dir
* Run k8stls.service to fetch TLS assets if path is missing
* Make Kubelet depend upon a kubelet.path path unit
* Fixes to file mode (octal allowed in YAML)
* Upgrade CoreOS Version to Alpha 962.0.0
* rkt run kubelet with the kubelet-wrapper
* Switch to quay.io/coreos/hyperkube
* Provision an ext4 root partition
* 4GB disk is too small and routinely fails in libvirt
* Depending on build env, dnsmasq ACI can have an old /etc/resolv.conf
DNS entry embedded. Can rebuild to fix, but use --dns=8.8.8.8 too
* Fix issue where etcd_proxy profile failed to pull metadata
so etcd_proxy wasn't configured after CoreOS install
* docs: Compile bootcfg from source, start a systemd unit
* Install bootcfg static binary to /usr/local/bin
* Add default configs /etc/bootcfg.conf and /etc/bootcfg
* Remove api.Store and use storagepb.Store instead
* Remove api.Spec and use storagepb.Profile instead
* Switch from api.Group to storagepb.Group
* Move api.Group to config for YAML config decoding only
* Allow bootcfg groups metadata to be defined via YAML
* Convert YAML metadata to JSON bytes field in a protobuf message
* Strip metadata except strings, string slices, and string-keyed maps
with nesting
* Supplied metal0 CNI definition restricted traffic destinations
to the metal0 bridge. Update to allow outbount traffic.
* Fixes the coreos-install example for use with rkt
<!-- READ: Issues are used to receive focused bug reports from users and to track planned future enhancements by the authors. Topics like support, debugging help, advice, and operation are out of scope and should not use issues-->
**Description**
A clear and concise description of what the bug is.
**Steps to Reproduce**
Provide clear steps to reproduce the bug.
- [ ] Relevant error messages if appropriate (concise, not a dump of everything).
**Expected behavior**
A clear and concise description of what you expected to happen.
* Release: Matchbox version or Git SHA (reporting latest is **not** helpful)
**Possible Solution**
<!-- Most bug reports should have some inkling about solutions. Otherwise, your report may be less of a bug and more of a support request (see top).-->
* Update butane from v0.17.0 to v0.18.0 ([#1079](https://github.com/poseidon/matchbox/pull/1079))
* Add support for `fcos` [v1.5.0](https://coreos.github.io/butane/config-fcos-v1_5/) Butane Configs
* Add support for `flatcar` [v1.1.0](https://coreos.github.io/butane/config-flatcar-v1_1/) Butane Configs
* Render Ignition as Ignition spec [v3.4.0](https://coreos.github.io/ignition/configuration-v3_4/)
## v0.10.0
* Remove support for Ignition v0.35.0 (Ignition spec v2.x)
* Remove support for Container Linux Configs (**action required**)
* Container Linux Configs were a YAML format that rendered to Ignition (spec v2.x)
* Flatcar Linux now supports Ignition v2 (spec v3.x)
* Butane is a suitable YAML format that renders Ignition v2 (spec v3.x)
* Upgrade Ignition from v0.35.0 (spec v2.x) to v2.14.0 (spec v3.x)
* Update Go version (v1.20.2) and alpine base image (v3.17.3)
* Add limited support for Matchbox rendering Butane configs ([#997](https://github.com/poseidon/matchbox/pull/997)) ([docs](https://matchbox.psdn.io/ignition/#matchbox-rendering))
* Recommend writing Butane via external tools (**action required**)
* For Terraform, use [poseidon/terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct)
* For a CLI, use [`butane`](https://github.com/coreos/butane)
* Parse Ignition and render forward to Ignition v2 (spec v3.3)
* Ignition is [forward](https://github.com/coreos/ignition/blob/main/config/v3_3/config.go#L61) compatible (e.g. a `v3.1` spec can be rendered as `v3.3` safely)
If you still template Container Linux Configs via Matchbox, [migrate](https://www.flatcar.org/docs/latest/provisioning/config-transpiler/) to Butane by prepending:
```yaml
variant:flatcar
version:1.0.0
```
## v0.9.1
* Add dependabot Go module update automation ([#833](https://github.com/poseidon/matchbox/pull/833))
* Update Go version (v1.18.4) and alpine base image (v3.16.1)
* Move `dnsmasq` container image to its own [repo](https://github.com/poseidon/dnsmasq) ([#840](https://github.com/poseidon/matchbox/pull/840))
* Deprecate rendering Container Linux Configs
* Please migrate to serving CoreOS Ignition directly
* Use tools like [poseidon/ct](https://github.com/poseidon/terraform-provider-ct) or [butane](https://coreos.github.io/butane/getting-started/) to validate and convert a Butane Config (`focs` or `flatcar`) to Ignition (for Matchbox to serve)
### Docs/Examples
* Migrate docs website to GitHub Pages ([#976](https://github.com/poseidon/matchbox/pull/976))
* Update Fedora CoreOS images and configuration ([#972](https://github.com/poseidon/matchbox/pull/972))
* Update Fedora CoreOS initrd karg for UEFI ([#978](https://github.com/poseidon/matchbox/pull/978))
* Update Flatcar Linux examples to use Ignition v3.3.0 ([#980](https://github.com/poseidon/matchbox/pull/980))
## v0.9.0
* Refresh docs and examples for Fedora CoreOS and Flatcar Linux ([#815](https://github.com/poseidon/matchbox/pull/815), [#816](https://github.com/poseidon/matchbox/pull/816))
* Publish Matchbox images from internal infra to Quay (`quay.io/poseidon/matchbox`)
* Update Go version from v1.13.4 to v1.14.9
* Update base image from `alpine:3.10` to `alpine:3.12` ([#784](https://github.com/poseidon/matchbox/pull/784))
* Include `contrib/k8s` in release tarballs ([#788](https://github.com/poseidon/matchbox/pull/788))
* Remove outdated systemd units ([#817](https://github.com/poseidon/matchbox/pull/817))
* Remove RPM spec file (Copr publishing stopped in v0.6)
## v0.8.3
* Publish docs to [https://matchbox.psdn.io](https://matchbox.psdn.io/) ([#769](https://github.com/poseidon/matchbox/pull/769))
* Update Go version from v1.11.7 to v1.13.4 ([#766](https://github.com/poseidon/matchbox/pull/766), [#770](https://github.com/poseidon/matchbox/pull/770))
* Update container image base from `alpine:3.9` to `alpine:3.10` ([#761](https://github.com/poseidon/matchbox/pull/761))
* Include `get-fedora-coreos` convenience script ([#763](https://github.com/poseidon/matchbox/pull/763))
* Remove rkt tutorials and docs ([#765](https://github.com/poseidon/matchbox/pull/765))
## v0.8.1 - v0.8.2
Releases `v0.8.1` and `v0.8.2` were not built cleanly
* Release tags and container images have been removed
* Caused by go get golint (module-aware) mutating `go.mod` on Travis (see [#775](https://github.com/poseidon/matchbox/pull/775), [#777](https://github.com/poseidon/matchbox/pull/777))
## v0.8.0
* Transfer Matchbox repo from coreos to poseidon GitHub Org
* Publish container images at [quay.io/poseidon/matchbox](https://quay.io/repository/poseidon/matchbox)
* Build Matchbox with Go v1.11.7 for images and binaries
* Update container image base from alpine:3.6 to alpine:3.9
* Render Container Linux Configs as Ignition v2.2.0
* Validate raw Ignition configs with the v2.2 spec (warn-only)
* Fix warnings that v2.2 configs are too new
Note: Release signing key [has changed](https://github.com/poseidon/matchbox/blob/v0.8.0/Documentation/deployment.md) with the project move.
### Examples
* Update Kubernetes example clusters to v1.14.1 (Terraform-based)
## v0.7.1 (2018-11-01)
* Add `kernel_args` variable to the terraform bootkube-install cluster definition
* Add `get-flatcar` helper script
* Add optional TLS support to read-only HTTP API
* Build Matchbox with Go 1.11.1 for images and binaries
### Examples
* Upgrade Kubernetes example clusters to v1.10.0 (Terraform-based)
* Upgrade Kubernetes example clusters to v1.8.5
## v0.7.0 (2017-12-12)
* Add gRPC API endpoints for managing generic (experimental) templates
* Update Container Linux config transpiler to v0.5.0
* Update Ignition to v0.19.0, render v2.1.0 Ignition configs
* Drop support for Container Linux versions below 1465.0.0 (breaking)
* Build Matchbox with Go 1.8.5 for images and binaries
* Remove Profile `Cmdline` map (deprecated in v0.5.0), use `Args` slice instead
* Remove pixiecore support (deprecated in v0.5.0)
* Remove `ContextHandler`, `ContextHandlerFunc`, and `NewHandler` from the `matchbox/http` package.
### Examples / Modules
* Upgrade Kubernetes example clusters to v1.8.4
* Kubernetes examples clusters enable etcd TLS
* Deploy the Container Linux Update Operator (CLUO) to coordinate reboots of Container Linux nodes in Kubernetes clusters. See the cluster [addon docs](Documentation/cluster-addons.md).
* Kubernetes examples (terraform and non-terraform) mask locksmithd
* Terraform modules `bootkube` and `profiles` (Kubernetes) mask locksmithd
## v0.6.1 (2017-05-25)
* Improve the installation documentation
* Move examples/etc/matchbox/cert-gen to scripts/tls
* Build Matchbox with Go 1.8.3 for images and binaries
### Examples
* Upgrade self-hosted Kubernetes cluster examples to v1.6.4
* Add NoSchedule taint to self-hosted Kubernetes controllers
* Remove static Kubernetes and rktnetes cluster examples
## v0.6.0 (2017-04-25)
* New [terraform-provider-matchbox](https://github.com/coreos/terraform-provider-matchbox) plugin for Terraform users!
* New hosted [documentation](https://coreos.com/matchbox/docs/latest) on coreos.com
* Add `ProfileDelete`, `GroupDelete`, `IgnitionGet` and `IgnitionDelete` gRPC endpoints
* Build matchbox with Go 1.8 for container images and binaries
* Generate code with gRPC v1.2.1 and matching Go protoc-gen-go plugin
* Update Ignition to v0.14.0 and coreos-cloudinit to v1.13.0
* Update "fuze" docs to the new name [Container Linux Configs](https://coreos.com/os/docs/latest/configuration.html)
* Remove `bootcmd` binary from release tarballs
### Examples
* Upgrade Kubernetes v1.5.5 (static) example clusters
* Upgrade Kubernetes v1.6.1 (self-hosted) example cluster
* Use etcd3 by default in all clusters (remove etcd2 clusters)
* Add Terraform examples for etcd3 and self-hosted Kubernetes 1.6.1
## v0.5.0 (2017-01-23)
* Rename project to CoreOS `matchbox`!
* Add Profile `args` field to list kernel args
* Update [Fuze](https://github.com/coreos/container-linux-config-transpiler) and [Ignition](https://github.com/coreos/ignition) to v0.11.2
* Switch from `golang.org/x/net/context` to `context`
* Deprecate Profile `cmd` field map of kernel args
* Deprecate Pixiecore support
* Drop build support for Go 1.6
#### Rename
* Move repo from github.com/coreos/coreos-baremetal to github.com/coreos/matchbox
* Rename `bootcfg` binary to `matchbox`
* Rename `bootcfg` packages to `matchbox`
* Publish a `quay.io/coreos/matchbox` container image. The `quay.io/coreos/bootcfg` image will no longer be updated.
* Rename environment variable prefix from `BOOTCFG*` to `MATCHBOX*`
* Change config directory to `/etc/matchbox`
* Change default `-data-path` to `/var/lib/matchbox`
* Change default `-assets-path` to `/var/lib/matchbox/assets`
#### Examples
* Upgrade Kubernetes v1.5.1 (static) example clusters
* Upgrade Kubernetes v1.5.1 (self-hosted) example cluster
* Switch Kubernetes (self-hosted) to run flannel as pods
* Combine rktnetes Ignition into Kubernetes static cluster
#### Migration
* binary users should install the `matchbox` binary (see [installation](Documentation/deployment.md))
* rkt/docker users should start using `quay.io/coreos/matchbox` (see [installation](Documentation/deployment.md))
* RPM users should uninstall bootcfg and install matchbox (see [installation](Documentation/deployment.md))
* Move `/etc/bootcfg` configs and certificates to `/etc/matchbox`
* Move `/var/lib/bootcfg` data to `/var/lib/matchbox`
* See the new [contrib/systemd](contrib/systemd) service examples
* Remove the old `bootcfg` user if you created one
## v0.4.2 (2016-12-7)
#### Improvements
* Add RPM packages to Copr
* Fix packaged `contrib/systemd` units
* Update Go version to 1.7.4
#### Examples
* Upgrade Kubernetes v1.4.6 (static manifest) example clusters
* Upgrade Kubernetes v1.4.6 (rktnetes) example clusters
* Upgrade Kubernetes v1.4.6 (self-hosted) example cluster
## v0.4.1 (2016-10-17)
#### Improvements
* Add ARM and ARM64 release architectures (#309)
* Add guide for installing bootcfg on CoreOS (#306)
* Improvements to the bootcfg cert-gen script (#310)
#### Examples
* Add Kubernetes example with rkt container runtime (i.e. rktnetes)
* Upgrade Kubernetes v1.4.1 (static manifest) example clusters
* Upgrade Kubernetes v1.4.1 (rktnetes) example clusters
* Upgrade Kubernetes v1.4.1 (self-hosted) example cluster
* Add etcd3 example cluster (PXE in-RAM or install to disk)
* Use DNS names (instead of IPs) in example clusters (except bootkube)
## v0.4.0 (2016-07-21)
#### Features
* Add/improve rkt, Docker, Kubernetes, and binary/systemd deployment docs
* TLS Client Authentication:
* Add gRPC API TLS and TLS client-to-server authentication (#140)
* Enable gRPC API by providing a TLS server `-cert-file` and `-key-file`, and a `-ca-file` to authenticate client certificates
* Provide the `bootcmd` tool a TLS client `-cert-file` and `-key-file`, and a `-ca-file` to verify the server identity.
* Improvements to Ignition Support:
* Allow Fuze YAML template files for Ignition 2.0.0 (#141)
* Stop requiring Ignition templates to use file extensions (#176)
* Logging Improvements:
* Add structured logging with Logrus (#254, #268)
* Log requests for bootcfg assets (#214)
* Show `bootcfg` message at the home path `/`
* Fix http package log messages (#173)
* Templating:
* Allow query parameters to be used as template variables as `{{.request.query.foo}}` (#182)
* Support nested maps in responses from the "env file" metadata endpoint (#84)
* Error when a template is rendered with variables which are missing a referenced key. Previously, missing lookups defaulted to "no value" (#210)
* gRPC API
* Add DialTimeout to gRPC client config (#273)
* Add IgnitionPut and Close to the client (#160,#193)
#### Changes
* gRPC API requires TLS client authentication
* Replace Ignition YAML templates with Fuze templates
- Fuze formalizes the transform from Fuze configs (YAML) to Ignition 2.0.0 (JSON)
- [Migrate templates from v0.3.0](Documentation/ignition.md#migration-from-v030)
- Require CoreOS 1010.1.0 or newer
- Drop support for Ignition v1 format
* Replace template variable `{{.query}}` with `{{.request.raw_query}}`
#### Examples
* Kubernetes
* Upgrade Kubernetes v1.3.0 (static manifest) example clusters
* Add Kubernetes v1.3.0-beta.2 (self-hosted) example cluster
* Mount /etc/resolv.conf into host kubelet for skydns and pod DNS lookups (#237,#260)
* Fix a bug in the k8s example k8s-certs@.service file check (#156)
* Avoid systemd dependency failures by restarting components (#257,#274)
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) IRC channel on freenode.org
Please avoid emailing maintainers found in the MAINTAINERS file directly. They
are very busy and read the mailing lists.
## Getting Started
- Fork the repository on GitHub
- Read the [README](README.md) for build and test instructions
- Play with the project, submit bugs, submit patches!
## Contribution Flow
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where you want to base your work (usually master).
- Make commits of logical units.
- Make sure your commit messages are in the proper format (see below).
- Push your changes to a topic branch in your fork of the repository.
- Make sure the tests pass, and add any new tests as appropriate.
- Submit a pull request to the original repository.
Thanks for your contributions!
### Coding Style
CoreOS projects written in Go follow a set of style guidelines that we've documented
[here](https://github.com/coreos/docs/tree/master/golang). Please follow them when
working on your contributions.
### Format of the Commit Message
We follow a rough convention for commit messages that is designed to answer two
questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
```
scripts: add the test-cluster command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
Fixes #38
```
The format can be described more formally as follows:
```
<subsystem>: <what changed>
<BLANK LINE>
<why this change was made>
<BLANK LINE>
<footer>
```
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
By contributing, you agree to the Linux Foundation's Developer Certificate of Origin ([DCO](DCO)). The DCO is a statement that you, the contributor, have the legal right to make your contribution and understand the contribution will be distributed as part of this project.
Serves a static iPXE boot script which gathers client machine attributes and chainloads to the iPXE endpoint. Configure your DHCP server or iPXE server to boot from this script (e.g. set `dhcp-boot:dhcp-boot=tag:ipxe,http://bootcfg.domain.com/ipxe/boot.ipxe` if using `dnsmasq`).
Finds the spec matching the attribute query parameters and renders the boot settings as JSON to implement the Pixiecore API [spec](https://github.com/danderson/pixiecore/blob/master/README.api.md). Currently, Pixiecore only provides the machine's MAC address for matching.
`bootcfg` is an HTTP service that renders signed [Ignition configs](https://coreos.com/ignition/docs/latest/what-is-ignition.html), [cloud-configs](https://coreos.com/os/docs/latest/cloud-config.html), network boot configs, and metadata to machines to create clusters of CoreOS machines. The service maintains a list of **Specs** which are named sets of configuration data (e.g. Ignition config, cloud-config, kernel, initrd). When started, `bootcfg` loads a list of **Group** definitions, which match machines to Specs and metadata based on attributes (e.g. UUID, MAC address, stage) passed as query arguments.
The aim is to use CoreOS Linux's early-boot capabilities to boot machines into functional cluster members with end to end [Distributed Trusted Computing](https://coreos.com/blog/coreos-trusted-computing.html). PXE, iPXE, and [Pixiecore](https://github.com/danderson/pixiecore/blob/master/README.api.md) endpoints provide support for network booting. The `bootcfg` service can be run as an [application container](https://github.com/appc/spec) with rkt, as a Docker container, or as a binary.
## Usage
Fetch the application container image (ACI) from [Quay](https://quay.io/repository/coreos/bootcfg?tab=tags).
The `latest` image corresponds to the most recent commit on master, so choose a tagged [release](https://github.com/coreos/coreos-baremetal/releases) if you require more stability.
Get started running `bootcfg` with rkt or Docker to network boot libvirt VMs on your laptop into CoreOS clusters.
* [Getting Started with rkt](getting-started-rkt.md)
* [Getting Started with Docker](getting-started-docker.md)
Once you've tried those examples, you're ready to write your own configs.
## Data
A `Store` stores Ignition configs, cloud-configs, and named Specs. By default, `bootcfg` uses a `FileStore` to search a data directory (`-data-path`) for these resources.
Prepare a data directory similar to the [examples](../examples) directory, with `ignition`, `cloud`, and `specs` subdirectories. You might keep this directory under version control since it will define the early boot behavior of your machines.
data
├── config.yaml
├── cloud
│ ├── cloud.yaml
│ └── worker.sh
├── ignition
│ └── hello.json
│ └── etcd.yaml
│ └── simple_networking.yaml
└── specs
└── etcd
└── spec.json
└── worker
└── spec.json
Ignition files can be JSON files or Ignition YAML. Cloud-Configs can be YAML or scripts. Both may contain may contain [Go template](https://golang.org/pkg/text/template/) elements which will be evaluated with [metadata](#groups-and-metadata). For details and examples:
* [Ignition Config](ignition.md)
* [Cloud-Config](cloud-config.md)
#### Spec
Specs specify the Ignition config, cloud-config, and PXE boot settings (kernel, options, initrd) of a matched machine.
The `"boot"` settings will be used to render configs to the network boot programs used in PXE, iPXE, or Pixiecore setups. You may reference remote kernel and initrd assets or [local assets](#assets).
To use cloud-config, set the `cloud-config-url` kernel option to the `bootcfg` [Cloud-Config endpoint](api.md#cloud-config) `/cloud?param=val`, which will render the `cloud_id` file.
To use Ignition, set the `coreos.config.url` kernel option to the `bootcfg` [Ignition endpoint](api.md#ignition-config) `/ignition?param=val`, which will render the `ignition_id` file. Be sure to add the `coreos.first_boot` option as well.
## Groups and Metadata
Groups define a set of required tags which match zero or more machines. Machines matching a group will boot and provision themselves according to the group's `spec` and metadata. Currently, `bootcfg` loads group definitions from a YAML config file specified by the `-config` flag. When running `bootcfg` as a container, it is easiest to keep the config file in the [data](#data) directory so it is mounted and versioned.
Define a list of named groups, name the `Spec` that should be applied, add the tags required to match the group, and add your own `metadata` needed to render your Ignition or Cloud configs.
Requirements are AND'd together and evaluated from most constraints to least, in a deterministic order. For most endpoints, "tags" correspond to query arguments in machine requests. Machines are free to query `bootcfg` with additional information (query arguments) about themselves, but they must supply the required set of tags to match a group.
For example, a request to `/cloud?mac=52:54:00:89:d8:10` would render the cloud-config named in "etcd_proxy" `Spec` with the etcd proxy metadata. A request to `/cloud` would match the default group (which has no requirements) and serve the cloud-config from the "discovery" `Spec`. Avoid defining multiple default groups as resolution will not be deterministic.
### Reserved Attributes
The following attributes/tags have reserved semantic purpose. Do not use these tags for other purposes as they may be normalized or parsed specially.
Client's booted with the `/ipxe.boot` endpoint will introspect and make a request to `/ipxe` with the `uuid`, `mac`, `hostname`, and `serial` value as query arguments. Pixiecore which can only detect MAC addresss and cannot substitute it into later config requests ([issue](https://github.com/coreos/coreos-baremetal/issues/36)).
## Assets
`bootcfg` can serve static assets from the `-assets-path` at `/assets`. This is helpful for reducing bandwidth usage when serving the kernel and initrd to network booted machines.
Run the `get-coreos` script to quickly download kernel and initrd image assets.
./scripts/get-coreos # beta, 899.6.0
./scripts/get-coreos alpha 942.0.0
This will create:
assets/
└── coreos
└── 899.6.0
├── coreos_production_pxe.vmlinuz
└── coreos_production_pxe_image.cpio.gz
To reference local assets, change `kernel` and `initrd` in a `Spec` from `http://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe.vmlinuz` to `/assets/coreos/899.6.0/coreos_production_pxe.vmlinuz`, for example.
## Endpoints
The [API](api.md) documents the available endpoints.
## Network
`bootcfg` does not implement a DHCP/TFTP server or monitor running instances. If you need a quick DHCP, proxyDHCP, TFTP, or DNS setup, the [coreos/dnsmasq](../contrib/dnsmasq) image can create a suitable network boot environment on a virtual or physical network. Use `--net` to specify a network bridge and `--dhcp-boot` to point clients to `bootcfg`.
## Virtual and Physical Machine Guides
Next, setup a network of virtual machines with libvirt or boot a cluster of physical hardware. Follow the [libvirt guide](virtual-hardware.md) or [physical hardware guide](physical-hardware.md).
CoreOS cloud-config is a system for configuring machines with a cloud-config file or executable script from user-data. Cloud-Config runs in userspace on each boot and implements a subset of the [cloud-init spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data). See the cloud-config [docs](https://coreos.com/os/docs/latest/cloud-config.html) for details.
Cloud-Config files and scripts can be added in a `cloud` subdirectory of the `bootcfg` data directory. The files may contain [Go template](https://golang.org/pkg/text/template/) elements which will be rendered with `metadata` when served.
data/
├── cloud
│ ├── cloud.yaml
│ ├── kubernetes-master.sh
│ └── kubernetes-worker.sh
├── ignition
└── specs
Add a cloud-config to a `Spec` by adding the `cloud_id` field. When PXE booting, use the kernel option `cloud-config-url` to point to `bootcfg` cloud config endpoint.
Here is an example cloud-config which starts some units and writes a file.
#cloud-config
coreos:
units:
- name: etcd2.service
command: start
- name: fleet.service
command: start
write_files:
- path: "/home/core/welcome"
owner: "core"
permissions: "0644"
content: |
{{.greeting}}
### Examples
See [examples/cloud](../examples/cloud) for example cloud-config files.
### Validator
The cloud-config [validator](https://coreos.com/validate/) is useful for checking your cloud-config files for errors.
## Endpoint
The `bootcfg` [cloud-config endpoint](api.md#cloud-config) `/cloud?param=val` endpoint matches parameters to a machine `Spec` and renders the corresponding cloud-config with `metadata`.
## Comparison with Ignition
Cloud-Config starts after userspace has started and runs on every boot. Ignition starts earlier and only runs on the first boot to provision disk state. Often, tasks do not need to be repeated on each boot (e.g. writing systemd unit files) and can be performed more easily before systemd starts (e.g. configuring networking). Ignition is recommended unless a task requires re-execution on each boot.
If a service needs to be started with dynamic data, a good approach is to use Ignition to write static files which leverage systemd's environment file expansion and start a metadata service to fetch runtime data for services which require it.
Get started with `bootcfg` on your Linux machine with Docker. If you're ready to try [rkt](https://coreos.com/rkt/docs/latest/), see [Getting Started with rkt](getting-started-rkt.md).
In this tutorial, we'll run `bootcfg` to boot and provision a cluster of four VM machines on the `docker0` bridge. You'll be able to boot etcd clusters, Kubernetes clusters, and more, while testing different network setups.
## Requirements
Install the dependencies.
sudo dnf install virt-install docker virt-manager
sudo systemctl start docker
Clone the [coreos-baremetal](https://github.com/coreos/coreos-baremetal) source which contains the examples and scripts.
Create four VM nodes which have known hardware attributes. The nodes will be attached to the `docker0` bridge where your containers run.
sudo ./scripts/libvirt create-docker
Download the CoreOS PXE image assets to `assets/coreos`. The examples instruct machines to load these from the Config server, though you could change this.
./scripts/get-coreos
## Containers
Run `bootcfg` on the default bridge `docker0`. The bridge should assign it the IP 172.17.0.2 (`sudo docker network inspect bridge`).
Take a look at [etcd-docker.yaml](../examples/etcd-docker.yaml) to get an idea of how machines are matched to specifications. Explore some endpoints port mapped to localhost:8080.
Since the virtual network has no network boot services, use the `dnsmasq` container to set up an example iPXE environment which runs DHCP, DNS, and TFTP. The `dnsmasq` container can help test different network setups.
In this case, it runs a DHCP server allocating IPs to VMs between 172.17.0.43 and 172.17.0.99, resolves bootcfg.foo to 172.17.0.2 (the IP where `bootcfg` runs), and points iPXE clients to `http://bootcfg.foo:8080/boot.ipxe`.
## Verify
Reboot the VM machines and use `virt-manager` to watch the console.
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt start
At this point, the VMs will PXE boot and use Ignition (preferred over cloud config) to set up a three node etcd cluster, with other nodes behaving as etcd proxies.
On VMs with autologin, check etcd2 works between different nodes.
systemctl status etcd2
etcdctl set /message hello
etcdctl get /message
Clean up the VM machines.
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt destroy
sudo ./scripts/libvirt delete-disks
## Going Further
Explore the [examples](../examples). Try the `k8s-docker.yaml` [example](../examples/README.md#kubernetes) to produce a TLS-authenticated Kubernetes cluster you can access locally with `kubectl`.
Learn more about [bootcfg](bootcfg.md), enable [OpenPGP signing](openpgp.md), or adapt an example for your own [physical hardware](physical-hardware.md) and network.
Get started with `bootcfg` on your Linux machine with rkt, CNI, and appc.
In this tutorial, we'll run `bootcfg` to boot and provision a cluster of four VM machines on a CNI bridge (`metal0`). You'll be able to boot etcd clusters, Kubernetes clusters, and more, while testing different network setups.
## Requirements
**Note**: Currently, rkt and the Fedora/RHEL/CentOS SELinux policies aren't supported. See the [issue](https://github.com/coreos/rkt/issues/1727) tracking the work and policy changes. To test these examples on your laptop, set SELinux enforcement to permissive if you are comfortable (`sudo setenforce 0`). Enable it again when you are finished.
Install [rkt](https://github.com/coreos/rkt/releases), [acbuild](https://github.com/appc/acbuild), and package dependencies.
sudo dnf install virt-install virt-manager
Clone the [coreos-baremetal](https://github.com/coreos/coreos-baremetal) source which contains the examples and scripts.
Download the CoreOS PXE image assets to `assets/coreos`. The examples instruct machines to load these from the Config server, though you could change this.
./scripts/get-coreos
Define the `metal0` virtual bridge with [CNI](https://github.com/appc/cni).
If you get an error about the IP assignment, garbage collect old pods.
sudo rkt gc --grace-period=0
Take a look at [etcd-rkt.yaml](../examples/etcd-rkt.yaml) to get an idea of how machines are matched to specifications. Explore some endpoints exposed by the service.
Create four VM nodes which have known hardware attributes. The nodes will be attached to the `metal0` bridge where your pods run.
sudo ./scripts/libvirt create-rkt
## Network
In your **Firewall Configuration**, add the `metal0` interface to the trusted zone.
Since the virtual network has no network boot services, use the `dnsmasq` ACI to set up an example iPXE environment which runs DHCP, DNS, and TFTP. The `dnsmasq` container can help test different network setups.
Build the `dnsmasq.aci` ACI.
cd contrib/dnsmasq
sudo ./build-aci
Run `dnsmasq.aci` to create a DHCP and TFTP server pointing to config server.
In this case, dnsmasq runs a DHCP server allocating IPs to VMs between 172.15.0.50 and 172.15.0.99, resolves bootcfg.foo to 172.15.0.2 (the IP where `bootcfg` runs), and points iPXE clients to `http://bootcfg.foo:8080/boot.ipxe`.
## Verify
Reboot the VM machines and use `virt-manager` to watch the console.
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt start
At this point, the VMs will PXE boot and use Ignition (preferred over cloud config) to set up a three node etcd cluster, with other nodes behaving as etcd proxies.
On VMs with autologin, check etcd2 works between different nodes.
systemctl status etcd2
etcdctl set /message hello
etcdctl get /message
Press ^] three times to stop a rkt pod. Clean up the VM machines.
sudo ./scripts/libvirt poweroff
sudo ./scripts/libvirt destroy
sudo ./scripts/libvirt delete-disks
## Going Further
Explore the [examples](../examples). Try the `k8s-rkt.yaml` [example](../examples/README.md#kubernetes) to produce a TLS-authenticated Kubernetes cluster you can access locally with `kubectl`.
Learn more about [bootcfg](bootcfg.md), enable [OpenPGP signing](openpgp.md), or adapt an example for your own [physical hardware](physical-hardware.md) and network.
Ignition is a system for declaratively provisioning disks during the initramfs, before systemd starts. It runs only on the first boot and handles formatting partitions, writing files (systemd units, networkd units, dropins, regular files), and configuring users. See the Ignition [docs](https://coreos.com/ignition/docs/latest/) for details.
Ignition template files can be added in an `ignition` subdirectory of the `bootcfg` data directory. The files may contain [Go template](https://golang.org/pkg/text/template/) elements which should evaluate, with `metadata`, to Ignition JSON or to Ignition YAML (which will be rendered as JSON).
data
├── cloud
├── ignition
│ └── simple.json
│ └── etcd.yaml
│ └── etcd_proxy.yaml
│ └── networking.yaml
└── specs
Add an Ignition config to a `Spec` by adding the `ignition_id` field. When PXE booting, use the kernel option `coreos.first_boot=1` and `coreos.config.url` to point to the `bootcfg`ignition endpoint.
Note that Ignition does **not** allow variables - the response has been fully rendered with `metadata` for the requesting machine.
Ignition configs can be provided directly as JSON as well. This is useful for simple cases or if you prefer to use your own templating solution to generate Ignition configs.
See [examples/ignition](../examples/ignition) for example Ignition configs which setup networking, install CoreOS to disk, or start etcd.
## Endpoint
The `bootcfg` [Ignition endpoint](api.md#ignition-config) `/ignition?param=val` endpoint matches parameters to a machine `Spec` and renders the corresponding Ignition config with `metadata`, transforming YAML to JSON if needed.
These guides walk through booting and configuring CoreOS on real or virtual hardware on your network. This introductory document reviews network booting protocols and the requirments of your network environment.
Once you've reviewed the protocols, you'll be ready to [get started](getting-started-rkt.md) serving configs to machines during boot with [bootcfg](bootcfg.md).
## PXE
The Preboot eXecution Environment (PXE) defines requirements for consistent, hardware-independent network-based machine booting and configuration. Formally, PXE specifies pre-boot protocol services that client NIC firmware must provide (DHCP, TFTP, UDP/IP), specifies boot firmware requirements, and defines a client-server protocol for obtaining a network boot program (NBP) which automates OS installation and configuration.
<img src='img/pxelinux.png' class="img-center" alt="Basic PXE client server protocol flow"/>
At power-on, if a client machine's BIOS or UEFI boot firmware is set to perform network booting, the network interface card's PXE firmware broadcasts a DHCPDISCOVER packet identifying itself as a PXEClient to the network environment.
The network environment can be set up in a number of ways, which we'll discuss. In the simplest, a PXE-enabled DHCP Server responds with a DHCPOFFER with Options, which include a TFTP server IP ("next server") and the name of an NBP ("boot filename") to download (e.g. pxelinux.0). PXE firmware then downloads the NBP over TFTP and starts it. Finally, the NBP loads configs, scripts, and/or images it requires to run an OS.
### Network Boot Programs
Machines can be booted and configured with CoreOS using several network boot programs and approaches. Let's review them. If you're new to network booting or unsure which to choose, iPXE is a reasonable and flexible choice.
#### PXELINUX
[PXELINUX](http://www.syslinux.org/wiki/index.php/PXELINUX) is a common network boot program which loads a config file from `mybootdir/pxelinux.cfg/` over TFTP. The file is chosen based on the client's UUID, MAC address, IP address, or a default.
PXELINUX then downloads the specified kernel and init RAM filesystem images with TFTP.
This approach has a number of drawbacks. TFTP can be slow, managing config files can be tedious, and using different ignition or cloud configs on different machines requires separate pxelinux configs. These limitations spurred the development of various enhancements to PXE, discussed next.
In these guides, PXE is used to load the iPXE boot file so iPXE can chainload config scripts and HTTP images over HTTP. Continue to the [libvirt guide](virtual-hardware.md) or the [baremetal guide](physical-hardware.md) to boot PXE clients by chainloading iPXE. Consult [CoreOS with PXE](https://coreos.com/os/docs/latest/booting-with-pxe.html) for details about CoreOS support for PXE.
#### iPXE
[iPXE](http://ipxe.org/) is an enhanced implementation of the PXE client firmware and a network boot program which uses iPXE scripts rather than config files and can download scripts and images with HTTP.
<img src='img/ipxe.png' class="img-center" alt="iPXE client server protocol flow"/>
A DHCPOFFER to iPXE client firmware specifies an HTTP boot script such as `http://bootcfg.foo/boot.ipxe`.
Here is an example iPXE script for booting the remote CoreOS stable image.
```
#!ipxe
set base-url http://stable.release.core-os.net/amd64-usr/current
A TFTP server is used only to provide the `undionly.kpxe` boot program to older PXE firmware in order to bootstrap into iPXE.
CoreOS `bootcfg` can render signed iPXE scripts to machines based on their hardware attributes. Setup involves configuring your DHCP server to point iPXE clients to the `bootcfg` [iPXE endpoint](api.md#ipxe).
Continue to the [libvirt guide](virtual-hardware.md) or the [baremetal guide](physical-hardware.md) to use iPXE to boot PXE/iPXE client machines. Consult [CoreOS with iPXE](https://coreos.com/os/docs/latest/booting-with-ipxe.html) for details about CoreOS support for iPXE.
#### Pixiecore
[Pixiecore](https://github.com/danderson/pixiecore) is a newer service which implements a proxyDHCP server, TFTP server, and HTTP server all-in-one and calls through to an HTTP API. CoreOS `bootcfg` can serve Pixiecore JSON (optionally signed) based on the supplied MAC address, to implement the Pixiecore HTTP API.
Continue to the [libvirt guide](virtual-hardware.md) to use Pixiecore to network boot PXE client machines.
## Network Environments
### DHCP
Many networks have DHCP services which are impractical to modify or disable. Corporate DHCP servers are governed by network admin policies and home/office networks often have routers running a DHCP service which cannot supply PXE options to PXE clients.
To address this, PXE client firmware listens for DHCPOFFERs from a non-PXE DHCP server *and* a PXE-enabled **proxyDHCP server** configured to respond with the next server and boot filename only. Client firmware combines the two responses as if they had come from a single PXE-enabled DHCP server.
<img src='img/proxydhcp.png' class="img-center" alt="DHCP and proxyDHCP responses are merged to get PXE Options"/>
The [libvirt guide](virtual-hardware.md) shows how to setup a network environment with a standalone PXE-enabled DHCP server or with a separate DHCP server and proxyDHCP server.
The [baremetal guide](physical-hardware.md) shows how to check your network environment and run either a standalone PXE-enabled DHCP server, a proxyDHCP server to compliment your existing network DHCP service, or configure existing services.
## Configuration Service
Now that you understand network booting protocols [get started](getting-started-rkt.md) serving configs to machines during boot with [bootcfg](bootcfg.md).
The `bootcfg` OpenPGP signature endpoints serve ASCII armored detached signatures of rendered configs, if enabled. Each config endpoint has a corresponding signature endpoint, usually suffixed by `.asc`.
To enable OpenPGP signing, provide the path to a secret keyring containing a single signing key with `-key-ring-path` or by setting `BOOTCFG_KEY_RING_PATH`. If a passphrase is required, set it via the `BOOTCFG_PASSPHRASE` environment variable.
Here are example signature endpoints without their query parameters.
In production, mount your signing keyring and source the passphrase from a [Kubernetes secret](http://kubernetes.io/v1.1/docs/user-guide/secrets.html). Use a signing subkey exported to a keyring used only for config signing, which can be revoked by a master if needed.
To try it locally, you may use the test fixture keyring. **Warning: The test fixture keyring is for examples only.**
Create a signing key or subkey according to your requirements and security policies. Here are some basic [guides](https://coreos.com/rkt/docs/latest/signing-and-verification-guide.html).
Physical or baremetal hardware can be booted and configured with CoreOS by PXE, iPXE, and Pixiecore network environments. This guide will show how to setup and test network boot environments with physical clients.
DHCP, TFTP, and HTTP services can run on separate network hosts or on the same host in PXE network environments. We'll focus on configuring a single *provisioner* Linux host to run these services as containers.
## Requirements
Client hardware boot firmware must support PXE or iPXE and the client must have at least one PXE-capable network interface. Most boot firmware and network cards support PXE so you're probably ok.
## Inspect
Identify whether the network runs a DHCP service which can be configured or whether you'll need to run a proxyDHCP service. Check a host's routing table to find the gateway where DHCP is likely to be running.
route -n # e.g. Gateway 192.168.1.1
## bootcfg
Set up `bootcfg` according to the [docs](bootcfg.md). Pull the `coreos/bootcfg` image, prepare a data volume with `Machine` definitions, `Spec` definitions and ignition/cloud configs. Optionally, include a volume of downloaded image assets.
Run the `bootcfg` container to serve configs for any of the network environments we'll discuss next.
Note, the kernel options in the `Spec` [examples](../examples) reference 172.17.0.2 (the libvirt case). Your kernel cmdline options should reference the IP or DNS name where `bootcfg` runs.
## Network Setups
Your network may already have a configurable PXE or iPXE server, configurable DHCP, a DHCP server you cannot modify, or no DHCP server at all. We'll show how to setup each network environment to talk to `bootcfg`, depending on your circumstances.
Otherwise create a PXE, iPXE, or Pixiecore network boot environment using the CoreOS [dnsmasq](../contrib/dnsmasq) container image which can run DHCP, proxyDHCP, TFTP, and/or DNS with `dnsmasq`. Use `--net=host` to run the services on the host and use `--dhcp-boot` to point clients to `bootcfg`.
### Configurable iPXE
If your network environment already supports iPXE, edit your iPXE boot script to chainload from `bootcfg`
# boot.ipxe
chain http://192.168.1.100:8080/boot.ipxe
Substitute the name or IP and port where `bootcfg` runs.
### Configurable DHCP
If the DHCP server on your network is PXE-enabled and configurable, send the `bootcfg` iPXE endpoint as the boot filename option (e.g. `http://192.168.1.100:8080/boot.ipxe`). Substitute the name or IP and port where `bootcfg` runs.
Optionally, respond to older PXE client firmware with the location of the `undionly.kpxe` boot program on your TFTP server.
With `dnsmasq`, here is an example `dnsmasq.conf`
# dnsmasq.conf
dhcp-range=192.168.1.1,192.168.1.254,30m
enable-tftp
tftp-root=/var/lib/tftpboot
# set tag "ipxe" if request comes from iPXE ("iPXE" user class)
dhcp-userclass=set:ipxe,iPXE
# if PXE request came from regular firmware, serve iPXE firmware (via TFTP)
dhcp-boot=tag:!ipxe,undionly.kpxe
# if PXE request came from iPXE, serve an iPXE boot script (via HTTP)
If the network already runs a DHCP service, setup a PXE/iPXE network environment alongside it with proxyDHCP and TFTP.
Run DHCP in proxy mode to respond to DHCP requests on the subnet. Optionally, serve the `undionly.pxe` boot file to older, non-iPXE clients (the '#' means not). Detect iPXE clients by the user class sent in their DHCPDISCOVER (or by Option 175) and point them to the `bootcfg` iPXE boot script.
```
sudo docker run --net=host --rm --cap-add=NET_ADMIN quay.io/coreos/dnsmasq -d -q -i enp0s25 --dhcp-range=192.168.1.1,proxy,255.255.255.0 --enable-tftp --tftp-root=/var/lib/tftpboot --dhcp-userclass=set:ipxe,iPXE --pxe-service=tag:#ipxe,x86PC,"PXE chainload to iPXE",undionly.kpxe --pxe-service=tag:ipxe,x86PC,"iPXE",http://192.168.1.100:8080/boot.ipxe
```
Change the `dhcp-range`, `-i interface`, and boot.ipxe endpoint to match your environment.
In this example, an existing router at 192.168.1.1 runs DHCP to allocate IP addresses between 192.168.1.2 and 192.168.1.254. The proxyDHCP is configured to respond to disocver requests on the 192.168.1.0/24 subnet. `bootcfg` runs on host (192.168.1.100) to serve iPXE boot scripts.
### DHCP
If the network does not already run a DHCP service, you can run one yourself and provide PXE options to baremetal clients. This is the case if your baremetal machines are connected to an isolated switch.
Identify a host machine which should run the DHCP service. If this machine has two NICs, it can serve as a router, using one for the uplink connection and the other to connect to the subnet with baremetal clients.
Run DHCP to allocate IP address leases and TFTP to serve the `undionly.pxe` boot file to older, non-iPXE clients (the '#' means not). Point iPXE clients to the `bootcfg` iPXE boot script.
Change the `dhcp-range`, `-i interface`, and boot.ipxe endpoint to match your environment.
In this example, a DHCP server is configured to allocate IP addresses between 192.168.1.101 and 192.168.1.150. `bootcfg` runs on host (192.168.1.100) to serve iPXE boot scripts.
You may have to explicitly assign the interface (-i) a network address and mark the interface as up.
ip addr add 192.168.1.100/24 dev enp0s20u1
ip link set dev enp0s20u1 up
## Troubleshooting
See [troubleshooting](troubleshooting.md).
### Alternatives
If you prefer, [Debian](http://www.debian-administration.org/article/478/Setting_up_a_server_for_PXE_network_booting), [Fedora](https://docs.fedoraproject.org/en-US/Fedora/7/html/Installation_Guide/ap-pxe-server.html), and [Ubuntu](https://help.ubuntu.com/community/DisklessUbuntuHowto) provide guides on PXE server setups. CoreOS-baremetal also includes [Vagrantfiles](vagrant) to quickly setup example Fedora PXE, iPXE, and Pixiecore servers on libvirt.
Running DHCP or proxyDHCP with `coreos/dnsmasq` on a host requires that the Firewall allow DHCP and TFTP (for chainloading) services to run.
## Port Collision
Running DHCP or proxyDHCP can cause port already in use collisions depending on what's running. Fedora runs bootp listening on udp/67 for example. Find the service using the port.
sudo lsof -i :67
Evaluate whether you can configure the existing service or whether you'd like to stop it and test with `coreos/dnsmasq`.
## No boot filename received
PXE client firmware did not receive a DHCP Offer with PXE-Options after several attempts. If you're using the `coreos/dnsmasq` image with `-d`, each request should log to stdout. Using the wrong `-i` interface is the most common reason DHCP requests are not received. Otherwise, wireshark can be useful for investigating.
CoreOS can be booted and configured on virtual hardware within a libvirt environment (under Linux) with different network services running as Docker containers on the `docker0` virtual bridge. Client VMs or even baremetal hardware attached to the bridge can be booted and configured from the network.
Docker containers run on the `docker0` virtual bridge, typically on a subnet 172.17.0.0/16. Docker assigns IPs to containers started through the docker cli, but the bridge does not run a DHCP service. List network bridges on your host and inspect the bridge Docker 1.9+ created (Docker cli refers to `docker0` as `bridge`).
brctl show
docker network inspect bridge
## bootcfg
Set up `bootcfg` according to the [docs](bootcfg.md). Pull the `coreos/bootcfg` image, prepare a data volume with `Spec` definitions and ignition/cloud configs. Optionally, include a volume of downloaded image assets.
Run the `bootcfg` container to serve configs for any of the network environments we'll discuss next.
Note, the kernel options in the `Spec` [examples](../examples) reference 172.17.0.2, the first container IP Docker is likely to assign to `bootcfg`. Ensure your kernel options point to where `bootcfg` runs.
docker inspect bootcfg # look for Networks, bridge, IPAddress
## Network Setups
Create a PXE, iPXE, or Pixiecore network boot environment using the CoreOS [dnsmasq](../contrib/dnsmasq) container image which can run DHCP, proxyDHCP, TFTP, and/or DNS with `dnsmasq`. Use `--net` to specify a virtual bridge and `--dhcp-boot` to point clients to `bootcfg`.
### PXE
To boot PXE clients, configure a PXE network environment to [chainload iPXE](http://ipxe.org/howto/chainloading). The iPXE setup below configures DHCP to send `undionly.kpxe` over TFTP to older PXE clients for this purpose.
With `dnsmasq`, the relevant `dnsmasq.conf` settings would be:
enable-tftp
tftp-root=/var/lib/tftpboot
# if PXE request came from regular firmware, serve iPXE firmware (via TFTP)
dhcp-boot=tag:!ipxe,undionly.kpxe
### iPXE
Create a PXE/iPXE network environment by running a PXE-enabled DHCP server and TFTP server on the `docker0` bridge, alongside `bootcfg`.
The `coreos/dnsmasq` image runs DHCP and TFTP as a container. It allocates IPs in the `docker0` subnet to VMs and sends options to chainload older PXE clients to iPXE. iPXE clients are pointed to the `bootcfg` service iPXE endpoint (assumed to be running on 172.17.0.2:8080).
To run dnsmasq as a service, rather than from the commandline, the `dnsmasq.conf` might be:
```
# dnsmasq.conf
dhcp-range=172.17.0.43,172.17.0.99,30m
enable-tftp
tftp-root=/var/lib/tftpboot
# set tag "ipxe" if request comes from iPXE ("iPXE" user class)
dhcp-userclass=set:ipxe,iPXE
# if PXE request came from regular firmware, serve iPXE firmware (via TFTP)
dhcp-boot=tag:!ipxe,undionly.kpxe
# if PXE request came from iPXE, serve an iPXE boot script (via HTTP)
Continue to [clients](#clients) to create a client VM or attach a baremetal machine to boot.
### Pixiecore
Create a Pixiecore network environment by running a DHCP server and `danderson/pixiecore` on the `docker0` bridge, alongside `bootcfg`. Pixiecore is a combined proxyDHCP/TFTP/HTTP server.
Run a DHCP server (not PXE-enabled)
```
sudo docker run --rm --cap-add=NET_ADMIN quay.io/coreos/dnsmasq -d -q --dhcp-range=172.17.0.43,172.17.0.99
```
Run Pixiecore, using a script which detects the `bootcfg` container IP:port on docker0.
./scripts/pixiecore
Continue to [clients](#clients) to create a client VM or attach a baremetal machine to boot.
## Clients
Create or attach PXE client machines to the network boot environment on the `docker0` bridge.
### libvirt VM
Create libvirt VM nodes configured to boot from the network. The `scripts/libvirt` script will create four VM nodes with known hardware attributes, on the `docker0` bridge network.
sudo ./scripts/libvirt
USAGE: libvirt <command>
Commands:
create-docker create 4 libvirt nodes on the docker0 bridge
create-rkt create 4 libvirt nodes on a rkt CNI metal0 bridge
start start the 4 libvirt nodes
reboot reboot the 4 libvirt nodes
shutdown shutdown the 4 libvirt nodes
poweroff poweroff the 4 libvirt nodes
destroy destroy the 4 libvirt nodes
remove-disks delete the allocated disks
You may use `virt-manager` to create your own VMs and view the console/state/attributes of existing VM nodes.
When creating your own VMs, select "Network Boot with PXE" and for network selection use "Specify Shared Device" with the bridge name `docker0`.
Learn more by checking [examples](../examples) for clusters you can provision on clients.
### Bare Metal
Connect a baremetal client machine to your host's `docker0` bridge and ensure the boot firmware (probably BIOS) is configured to prefer PXE booting.
Find the network interface and attach it to the virtual bridge.
ip link show # find new link e.g. enp0s20u2
brctl addif docker0 enp0s20u2
Restart the client machine and it should PXE boot using settings from `bootcfg`.
## Next
If you'd like to boot and configure a baremetal machine network, follow the [baremetal guide](physical-hardware.md).
InitialAdvertisePeerURLsstring`yaml:"initial_advertise_peer_urls" env:"ETCD_INITIAL_ADVERTISE_PEER_URLS" deprecated:"etcd2 options no longer work for etcd"`
InitialClusterstring`yaml:"initial_cluster" env:"ETCD_INITIAL_CLUSTER" deprecated:"etcd2 options no longer work for etcd"`
InitialClusterStatestring`yaml:"initial_cluster_state" env:"ETCD_INITIAL_CLUSTER_STATE" deprecated:"etcd2 options no longer work for etcd"`
InitialClusterTokenstring`yaml:"initial_cluster_token" env:"ETCD_INITIAL_CLUSTER_TOKEN" deprecated:"etcd2 options no longer work for etcd"`
SSHImportGithubUserstring`yaml:"coreos_ssh_import_github" deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
SSHImportGithubUsers[]string`yaml:"coreos_ssh_import_github_users" deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
SSHImportURLstring`yaml:"coreos_ssh_import_url" deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
There are far too many logging packages out there, with varying degrees of licenses, far too many features (colorization, all sorts of log frameworks) or are just a pain to use (lack of `Fatalln()`?).
capnslog provides a simple but consistent logging interface suitable for all kinds of projects.
### Design Principles
##### `package main` is the place where logging gets turned on and routed
A library should not touch log options, only generate log entries. Libraries are silent until main lets them speak.
##### All log options are runtime-configurable.
Still the job of `main` to expose these configurations. `main` may delegate this to, say, a configuration webhook, but does so explicitly.
##### There is one log object per package. It is registered under its repository and package name.
`main` activates logging for its repository and any dependency repositories it would also like to have output in its logstream. `main` also dictates at which level each subpackage logs.
##### There is *one* output stream, and it is an `io.Writer` composed with a formatter.
Splitting streams is probably not the job of your program, but rather, your log aggregation framework. If you must split output streams, again, `main` configures this and you can write a very simple two-output struct that satisfies io.Writer.
Fancy colorful formatting and JSON output are beyond the scope of a basic logging framework -- they're application/log-collector dependant. These are, at best, provided as options, but more likely, provided by your application.
##### Log objects are an interface
An object knows best how to print itself. Log objects can collect more interesting metadata if they wish, however, because text isn't going away anytime soon, they must all be marshalable to text. The simplest log object is a string, which returns itself. If you wish to do more fancy tricks for printing your log objects, see also JSON output -- introspect and write a formatter which can handle your advanced log interface. Making strings is the only thing guaranteed.
##### Log levels have specific meanings:
* Critical: Unrecoverable. Must fail.
* Error: Data has been lost, a request has failed for a bad reason, or a required resource has been lost
* Warning: (Hopefully) Temporary conditions that may cause errors, but may work fine. A replica disappearing (that may reconnect) is a warning.
* Notice: Normal, but important (uncommon) log information.
* Info: Normal, working log information, everything is fine, but helpful notices for auditing or common operations.
* Debug: Everything is still fine, but even common operations may be logged, and less helpful but more quantity of notices.
* Trace: Anything goes, from logging every function call as part of a common operation, to tracing execution of a query.
// command-line flags take precedence over env vars
os.Setenv("MYPROJ_C","woof")
iferr:=fs.Set("c","quack");err!=nil{
t.Fatal(err)
}
// first verify that flags are as expected before reading the env
forf,want:=rangemap[string]string{
"a":"",
"b":"bar",
"c":"quack",
}{
ifgot:=fs.Lookup(f).Value.String();got!=want{
t.Fatalf("flag %q=%q, want %q",f,got,want)
}
}
// now read the env and verify flags were updated as expected
err:=SetFlagsFromEnv(fs,"MYPROJ")
iferr!=nil{
t.Errorf("err=%v, want nil",err)
}
forf,want:=rangemap[string]string{
"a":"foo",
"b":"bar",
"c":"quack",
}{
ifgot:=fs.Lookup(f).Value.String();got!=want{
t.Errorf("flag %q=%q, want %q",f,got,want)
}
}
}
funcTestSetFlagsFromEnvBad(t*testing.T){
// now verify that an error is propagated
fs:=flag.NewFlagSet("testing",flag.ExitOnError)
fs.Int("x",0,"")
os.Setenv("MYPROJ_X","not_a_number")
iferr:=SetFlagsFromEnv(fs,"MYPROJ");err==nil{
t.Errorf("err=nil, want != nil")
}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
