mirror of
https://github.com/outbackdingo/matchbox.git
synced 2026-01-27 10:19:35 +00:00
Remove Kubernetes cluster provisioning examples
* Matchbox examples should be simple and educational to show how to PXE provision machines into clusters. Today, these goals are achieved well enough by the 3-node etcd cluster example * Several years ago, I put together examples PXE booting Kubernetes clusters with Matchbox. That was before we wrote Tectonic or Kubernetes was as popular as it is. Today, a Kubernetes distro is a project in its own right. It no longer makes sense to maintain (duplicate) a Kubernetes distro as "an example" inside Matchbox. * Matchbox is now used for Kubernetes cluster provisioning in more organizations than ever. It backs the poseidon/Typhoon and kinvolk/Locomotive distros. These both serve as great external examples of using Matchbox to provision Kubernetes clusters Attention: If you relied on Matchbox Kubernetes docs, you can find a similar guide at https://typhoon.psdn.io/cl/bare-metal/ (same author). https://github.com/poseidon/typhoon/
This commit is contained in:
Dalton Hubble
@@ -1,143 +0,0 @@
|
||||
# Upgrading self-hosted Kubernetes
|
||||
|
||||
CoreOS Kubernetes clusters "self-host" the apiserver, scheduler, controller-manager, flannel, kube-dns, and kube-proxy as Kubernetes pods, like ordinary applications (except with taint tolerations). This allows upgrades to be performed in-place using (mostly) `kubectl` as an alternative to re-provisioning.
|
||||
|
||||
Let's upgrade a Kubernetes v1.6.6 cluster to v1.6.7 as an example.
|
||||
|
||||
## Stability
|
||||
|
||||
This guide shows how to attempt a in-place upgrade of a Kubernetes cluster setup via the [examples](../examples). It does not provide exact diffs, migrations between breaking changes, the stability of a fresh re-provision, or any guarantees. Evaluate whether in-place updates are appropriate for your Kubernetes cluster and be prepared to perform a fresh re-provision if something goes wrong, especially between Kubernetes minor releases (e.g. 1.6 to 1.7).
|
||||
|
||||
## Inspect
|
||||
|
||||
Show the control plane daemonsets and deployments which will need to be updated.
|
||||
|
||||
```sh
|
||||
$ kubectl get daemonsets -n=kube-system
|
||||
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE-SELECTOR AGE
|
||||
kube-apiserver 1 1 1 1 1 node-role.kubernetes.io/master= 21d
|
||||
kube-etcd-network-checkpointer 1 1 1 1 1 node-role.kubernetes.io/master= 21d
|
||||
kube-flannel 4 4 4 4 4 <none> 21d
|
||||
kube-proxy 4 4 4 4 4 <none> 21d
|
||||
pod-checkpointer 1 1 1 1 1 node-role.kubernetes.io/master= 21d
|
||||
|
||||
$ kubectl get deployments -n=kube-system
|
||||
kube-controller-manager 2 2 2 2 21d
|
||||
kube-dns 1 1 1 1 21d
|
||||
kube-scheduler 2 2 2 2 21d
|
||||
```
|
||||
|
||||
Check the current Kubernetes version.
|
||||
|
||||
```sh
|
||||
$ kubectl version
|
||||
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.2", GitCommit:"477efc3cbe6a7effca06bd1452fa356e2201e1ee", GitTreeState:"clean", BuildDate:"2017-04-19T20:33:11Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}
|
||||
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.6+coreos.1", GitCommit:"42a5c8b99c994a51d9ceaed5d0254f177e97d419", GitTreeState:"clean", BuildDate:"2017-06-21T01:10:07Z", GoVersion:"go1.7.6", Compiler:"gc", Platform:"linux/amd64"}
|
||||
```
|
||||
|
||||
```sh
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
node1.example.com Ready 21d v1.6.6+coreos.1
|
||||
node2.example.com Ready 21d v1.6.6+coreos.1
|
||||
node3.example.com Ready 21d v1.6.6+coreos.1
|
||||
node4.example.com Ready 21d v1.6.6+coreos.1
|
||||
```
|
||||
|
||||
## Strategy
|
||||
|
||||
Update control plane components with `kubectl`. Then update the `kubelet` systemd unit on each host.
|
||||
|
||||
Prepare the changes to the Kubernetes manifests by generating assets for a target Kubernetes cluster (e.g. bootkube `v0.5.0` produces Kubernetes 1.6.6 and bootkube `v0.5.1` produces Kubernetes 1.6.7). Choose the tool used during creation of the cluster:
|
||||
|
||||
* [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube) - install the `bootkube` binary for the target version and render assets
|
||||
* [poseidon/bootkube-terraform](https://github.com/poseidon/bootkube-terraform) - checkout the tag for the target version and `terraform apply` to render assets
|
||||
|
||||
Diff the generated assets against the assets used when originally creating the cluster. In simple cases, you may only need to bump the hyperkube image. In more complex cases, some manifests may have new flags or configuration.
|
||||
|
||||
## Control Plane
|
||||
|
||||
### kube-apiserver
|
||||
|
||||
Edit the `kube-apiserver` daemonset to rolling update the apiserver.
|
||||
|
||||
```sh
|
||||
$ kubectl edit daemonset kube-apiserver -n=kube-system
|
||||
```
|
||||
|
||||
If you only have one apiserver, the cluster may be momentarily unavailable.
|
||||
|
||||
### kube-scheduler
|
||||
|
||||
Edit the `kube-scheduler` deployment to rolling update the scheduler.
|
||||
|
||||
```sh
|
||||
$ kubectl edit deployments kube-scheduler -n=kube-system
|
||||
```
|
||||
|
||||
### kube-controller-manager
|
||||
|
||||
Edit the `kube-controller-manager` deployment to rolling update the controller manager.
|
||||
|
||||
```sh
|
||||
$ kubectl edit deployments kube-controller-manager -n=kube-system
|
||||
```
|
||||
|
||||
### kube-proxy
|
||||
|
||||
Edit the `kube-proxy` daemonset to rolling update the proxy.
|
||||
|
||||
```sh
|
||||
$ kubectl edit daemonset kube-proxy -n=kube-system
|
||||
```
|
||||
|
||||
### Others
|
||||
|
||||
If there are changes between the prior version and target version manifests, update the `kube-dns` deployment, `kube-flannel` daemonset, or `pod-checkpointer` daemonset.
|
||||
|
||||
### Verify
|
||||
|
||||
Verify the control plane components updated.
|
||||
|
||||
```sh
|
||||
$ kubectl version
|
||||
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.2", GitCommit:"477efc3cbe6a7effca06bd1452fa356e2201e1ee", GitTreeState:"clean", BuildDate:"2017-04-19T20:33:11Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}
|
||||
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.7+coreos.0", GitCommit:"c8c505ee26ac3ab4d1dff506c46bc5538bc66733", GitTreeState:"clean", BuildDate:"2017-07-06T17:38:33Z", GoVersion:"go1.7.6", Compiler:"gc", Platform:"linux/amd64"}
|
||||
```
|
||||
|
||||
```sh
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
node1.example.com Ready 21d v1.6.7+coreos.0
|
||||
node2.example.com Ready 21d v1.6.7+coreos.0
|
||||
node3.example.com Ready 21d v1.6.7+coreos.0
|
||||
node4.example.com Ready 21d v1.6.7+coreos.0
|
||||
```
|
||||
|
||||
## kubelet
|
||||
|
||||
SSH to each node and update `/etc/kubernetes/kubelet.env`. Restart the `kubelet.service`.
|
||||
|
||||
```sh
|
||||
ssh core@node1.example.com
|
||||
sudo vim /etc/kubernetes/kubelet.env
|
||||
sudo systemctl restart kubelet
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
Verify the kubelet and kube-proxy of each node updated.
|
||||
|
||||
```sh
|
||||
$ kubectl get nodes -o yaml | grep 'kubeletVersion\|kubeProxyVersion'
|
||||
kubeProxyVersion: v1.6.7+coreos.0
|
||||
kubeletVersion: v1.6.7+coreos.0
|
||||
kubeProxyVersion: v1.6.7+coreos.0
|
||||
kubeletVersion: v1.6.7+coreos.0
|
||||
kubeProxyVersion: v1.6.7+coreos.0
|
||||
kubeletVersion: v1.6.7+coreos.0
|
||||
kubeProxyVersion: v1.6.7+coreos.0
|
||||
kubeletVersion: v1.6.7+coreos.0
|
||||
```
|
||||
|
||||
Kubernetes control plane components have been successfully updated!
|
||||
@@ -1,139 +0,0 @@
|
||||
# Kubernetes
|
||||
|
||||
The Kubernetes example provisions a 3 node Kubernetes v1.8.5 cluster. [bootkube](https://github.com/kubernetes-incubator/bootkube) is run once on a controller node to bootstrap Kubernetes control plane components as pods before exiting. An etcd3 cluster across controllers is used to back Kubernetes.
|
||||
|
||||
## Requirements
|
||||
|
||||
Ensure that you've gone through the [matchbox with docker](getting-started-docker.md) guide and understand the basics. In particular, you should be able to:
|
||||
|
||||
* Use Docker to start `matchbox`
|
||||
* Create a network boot environment with `poseidon/dnsmasq`
|
||||
* Create the example libvirt client VMs
|
||||
* `/etc/hosts` entries for `node[1-3].example.com`
|
||||
|
||||
Install [bootkube](https://github.com/kubernetes-incubator/bootkube/releases) v0.9.1 and add it on your $PATH.
|
||||
|
||||
```sh
|
||||
$ bootkube version
|
||||
Version: v0.9.1
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
The [examples](../examples) statically assign IP addresses to libvirt client VMs created by `scripts/libvirt`. The examples can be used for physical machines if you update the MAC addresses. See [network setup](network-setup.md) and [deployment](deployment.md).
|
||||
|
||||
* [bootkube](../examples/groups/bootkube) - iPXE boot a self-hosted Kubernetes cluster
|
||||
* [bootkube-install](../examples/groups/bootkube-install) - Install a self-hosted Kubernetes cluster
|
||||
|
||||
## Assets
|
||||
|
||||
Download the CoreOS Container Linux image assets referenced in the target [profile](../examples/profiles).
|
||||
|
||||
```sh
|
||||
$ ./scripts/get-coreos stable 1967.3.0 ./examples/assets
|
||||
```
|
||||
|
||||
Add your SSH public key to each machine group definition [as shown](../examples/README.md#ssh-keys).
|
||||
|
||||
```json
|
||||
{
|
||||
"profile": "bootkube-worker",
|
||||
"metadata": {
|
||||
"ssh_authorized_keys": ["ssh-rsa pub-key-goes-here"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use the `bootkube` tool to render Kubernetes manifests and credentials into an `--asset-dir`. Set the `--network-provider` to `flannel` (default) or `experimental-calico` if desired.
|
||||
|
||||
```sh
|
||||
bootkube render --asset-dir=assets --api-servers=https://node1.example.com:443 --api-server-alt-names=DNS=node1.example.com --etcd-servers=https://node1.example.com:2379
|
||||
```
|
||||
|
||||
Later, a controller will use `bootkube` to bootstrap these manifests and the credentials will be used to access your cluster.
|
||||
|
||||
## Containers
|
||||
|
||||
Use docker to start `matchbox` and mount the desired example resources. Create a network boot environment and power-on your machines. Revisit [matchbox with Docker](getting-started-docker.md) for help.
|
||||
|
||||
Client machines should boot and provision themselves. Local client VMs should network boot Container Linux and become available via SSH in about 1 minute. If you chose `bootkube-install`, notice that machines install Container Linux and then reboot (in libvirt, you must hit "power" again). Time to network boot and provision physical hardware depends on a number of factors (POST duration, boot device iteration, network speed, etc.).
|
||||
|
||||
## bootkube
|
||||
|
||||
We're ready to use bootkube to create a temporary control plane and bootstrap a self-hosted Kubernetes cluster.
|
||||
|
||||
Secure copy the etcd TLS assets to `/etc/ssl/etcd/*` on **every controller** node.
|
||||
|
||||
```sh
|
||||
for node in 'node1'; do
|
||||
scp -r assets/tls/etcd-* assets/tls/etcd core@$node.example.com:/home/core/
|
||||
ssh core@$node.example.com 'sudo mkdir -p /etc/ssl/etcd && sudo mv etcd-* etcd /etc/ssl/etcd/ && sudo chown -R etcd:etcd /etc/ssl/etcd && sudo chmod -R 500 /etc/ssl/etcd/'
|
||||
done
|
||||
```
|
||||
|
||||
Secure copy the `kubeconfig` to `/etc/kubernetes/kubeconfig` on **every node** to path activate the `kubelet.service`.
|
||||
|
||||
```sh
|
||||
for node in 'node1' 'node2' 'node3'; do
|
||||
scp assets/auth/kubeconfig core@$node.example.com:/home/core/kubeconfig
|
||||
ssh core@$node.example.com 'sudo mv kubeconfig /etc/kubernetes/kubeconfig'
|
||||
done
|
||||
```
|
||||
|
||||
Secure copy the `bootkube` generated assets to **any controller** node and run `bootkube-start` (takes ~10 minutes).
|
||||
|
||||
```sh
|
||||
scp -r assets core@node1.example.com:/home/core
|
||||
ssh core@node1.example.com 'sudo mv assets /opt/bootkube/assets && sudo systemctl start bootkube'
|
||||
```
|
||||
|
||||
Watch the Kubernetes control plane bootstrapping with the bootkube temporary api-server. You will see quite a bit of output.
|
||||
|
||||
```sh
|
||||
$ ssh core@node1.example.com 'journalctl -f -u bootkube'
|
||||
[ 299.241291] bootkube[5]: Pod Status: kube-api-checkpoint Running
|
||||
[ 299.241618] bootkube[5]: Pod Status: kube-apiserver Running
|
||||
[ 299.241804] bootkube[5]: Pod Status: kube-scheduler Running
|
||||
[ 299.241993] bootkube[5]: Pod Status: kube-controller-manager Running
|
||||
[ 299.311743] bootkube[5]: All self-hosted control plane components successfully started
|
||||
```
|
||||
|
||||
[Verify](#verify) the Kubernetes cluster is accessible once complete. Then install **important** cluster [addons](cluster-addons.md). You may cleanup the `bootkube` assets on the node, but you should keep the copy on your laptop. It contains a `kubeconfig` used to access the cluster.
|
||||
|
||||
## Verify
|
||||
|
||||
[Install kubectl](https://coreos.com/kubernetes/docs/latest/configure-kubectl.html) on your laptop. Use the generated kubeconfig to access the Kubernetes cluster. Verify that the cluster is accessible and that the apiserver, scheduler, and controller-manager are running as pods.
|
||||
|
||||
```sh
|
||||
$ export KUBECONFIG=assets/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
node1.example.com Ready 11m v1.8.5
|
||||
node2.example.com Ready 11m v1.8.5
|
||||
node3.example.com Ready 11m v1.8.5
|
||||
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system kube-apiserver-zd1k3 1/1 Running 0 7m
|
||||
kube-system kube-controller-manager-762207937-2ztxb 1/1 Running 0 7m
|
||||
kube-system kube-controller-manager-762207937-vf6bk 1/1 Running 1 7m
|
||||
kube-system kube-dns-2431531914-qc752 3/3 Running 0 7m
|
||||
kube-system kube-flannel-180mz 2/2 Running 1 7m
|
||||
kube-system kube-flannel-jjr0x 2/2 Running 0 7m
|
||||
kube-system kube-flannel-mlr9w 2/2 Running 0 7m
|
||||
kube-system kube-proxy-0jlq7 1/1 Running 0 7m
|
||||
kube-system kube-proxy-k4mjl 1/1 Running 0 7m
|
||||
kube-system kube-proxy-l4xrd 1/1 Running 0 7m
|
||||
kube-system kube-scheduler-1873228005-5d2mk 1/1 Running 0 7m
|
||||
kube-system kube-scheduler-1873228005-s4w27 1/1 Running 0 7m
|
||||
kube-system pod-checkpointer-hb960 1/1 Running 0 7m
|
||||
kube-system pod-checkpointer-hb960-node1.example.com 1/1 Running 0 6m
|
||||
```
|
||||
|
||||
## Addons
|
||||
|
||||
Install **important** cluster [addons](cluster-addons.md).
|
||||
|
||||
## Going further
|
||||
|
||||
[Learn](bootkube-upgrades.md) to upgrade a self-hosted Kubernetes cluster.
|
||||
@@ -1,30 +0,0 @@
|
||||
## Cluster Addons
|
||||
|
||||
Kubernetes clusters run cluster addons atop Kubernetes itself. Addons may be considered essential for bootstrapping (non-optional), important (highly recommended), or optional.
|
||||
|
||||
## Essential
|
||||
|
||||
Several addons are considered essential. CoreOS cluster creation tools ensure these addons are included. Kubernetes clusters deployed via the Matchbox examples or using our Terraform Modules include these addons as well.
|
||||
|
||||
### kube-proxy
|
||||
|
||||
`kube-proxy` is deployed as a DaemonSet.
|
||||
|
||||
### kube-dns
|
||||
|
||||
`kube-dns` is deployed as a Deployment.
|
||||
|
||||
## Important
|
||||
|
||||
### Container Linux Update Operator
|
||||
|
||||
The [Container Linux Update Operator](https://github.com/coreos/container-linux-update-operator) (i.e. CLUO) coordinates reboots of auto-updating Container Linux nodes so that one node reboots at a time and nodes are drained before reboot. CLUO enables the auto-update behavior Container Linux clusters are known for, but does it in a Kubernetes native way. Deploying CLUO is strongly recommended.
|
||||
|
||||
Create the `update-operator` deployment and `update-agent` DaemonSet.
|
||||
|
||||
```
|
||||
kubectl apply -f examples/addons/cluo/update-operator.yaml
|
||||
kubectl apply -f examples/addons/cluo/update-agent.yaml
|
||||
```
|
||||
|
||||
*Note, CLUO replaces `locksmithd` reboot coordination. The `update_engine` systemd unit on hosts still performs the Container Linux update check, download, and install to the inactive partition.*
|
||||
@@ -29,10 +29,8 @@
|
||||
* [matchbox with Docker](Documentation/getting-started-docker.md)
|
||||
* Clusters
|
||||
* [etcd3](Documentation/getting-started-docker.md) - Install a 3-node etcd3 cluster
|
||||
* [Kubernetes](Documentation/bootkube.md) - Install a 3-node Kubernetes v1.8.5 cluster
|
||||
* Clusters (Terraform-based)
|
||||
* [etcd3](examples/terraform/etcd3-install/README.md) - Install a 3-node etcd3 cluster
|
||||
* [Kubernetes](examples/terraform/bootkube-install/README.md) - Install a 3-node Kubernetes v1.14.1 cluster
|
||||
|
||||
## Contrib
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ These examples use [Terraform](https://www.terraform.io/intro/) as a client to M
|
||||
|-------------------------------|-------------------------------|
|
||||
| [simple-install](terraform/simple-install/) | Install Container Linux with an SSH key |
|
||||
| [etcd3-install](terraform/etcd3-install/) | Install a 3-node etcd3 cluster |
|
||||
| [bootkube-install](terraform/bootkube-install/) | Install a 3-node Kubernetes v1.14.1 cluster |
|
||||
|
||||
### Customization
|
||||
|
||||
@@ -27,8 +26,6 @@ These examples mount raw Matchbox objects into a Matchbox server's `/var/lib/mat
|
||||
| grub | CoreOS Container Linux via GRUB2 Netboot | stable/1967.3.0 | RAM | NA |
|
||||
| etcd3 | PXE boot a 3-node etcd3 cluster with proxies | stable/1967.3.0 | RAM | None |
|
||||
| etcd3-install | Install a 3-node etcd3 cluster to disk | stable/1967.3.0 | Disk | None |
|
||||
| bootkube | PXE boot a 3-node Kubernetes v1.8.5 cluster | stable/1967.3.0 | Disk | [tutorial](../Documentation/bootkube.md) |
|
||||
| bootkube-install | Install a 3-node Kubernetes v1.8.5 cluster | stable/1967.3.0 | Disk | [tutorial](../Documentation/bootkube.md) |
|
||||
|
||||
### Customization
|
||||
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
name: container-linux-update-agent
|
||||
namespace: kube-system
|
||||
spec:
|
||||
updateStrategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxUnavailable: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: container-linux-update-agent
|
||||
spec:
|
||||
containers:
|
||||
- name: update-agent
|
||||
image: quay.io/coreos/container-linux-update-operator:v0.3.1
|
||||
command:
|
||||
- "/bin/update-agent"
|
||||
volumeMounts:
|
||||
- mountPath: /var/run/dbus
|
||||
name: var-run-dbus
|
||||
- mountPath: /etc/coreos
|
||||
name: etc-coreos
|
||||
- mountPath: /usr/share/coreos
|
||||
name: usr-share-coreos
|
||||
- mountPath: /etc/os-release
|
||||
name: etc-os-release
|
||||
env:
|
||||
# read by update-agent as the node name to manage reboots for
|
||||
- name: UPDATE_AGENT_NODE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: spec.nodeName
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/master
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
volumes:
|
||||
- name: var-run-dbus
|
||||
hostPath:
|
||||
path: /var/run/dbus
|
||||
- name: etc-coreos
|
||||
hostPath:
|
||||
path: /etc/coreos
|
||||
- name: usr-share-coreos
|
||||
hostPath:
|
||||
path: /usr/share/coreos
|
||||
- name: etc-os-release
|
||||
hostPath:
|
||||
path: /etc/os-release
|
||||
@@ -1,22 +0,0 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: container-linux-update-operator
|
||||
namespace: kube-system
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: container-linux-update-operator
|
||||
spec:
|
||||
containers:
|
||||
- name: update-operator
|
||||
image: quay.io/coreos/container-linux-update-operator:v0.3.1
|
||||
command:
|
||||
- "/bin/update-operator"
|
||||
env:
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
@@ -1,183 +0,0 @@
|
||||
---
|
||||
systemd:
|
||||
units:
|
||||
- name: etcd-member.service
|
||||
enable: true
|
||||
dropins:
|
||||
- name: 40-etcd-cluster.conf
|
||||
contents: |
|
||||
[Service]
|
||||
Environment="ETCD_IMAGE_TAG=v3.2.0"
|
||||
Environment="ETCD_NAME={{.etcd_name}}"
|
||||
Environment="ETCD_ADVERTISE_CLIENT_URLS=https://{{.domain_name}}:2379"
|
||||
Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://{{.domain_name}}:2380"
|
||||
Environment="ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379"
|
||||
Environment="ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380"
|
||||
Environment="ETCD_INITIAL_CLUSTER={{.etcd_initial_cluster}}"
|
||||
Environment="ETCD_STRICT_RECONFIG_CHECK=true"
|
||||
Environment="ETCD_SSL_DIR=/etc/ssl/etcd"
|
||||
Environment="ETCD_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/server-ca.crt"
|
||||
Environment="ETCD_CERT_FILE=/etc/ssl/certs/etcd/server.crt"
|
||||
Environment="ETCD_KEY_FILE=/etc/ssl/certs/etcd/server.key"
|
||||
Environment="ETCD_CLIENT_CERT_AUTH=true"
|
||||
Environment="ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/peer-ca.crt"
|
||||
Environment="ETCD_PEER_CERT_FILE=/etc/ssl/certs/etcd/peer.crt"
|
||||
Environment="ETCD_PEER_KEY_FILE=/etc/ssl/certs/etcd/peer.key"
|
||||
Environment="ETCD_PEER_CLIENT_CERT_AUTH=true"
|
||||
- name: docker.service
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
mask: true
|
||||
- name: kubelet.path
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Watch for kubeconfig
|
||||
[Path]
|
||||
PathExists=/etc/kubernetes/kubeconfig
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: wait-for-dns.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Wait for DNS entries
|
||||
Wants=systemd-resolved.service
|
||||
Before=kubelet.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=true
|
||||
ExecStart=/bin/sh -c 'while ! /usr/bin/grep '^[^#[:space:]]' /etc/resolv.conf > /dev/null; do sleep 1; done'
|
||||
[Install]
|
||||
RequiredBy=kubelet.service
|
||||
- name: kubelet.service
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log \
|
||||
--insecure-options=image"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--allow-privileged \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns={{.k8s_dns_service_ip}} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override={{.domain_name}} \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/master \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: bootkube.service
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Bootstrap a Kubernetes control plane with a temp api-server
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/opt/bootkube
|
||||
ExecStart=/opt/bootkube/bootkube-start
|
||||
storage:
|
||||
{{ if index . "pxe" }}
|
||||
disks:
|
||||
- device: /dev/sda
|
||||
wipe_table: true
|
||||
partitions:
|
||||
- label: ROOT
|
||||
filesystems:
|
||||
- name: root
|
||||
mount:
|
||||
device: "/dev/sda1"
|
||||
format: "ext4"
|
||||
create:
|
||||
force: true
|
||||
options:
|
||||
- "-LROOT"
|
||||
{{end}}
|
||||
files:
|
||||
- path: /etc/kubernetes/kubelet.env
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=docker://gcr.io/google_containers/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.8.5
|
||||
- path: /etc/ssl/etcd/.empty
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
empty
|
||||
- path: /etc/hostname
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline:
|
||||
{{.domain_name}}
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
inline: |
|
||||
fs.inotify.max_user_watches=16184
|
||||
- path: /opt/bootkube/bootkube-start
|
||||
filesystem: root
|
||||
mode: 0544
|
||||
user:
|
||||
id: 500
|
||||
group:
|
||||
id: 500
|
||||
contents:
|
||||
inline: |
|
||||
#!/bin/bash
|
||||
# Wrapper for bootkube start
|
||||
set -e
|
||||
BOOTKUBE_ACI="${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
|
||||
BOOTKUBE_VERSION="${BOOTKUBE_VERSION:-v0.9.1}"
|
||||
BOOTKUBE_ASSETS="${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
|
||||
exec /usr/bin/rkt run \
|
||||
--trust-keys-from-https \
|
||||
--volume assets,kind=host,source=$BOOTKUBE_ASSETS \
|
||||
--mount volume=assets,target=/assets \
|
||||
--volume bootstrap,kind=host,source=/etc/kubernetes \
|
||||
--mount volume=bootstrap,target=/etc/kubernetes \
|
||||
$RKT_OPTS \
|
||||
${BOOTKUBE_ACI}:${BOOTKUBE_VERSION} \
|
||||
--net=host \
|
||||
--dns=host \
|
||||
--exec=/bootkube -- start --asset-dir=/assets "$@"
|
||||
|
||||
{{ if index . "ssh_authorized_keys" }}
|
||||
passwd:
|
||||
users:
|
||||
- name: core
|
||||
ssh_authorized_keys:
|
||||
{{ range $element := .ssh_authorized_keys }}
|
||||
- {{$element}}
|
||||
{{end}}
|
||||
{{end}}
|
||||
@@ -1,126 +0,0 @@
|
||||
---
|
||||
systemd:
|
||||
units:
|
||||
- name: docker.service
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
mask: true
|
||||
- name: kubelet.path
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Watch for kubeconfig
|
||||
[Path]
|
||||
PathExists=/etc/kubernetes/kubeconfig
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: wait-for-dns.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Wait for DNS entries
|
||||
Wants=systemd-resolved.service
|
||||
Before=kubelet.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=true
|
||||
ExecStart=/bin/sh -c 'while ! /usr/bin/grep '^[^#[:space:]]' /etc/resolv.conf > /dev/null; do sleep 1; done'
|
||||
[Install]
|
||||
RequiredBy=kubelet.service
|
||||
- name: kubelet.service
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log \
|
||||
--insecure-options=image"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--allow-privileged \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns={{.k8s_dns_service_ip}} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override={{.domain_name}} \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
storage:
|
||||
{{ if index . "pxe" }}
|
||||
disks:
|
||||
- device: /dev/sda
|
||||
wipe_table: true
|
||||
partitions:
|
||||
- label: ROOT
|
||||
filesystems:
|
||||
- name: root
|
||||
mount:
|
||||
device: "/dev/sda1"
|
||||
format: "ext4"
|
||||
create:
|
||||
force: true
|
||||
options:
|
||||
- "-LROOT"
|
||||
{{end}}
|
||||
files:
|
||||
- path: /etc/kubernetes/kubelet.env
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=docker://gcr.io/google_containers/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.8.5
|
||||
- path: /etc/ssl/etcd/.empty
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
empty
|
||||
- path: /etc/hostname
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline:
|
||||
{{.domain_name}}
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
inline: |
|
||||
fs.inotify.max_user_watches=16184
|
||||
|
||||
{{ if index . "ssh_authorized_keys" }}
|
||||
passwd:
|
||||
users:
|
||||
- name: core
|
||||
ssh_authorized_keys:
|
||||
{{ range $element := .ssh_authorized_keys }}
|
||||
- {{$element}}
|
||||
{{end}}
|
||||
{{end}}
|
||||
@@ -1,18 +0,0 @@
|
||||
{
|
||||
"id": "bootkube-controller",
|
||||
"name": "bootkube Ready Controller",
|
||||
"boot": {
|
||||
"kernel": "/assets/coreos/1967.3.0/coreos_production_pxe.vmlinuz",
|
||||
"initrd": ["/assets/coreos/1967.3.0/coreos_production_pxe_image.cpio.gz"],
|
||||
"args": [
|
||||
"initrd=coreos_production_pxe_image.cpio.gz",
|
||||
"root=/dev/sda1",
|
||||
"coreos.config.url=http://matchbox.example.com:8080/ignition?uuid=${uuid}&mac=${mac:hexhyp}",
|
||||
"coreos.first_boot=yes",
|
||||
"console=tty0",
|
||||
"console=ttyS0",
|
||||
"coreos.autologin"
|
||||
]
|
||||
},
|
||||
"ignition_id": "bootkube-controller.yaml"
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
{
|
||||
"id": "bootkube-worker",
|
||||
"name": "bootkube Ready Worker",
|
||||
"boot": {
|
||||
"kernel": "/assets/coreos/1967.3.0/coreos_production_pxe.vmlinuz",
|
||||
"initrd": ["/assets/coreos/1967.3.0/coreos_production_pxe_image.cpio.gz"],
|
||||
"args": [
|
||||
"initrd=coreos_production_pxe_image.cpio.gz",
|
||||
"root=/dev/sda1",
|
||||
"coreos.config.url=http://matchbox.example.com:8080/ignition?uuid=${uuid}&mac=${mac:hexhyp}",
|
||||
"coreos.first_boot=yes",
|
||||
"console=tty0",
|
||||
"console=ttyS0",
|
||||
"coreos.autologin"
|
||||
]
|
||||
},
|
||||
"ignition_id": "bootkube-worker.yaml"
|
||||
}
|
||||
@@ -1,184 +0,0 @@
|
||||
# Kubernetes
|
||||
|
||||
The Kubernetes example shows how to use Matchbox to network boot and provision a 3 node Kubernetes v1.14.1 cluster. This example uses [Terraform](https://www.terraform.io/intro/index.html) and a module provided by [Typhoon](https://github.com/poseidon/typhoon) to describe cluster resources. [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube) is run once to bootstrap the Kubernetes control plane.
|
||||
|
||||
## Requirements
|
||||
|
||||
Follow the getting started [tutorial](../../../Documentation/getting-started.md) to learn about matchbox and set up an environment that meets the requirements:
|
||||
|
||||
* Matchbox v0.6+ [installation](../../../Documentation/deployment.md) with gRPC API enabled
|
||||
* Matchbox provider credentials `client.crt`, `client.key`, and `ca.crt`
|
||||
* PXE [network boot](../../../Documentation/network-setup.md) environment
|
||||
* Terraform v0.11.x, [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox), and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
|
||||
* Machines with known DNS names and MAC addresses
|
||||
|
||||
If you prefer to provision QEMU/KVM VMs on your local Linux machine, set up the matchbox [development environment](../../../Documentation/getting-started-docker.md).
|
||||
|
||||
```sh
|
||||
sudo ./scripts/devnet create
|
||||
```
|
||||
|
||||
## Terraform Setup
|
||||
|
||||
Install [Terraform](https://www.terraform.io/downloads.html) v0.11.x on your system.
|
||||
|
||||
```sh
|
||||
$ terraform version
|
||||
Terraform v0.11.7
|
||||
```
|
||||
|
||||
Add the [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
|
||||
|
||||
```sh
|
||||
wget https://github.com/poseidon/terraform-provider-matchbox/releases/download/v0.2.3/terraform-provider-matchbox-v0.2.3-linux-amd64.tar.gz
|
||||
tar xzf terraform-provider-matchbox-v0.2.3-linux-amd64.tar.gz
|
||||
mv terraform-provider-matchbox-v0.2.3-linux-amd64/terraform-provider-matchbox ~/.terraform.d/plugins/terraform-provider-matchbox_v0.2.3
|
||||
```
|
||||
|
||||
Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
|
||||
|
||||
```sh
|
||||
wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.3.1/terraform-provider-ct-v0.3.1-linux-amd64.tar.gz
|
||||
tar xzf terraform-provider-ct-v0.3.1-linux-amd64.tar.gz
|
||||
mv terraform-provider-ct-v0.3.1-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.3.1
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Clone the [matchbox](https://github.com/poseidon/matchbox) project and take a look at the cluster examples.
|
||||
|
||||
```sh
|
||||
$ git clone https://github.com/poseidon/matchbox.git
|
||||
$ cd matchbox/examples/terraform/bootkube-install
|
||||
```
|
||||
|
||||
Configure the Matchbox provider to use your Matchbox API endpoint and client certificate in a `providers.tf` file.
|
||||
|
||||
```
|
||||
provider "matchbox" {
|
||||
version = "0.2.3"
|
||||
endpoint = "matchbox.example.com:8081"
|
||||
client_cert = "${file("~/.matchbox/client.crt")}"
|
||||
client_key = "${file("~/.matchbox/client.key")}"
|
||||
ca = "${file("~/.matchbox/ca.crt")}"
|
||||
}
|
||||
|
||||
provider "ct" {
|
||||
version = "0.3.1"
|
||||
}
|
||||
...
|
||||
```
|
||||
|
||||
Copy the `terraform.tfvars.example` file to `terraform.tfvars`. It defines a few variables needed for examples. Set your `ssh_authorized_key` to use in the cluster definition.
|
||||
|
||||
Note: With `cached_install="true"`, machines will PXE boot and install Container Linux from matchbox [assets](https://github.com/poseidon/matchbox/blob/master/Documentation/api.md#assets). For convenience, `scripts/get-coreos` can download needed images.
|
||||
|
||||
## Terraform
|
||||
|
||||
Initialize Terraform from the `bootkube-install` directory.
|
||||
|
||||
```sh
|
||||
terraform init
|
||||
```
|
||||
|
||||
Plan the resources to be created.
|
||||
|
||||
```sh
|
||||
$ terraform plan
|
||||
Plan: 75 to add, 0 to change, 0 to destroy.
|
||||
```
|
||||
|
||||
Terraform will configure matchbox with profiles (e.g. `cached-container-linux-install`, `bootkube-controller`, `bootkube-worker`) and add groups to match machines by MAC address to a profile. These resources declare that each machine should PXE boot and install Container Linux to disk. `node1` will provision itself as a controller, while `node2` and `node3` provision themselves as workers.
|
||||
|
||||
The module referenced in `cluster.tf` will also generate bootkube assets to `assets_dir` (exactly like the [bootkube](https://github.com/kubernetes-incubator/bootkube) binary would). These assets include Kubernetes bootstrapping and control plane manifests as well as a kubeconfig you can use to access the cluster.
|
||||
|
||||
### ssh-agent
|
||||
|
||||
Initial bootstrapping requires `bootkube.service` be started on one controller node. Terraform uses `ssh-agent` to automate this step. Add your SSH private key to `ssh-agent`, otherwise `terraform apply` will hang.
|
||||
|
||||
```sh
|
||||
ssh-add ~/.ssh/id_rsa
|
||||
ssh-add -L
|
||||
```
|
||||
|
||||
### Apply
|
||||
|
||||
Apply the changes.
|
||||
|
||||
```sh
|
||||
$ terraform apply
|
||||
module.cluster.null_resource.copy-secrets.0: Still creating... (5m0s elapsed)
|
||||
module.cluster.null_resource.copy-secrets.1: Still creating... (5m0s elapsed)
|
||||
module.cluster.null_resource.copy-secrets.2: Still creating... (5m0s elapsed)
|
||||
...
|
||||
module.cluster.null_resource.bootkube-start: Still creating... (8m40s elapsed)
|
||||
...
|
||||
```
|
||||
|
||||
Apply will then loop until it can successfully copy credentials to each machine and start the one-time Kubernetes bootstrap service. Proceed to the next step while this loops.
|
||||
|
||||
## Machines
|
||||
|
||||
Power on each machine (with PXE boot device on next boot). Machines should network boot, install Container Linux to disk, reboot, and provision themselves as bootkube controllers or workers.
|
||||
|
||||
```sh
|
||||
$ ipmitool -H node1.example.com -U USER -P PASS chassis bootdev pxe
|
||||
$ ipmitool -H node1.example.com -U USER -P PASS power on
|
||||
```
|
||||
|
||||
For local QEMU/KVM development, create the QEMU/KVM VMs.
|
||||
|
||||
```sh
|
||||
$ sudo ./scripts/libvirt create
|
||||
$ sudo ./scripts/libvirt [start|reboot|shutdown|poweroff|destroy]
|
||||
```
|
||||
|
||||
## Verify
|
||||
|
||||
[Install kubectl](https://coreos.com/kubernetes/docs/latest/configure-kubectl.html) on your laptop. Use the generated kubeconfig to access the Kubernetes cluster. Verify that the cluster is accessible and that the apiserver, scheduler, and controller-manager are running as pods.
|
||||
|
||||
```sh
|
||||
$ export KUBECONFIG=assets/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
node1.example.com Ready 11m v1.14.1
|
||||
node2.example.com Ready 11m v1.14.1
|
||||
node3.example.com Ready 11m v1.14.1
|
||||
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system coredns-1187388186-mx9rt 3/3 Running 0 11m
|
||||
kube-system coredns-1187388186-dsfk3 3/3 Running 0 11m
|
||||
kube-system flannel-fqp7f 2/2 Running 1 11m
|
||||
kube-system flannel-gnjrm 2/2 Running 0 11m
|
||||
kube-system flannel-llbgt 2/2 Running 0 11m
|
||||
kube-system kube-apiserver-7336w 1/1 Running 0 11m
|
||||
kube-system kube-controller-manager-3271970485-b9chx 1/1 Running 0 11m
|
||||
kube-system kube-controller-manager-3271970485-v30js 1/1 Running 1 11m
|
||||
kube-system kube-proxy-50sd4 1/1 Running 0 11m
|
||||
kube-system kube-proxy-bczhp 1/1 Running 0 11m
|
||||
kube-system kube-proxy-mp2fw 1/1 Running 0 11m
|
||||
kube-system kube-scheduler-3895335239-fd3l7 1/1 Running 1 11m
|
||||
kube-system kube-scheduler-3895335239-hfjv0 1/1 Running 0 11m
|
||||
kube-system pod-checkpointer-wf65d 1/1 Running 0 11m
|
||||
kube-system pod-checkpointer-wf65d-node1.example.com 1/1 Running 0 11m
|
||||
```
|
||||
|
||||
## Optional
|
||||
|
||||
Several Terraform module variables can override cluster defaults. [Check upstream](https://typhoon.psdn.io/bare-metal/) for the full list of options.
|
||||
|
||||
```hcl
|
||||
...
|
||||
cached_install = "false"
|
||||
install_disk = "/dev/sda"
|
||||
networking = "calico"
|
||||
```
|
||||
|
||||
## Addons
|
||||
|
||||
Install **important** cluster [addons](../../../Documentation/cluster-addons.md).
|
||||
|
||||
## Going Further
|
||||
|
||||
Learn more about [matchbox](../../../Documentation/matchbox.md) or explore the other [example](../) clusters.
|
||||
@@ -1,46 +0,0 @@
|
||||
// Kubernetes cluster
|
||||
module "cluster" {
|
||||
source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.14.1"
|
||||
|
||||
providers = {
|
||||
local = "local.default"
|
||||
null = "null.default"
|
||||
template = "template.default"
|
||||
tls = "tls.default"
|
||||
}
|
||||
|
||||
# bare-metal
|
||||
cluster_name = "example"
|
||||
matchbox_http_endpoint = "${var.matchbox_http_endpoint}"
|
||||
os_channel = "coreos-stable"
|
||||
os_version = "1967.3.0"
|
||||
|
||||
# default iPXE firmware (used in dnsmasq image) doesn't offer https
|
||||
download_protocol = "http"
|
||||
|
||||
# configuration
|
||||
k8s_domain_name = "cluster.example.com"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
asset_dir = "assets"
|
||||
cached_install = "true"
|
||||
|
||||
# machines
|
||||
controller_names = ["node1"]
|
||||
controller_macs = ["52:54:00:a1:9c:ae"]
|
||||
controller_domains = ["node1.example.com"]
|
||||
|
||||
worker_names = [
|
||||
"node2",
|
||||
"node3",
|
||||
]
|
||||
|
||||
worker_macs = [
|
||||
"52:54:00:b2:2f:86",
|
||||
"52:54:00:c3:61:77",
|
||||
]
|
||||
|
||||
worker_domains = [
|
||||
"node2.example.com",
|
||||
"node3.example.com",
|
||||
]
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
// Configure the matchbox provider
|
||||
provider "matchbox" {
|
||||
version = "0.2.3"
|
||||
endpoint = "${var.matchbox_rpc_endpoint}"
|
||||
client_cert = "${file("~/.matchbox/client.crt")}"
|
||||
client_key = "${file("~/.matchbox/client.key")}"
|
||||
ca = "${file("~/.matchbox/ca.crt")}"
|
||||
}
|
||||
|
||||
provider "ct" {
|
||||
version = "0.3.1"
|
||||
}
|
||||
|
||||
provider "local" {
|
||||
version = "~> 1.0"
|
||||
alias = "default"
|
||||
}
|
||||
|
||||
provider "null" {
|
||||
version = "~> 1.0"
|
||||
alias = "default"
|
||||
}
|
||||
|
||||
provider "template" {
|
||||
version = "~> 1.0"
|
||||
alias = "default"
|
||||
}
|
||||
|
||||
provider "tls" {
|
||||
version = "~> 1.0"
|
||||
alias = "default"
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
matchbox_http_endpoint = "http://matchbox.example.com:8080"
|
||||
matchbox_rpc_endpoint = "matchbox.example.com:8081"
|
||||
ssh_authorized_key = "ADD ME"
|
||||
@@ -1,14 +0,0 @@
|
||||
variable "matchbox_http_endpoint" {
|
||||
type = "string"
|
||||
description = "Matchbox HTTP read-only endpoint (e.g. http://matchbox.example.com:8080)"
|
||||
}
|
||||
|
||||
variable "matchbox_rpc_endpoint" {
|
||||
type = "string"
|
||||
description = "Matchbox gRPC API endpoint, without the protocol (e.g. matchbox.example.com:8081)"
|
||||
}
|
||||
|
||||
variable "ssh_authorized_key" {
|
||||
type = "string"
|
||||
description = "SSH public key to set as an authorized_key on machines"
|
||||
}
|
||||
Reference in New Issue
Block a user