fix: prevent EBUSY when unmounting system disk

Reading /proc/mounts while simultaneously unmounting mountpoints
prevents unmounting all submounts under /var. This is due to the fact
that /proc/mounts will change as we perform unmounts, and that causes a
read of the file to become inaccurate. We now read /proc/mounts into
memory to get a snapshot of all submounts under /var, and then we
proceed with unmounting them.

This also adds some additional logging that I found to be useful while
debugging this. It also adds logic to skip of DaemonSet managed pods.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This commit is contained in:
Andrew Rynhard
2019-09-06 06:20:05 +00:00
parent f03975bdc3
commit 37a8ce78ae
3 changed files with 22 additions and 8 deletions

View File

@@ -104,6 +104,12 @@ func (h *Helper) Drain(node string) error {
for _, pod := range pods.Items {
go func(p corev1.Pod) {
defer wg.Done()
for _, ref := range p.ObjectMeta.OwnerReferences {
if ref.Kind == "DaemonSet" {
log.Printf("skipping DaemonSet pod %s\n", p.GetName())
return
}
}
if err := h.evict(p, int64(60)); err != nil {
log.Printf("WARNING: failed to evict pod: %v", err)
}