#1148905 fai-server: fai-diskimage cleanup() can rm -rf the build host's /dev when a umount fails

Package:
fai-server
Source:
fai-server
Submitter:
Marcus Love
Date:
2026-09-25 14:29:01 UTC
Severity:
normal
#1148905#5
Date:
2026-09-24 19:58:00 UTC
From:
To:
Dear Maintainer,

In bin/fai-diskimage, cleanup() runs "rm -rf $mnt" unconditionally after the
umount retry loop:

    for dir in $(mount | awk '{print $3}'| grep $mnt | sort -r); do
        for i in {1..15}; do
            umount $dir >&/dev/null && break
            ...
        done
        echo "ERROR: fai-diskimage cannot umount $dir" >&2
    done
    ...
    rm -rf $mnt

If a umount under $mnt fails, that mount is still live when rm -rf runs, and rm
recurses through it. When the failing mount is $mnt/dev -- which lib/updatebase
bind-mounts from the build host:

    if [ -f /etc/init.d/udev ]; then
      mount --bind /dev $FAI_ROOT/dev
      mount --make-private $FAI_ROOT/dev
#1148905#10
Date:
2026-09-25 09:54:29 UTC
From:
To:
    > A minimal fix would be to stop rm crossing the mount boundary:

    >     rm -rf --one-file-system $mnt

Thanks for the detailed description. This is now applied. See
https://salsa.debian.org/fai-team/fai/-/commit/743de14d4b4dc3b2f85393e613bccf5ba8d184f6


I'm not sure if bind mounting $target/run/udev is still needed. Do you
think we can remove it from updatebase?
#1148905#15
Date:
2026-09-25 14:13:11 UTC
From:
To:
Thanks for turning that around so fast!

On /run/udev, the commit that added it (a03e71b12a) closes #925247, where the symptom was lvm2's postinst hanging because it couldn't reach /run/udev inside /target. So the test is whether lvm2 still does that.

Possibly relevant: both that mount and the /dev bind mount sit inside if [ -f /etc/init.d/udev ]. That condition is really a sysvinit proxy, on a systemd host the file doesn't exist. That means fai-diskimage on a modern systemd host has never had /run/udev in the target. If people are building LVM images that way without hangs, that is some evidence the lvm2 problem has aged out.

If you would like, I can attempt to test since my test environment run sysvinit.

Thanks
--MKL
#1148905#20
Date:
2026-09-25 14:26:27 UTC
From:
To:
    > Thanks for turning that around so fast!
You're welcome.

    > On /run/udev, the commit that added it (a03e71b12a) closes #925247, where the symptom was lvm2's postinst hanging because it couldn't
    > reach /run/udev inside /target. So the test is whether lvm2 still does that.
No. I've just tested the FAI ISO 6.6 doing the xfce desktop
installation (which is using LVM). It does not hang any more.

I guess the bug aged out after we are using systemd during a FAI
installation.