#1053457 live-build: Building live image fails when unmounting /sys: 'target is busy'

#1053457#5
Date:
2023-10-04 13:10:09 UTC
From:
To:
Dear Maintainer,

Building a live image with `lb build` fails towards the end with the
following error:

 P: Begin unmounting /sys...
 umount: /tmp/sid-image/chroot/sys: target is busy.
 E: An unexpected failure occurred, exiting...

The issue can be reproduced as follows:

 lb config --distribution sid --updates false --archive-areas 'main non-free-firmware' --bootloaders grub-efi
 echo live-task-lxde > config/package-lists/desktop.list.chroot
 lb build --debug

#1053457#10
Date:
2023-11-08 11:06:47 UTC
From:
To:
The reason why /sys is busy is that efivarfs is mounted under
/sys/firmware/efi/efivars.

It seems that efivarfs gets mounted at the following stage during lb build:

 P: Begin installing packages (install pass)...
 [...]
 Nov 06 19:28:28 W: Possible missing firmware /lib/firmware/amdgpu/smu_13_0_10.bin for module amdgpu
 Nov 06 19:28:29 live-boot: core filesystems dm-verity devices utils udev blockdev dns.

#1053457#15
Date:
2023-11-08 17:50:34 UTC
From:
To:
tags +unreproducible
thanks

Hello Emanuele,

I've seen your (now aborted) merge request, and your additional info in
this bug ticket. Sorry for not replying sooner.

I've run the commands that you have provided, and am unable to reproduce
your case.

lb config --distribution sid --updates false --archive-areas 'main
non-free-firmware' --bootloaders grub-efi
echo live-task-lxde > config/package-lists/desktop.list.chroot
lb build --debug

My last line in the output is:
P: Build completed successfully

I'm running (lb --version) 20230502 on sid, all commands have run as root.

1) Can you provide more information about the system that you are using
to build the image on?
2) Could you also try to run the latest git version (see [1])

With kind regards,
Roland Clobus

[1] https://wiki.debian.org/ReproducibleInstalls/LiveImages

#1053457#20
Date:
2023-11-08 21:00:31 UTC
From:
To:
Hi Roland!

Yeah 20230502 is the version I'm using too.

I can reproduce on all systems I have available, including my amd64
workstation and my arm64 Macbook M1. However, the issue is *not*
reproducible on a VM created with debvm-create.

One of the differences seem to be efivars support. Are you by any chance
running the commands in a VM or more in general on a system without
efivars? What's the output of the following?

 sudo dmesg | grep efivars:

I can reproduce on systems where the output looks like:

[    0.035071] efivars: Registered efivars operations

Whereas on the VM (issue NOT reproducible) the command does not give any
output.

  Emanuele

#1053457#25
Date:
2023-11-08 21:16:52 UTC
From:
To:
Hello Emanuele,

You may have found the crucial difference. I'm running on a sid system
which is booted via BIOS, not via UEFI. So I don't have efivars mounted
on the host system, and therefore there are no attempts inside the
chroot to mount or unmount efivarsfs.

With kind regards,
Roland Clobus

#1053457#30
Date:
2023-12-05 15:35:21 UTC
From:
To:
Also had this issue for some time (maybe 2-3 months) on my amd64
machine.

But I am not so sure it is due to efivars, at least not only.


Instrumented chroot_sysfs with mount | grep chroot, and got this

devpts-live on /home/user/tmp-live-build/20231205T051432Z/chroot/dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000,x-gvfs-hide)
proc-live on /home/user/tmp-live-build/20231205T051432Z/chroot/proc type proc (rw,relatime,x-gvfs-hide)
sysfs-live on /home/user/tmp-live-build/20231205T051432Z/chroot/sys type sysfs (rw,relatime,x-gvfs-hide)


Outside of chroot efivars is indeed mounted tho:

efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)


I think it is a bit racy, because sometimes I do see efivars being mounted under chroot.

2-3 months ago my workaround was to just add a bit of extra sleep (just
few seconds were enough) before attempting to unmount chroot/sys.

Now, this workaround does not look to be really working, so I made
another workaround - try to unmount efivars, then try to unmount sys in
a loop with a bit of sleep, until it finally succeds.

Cheers

#1053457#35
Date:
2023-12-05 19:28:13 UTC
From:
To:
Hello Witold Baryluk,

 > ... so I made another workaround - try to unmount efivars, then try
to unmount sys in
 > a loop with a bit of sleep, until it finally succeeds.

I'm running live-build for some time now on a UEFI-booted machine
(previously I used BIOS boot) without issues.

I think that the fix is applied to the git version, see
https://salsa.debian.org/live-team/live-build/-/merge_requests/326

The fix does a proper unmount of efivars before unmounting sys.

Can you try to run live-build from the git version to see whether that
fixes the issue for you?

I've written some instructions on
https://wiki.debian.org/ReproducibleInstalls/LiveImages
The key to running the git version is in the definition of the
environment variable LIVE_BUILD which should point to the root of the
git clone.

With kind regards,
Roland Clobus