Package: src:linux
Version: 6.12.94-1
Severity: important
X-Debbugs-Cc: kibi@debian.org
Hi,
I've spotted a regression regarding loop devices, starting with
v6.12.94. Going back to v6.12.90 makes the issue disappear.
I'm experiencing this issue in Debian 13 VMs that are used to build
Debian 13 system images (for amd64 and for arm64), and I'm able to
reproduce this bug by building two system images for amd64 on the same
host: the first build is fine but leaves the system in a corrupted
state, which breaks the second build (one safeguard detects the
label-based device matching finds more than 1 device, and bails).
I have yet to come up with a minimal reproducer, but I was able to find
ways to assess the “corrupted state” after the first build, without
starting the second build:
root@iot-isobuilder-52:~# fallocate -l 1G foo.img
root@iot-isobuilder-52:~# kpartx -asv foo.img
root@iot-isobuilder-52:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 1G 0 loop
├─loop0p1 259:0 0 487M 0 part
└─loop0p2 259:1 0 2.3G 0 part
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 18.9G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 1.1G 0 part [SWAP]
root@iot-isobuilder-52:~# losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC
/dev/loop0 0 0 0 0 /root/foo.img 0 512
The foo.img file doesn't exist in the first place, it is not getting
partitioned, and yet, we have both loop0p1 and loop0p2 coming up, with
sizes that match the system image built previously. Another hint is that
we have 259:0 and 259:1 as MAJ:MIN, instead of 253:0 and 253:1 when the
system image was being worked on.
Without toying with foo.img, on another system, after a single build,
while lsblk and losetup report nothing related to loop0, it's quite
clear that it's not been cleaned up properly:
root@iot-isobuilder-54:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 18.9G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 1.1G 0 part [SWAP]
root@iot-isobuilder-54:~# losetup
root@iot-isobuilder-54:~# find /sys/ -name loop0'*'|sort
/sys/block/loop0
/sys/class/block/loop0
/sys/class/block/loop0p1
/sys/class/block/loop0p2
/sys/devices/virtual/block/loop0
/sys/devices/virtual/block/loop0/loop0p1
/sys/devices/virtual/block/loop0/loop0p2
/sys/kernel/debug/block/loop0
(Going back to the older kernel confirms that loop0pX are absent there.)
Regarding the system image build, /dev/shm/system.img is allocated, then
fdisk, kpartx, mkfs.vfat, mkfs.ext4, debootstrap, apt, and finally
virt-sparsify are involved, but I haven't been able to reproduce the
problem without the debootstrap and apt steps. I'll follow up once I
have a small reproducer than my customer's image builds.
Looking at the changes between v6.12.90 and v6.12.94, the following one
seems pretty plausible:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=3cef9aa17bf7dac59095b6972454c049a5fe97c0
backported from:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=267ec4d7223a783f029a980f41b93c39b17996da
From a quick look at loop.c in v6.12.94, the code mentioned in the
commit message looks like it's in place, so if the regression is
confirmed to come from this commit (which I'm not certain of yet), it
shouldn't be caused by assumptions valid in the master branch but not in
the stable branch.
Next steps for me (in no particular order):
- find a minimal reproducer;
- check what happens with a recent mainline kernel (e.g.
7.0.12-2~bpo13+1);
- check what happens with v6.12.94 once that commit is reverted.
Cheers,
Cyril Brulebois <kibi@debian.org> (2026-07-04):
Building a 6.12.94-2 version with just that patch reverted, deploying
the resulting linux-image-6.12.94+deb13.1-amd64-unsigned package, and
building a system image leaves the system in a consistent state again:
root@iot-isobuilder-54:~# find /sys/ -name loop0'*'|sort
/sys/block/loop0
/sys/class/block/loop0
/sys/devices/virtual/block/loop0
/sys/kernel/debug/block/loop0
(i.e. no more extraneous loop0pX.)
Toying with a fresh file doesn't bring back remnants from the image
build:
root@iot-isobuilder-54:~# fallocate -l 1G foo.img
root@iot-isobuilder-54:~# kpartx -asv foo.img
root@iot-isobuilder-54:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 1G 0 loop
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 18.9G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 1.1G 0 part [SWAP]
Cleaning this up, and partitioning also gives the expected brand new
partitions (2 × ~500M):
root@iot-isobuilder-54:~# kpartx -dsv foo.img
loop deleted : /dev/loop0
root@iot-isobuilder-54:~# fdisk /dev/foo.img
[…]
root@iot-isobuilder-54:~# kpartx -asv foo.img
add map loop0p1 (253:2): 0 1024000 linear 7:0 2048
add map loop0p2 (253:3): 0 1071104 linear 7:0 1026048
root@iot-isobuilder-54:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 1G 0 loop
├─loop0p1 253:0 0 500M 0 part
└─loop0p2 253:1 0 523M 0 part
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 18.9G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 1.1G 0 part [SWAP]
And finally, a second system image build succeeds again, as there are no
duplicates that trigger the safeguard's exit condition.
Cheers,
Why kpartx and not losetup -P? kpartx uses device mapper, while losetup -P uses the internal partition support of the kernel. So the devices switch between possibly device-mapper (dynamically allocated) and blkext (fixed 259). Does this also happen with a current kernel? Bastian
Bastian Blank <waldi@debian.org> (2026-07-04): That's been working for me for many years, and I never noticed losetup -P until today. kpartx also happens to be what's used by vmdb2 in various recipes, including the basis for that particular image build. I've just confirmed the same happens with linux-image-7.0.12+deb13-amd64 found in trixie-backports, and I'm currently building a similar revert on top of it to see what happens. Cheers,
Cyril Brulebois <kibi@debian.org> (2026-07-04):
Adding trixie-backports to that system's configuration and installing
linux-image-7.0.12+deb13-amd64 from there lets me reproduce this issue
as well.
Then, building a 7.0.12-2~bpo13+2 version with a revert of commit
9e19f817940113ecfe33b1f1440407c826586b5c, and deploying the resulting
linux-{base,binary,image,modules} packages, lets me confirm the issue
disappears as well.
Cheers,
Control: tag -1 moreinfo Please try replacing kpartx with losetup. Bastian
----- Forwarded message from Bastian Blank <waldi@debian.org> ----- Date: Mon, 6 Jul 2026 08:50:35 +0200 From: Bastian Blank <waldi@debian.org> To: 1141434@bugs.debian.org Subject: Bug#1141434: linux-image-6.12.94+deb13-amd64: regression in loop devices X-Spam-Status: No, score=-5.8 tagged_above=-9999 required=6.31 tests=[DKIM_INVALID=0.1, DKIM_SIGNED=0.1, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Reply-To: Bastian Blank <waldi@debian.org>, 1141434@bugs.debian.org Control: tag -1 moreinfo Please try replacing kpartx with losetup. Bastian----- End forwarded message -----
Hi Bastian,
Hi Daan De Meyer and Jens Axboe as well,
Bastian Blank <waldi@debian.org> (2026-07-06):
While I appreciate the suggestion to switch to newer tools, I'm not sure
it's OK to sweep what seems to be an obvious kernel regression under the
rug.
With v6.12.100 becoming available for trixie (Debian 13) today I've had
a quick look at upstream commits in the stable branch, and didn't see
anything that might fix this issue. Checking mainline master, this
commit from Daan De Meyer drew my attention:
commit c4f4c0fc551cbcdccbbbc2d874d7d6440c7b4983
Author: Daan De Meyer <daan@amutable.com>
Date: Wed Jul 15 21:39:57 2026 +0200
loop: remove manually added partitions on detach
Commit 267ec4d7223a ("loop: fix partition scan race between udev and
loop_reread_partitions()") stopped disk_force_media_change() from
setting GD_NEED_PART_SCAN because loop devices with LO_FLAGS_PARTSCAN
rescan partitions explicitly. However, partitions can also be added
manually with BLKPG while LO_FLAGS_PARTSCAN is clear.
When such a loop device is detached, __loop_clr_fd() skips
bdev_disk_changed(). Without GD_NEED_PART_SCAN, reopening the unbound
device no longer performs the previous lazy cleanup, leaving dead
partition devices behind. A subsequent LOOP_CONFIGURE can then fail its
partition scan with -EBUSY, as seen in blktests loop/009 after loop/008.
Call bdev_disk_changed() unconditionally during __loop_clr_fd(). The
disk capacity is already zero and the release path holds open_mutex, so
this drops all partitions without rescanning the detached backing file.
The new blktests loop/013 case covers this sequence by adding a partition
with BLKPG without LO_FLAGS_PARTSCAN, detaching the loop device, and
checking that the partition is gone when the device is reopened.
Fixes: 267ec4d7223a ("loop: fix partition scan race between udev and loop_reread_partitions()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Closes: https://lore.kernel.org/oe-lkp/202607150754.b660f5b9-lkp@intel.com
Signed-off-by: Daan De Meyer <daan@amutable.com>
Link: https://patch.msgid.link/20260715-b4-loop-partition-cleanup-v1-1-b9f59910cd1e@amutable.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Cherry-picking it locally on top of v6.12.100 (as packaged by Debian),
and deploying it on the previously-mentioned Debian 13 systems, then
performing a few builds seems to confirm this is indeed fixing the exact
issue I've been facing.
Seeing how this commit wasn't cc'd to stable@ in the first place, I've
taken the liberty of adding both signed-off-by-ers to recipients for
information.
Full bug report on the Debian side available at:
https://bugs.debian.org/1141434
Cheers,