#1121209 udev: after 259~rc1 upgrade, I cannot boot my nas with my own generated kernel

Package:
udev
Source:
udev
Description:
/dev/ and hotplug management daemon
Submitter:
valette
Date:
2025-12-20 15:27:04 UTC
Severity:
normal
Tags:
#1121209#5
Date:
2025-11-22 17:49:15 UTC
From:
To:
After systemd upgrade, my own generated kernels 6.12.57/58 do not boot anymore I'm stuck on initramfs with
no input available after lines of mdadm: No devices listed in conf file were found.

This looks exactly like this

https://unix.stackexchange.com/questions/494077/debian-mdadm-no-devices-listed-in-conf-file-were-found

However, burning an old CDROM, and booting on it, I was able to boot on my system, found everyting was working
at hardware level with md array raid10 correctly found.

I tried to reinstall a debian kernel just to test if something was mising in my kernel config that was working
great before (I have tested both 6.12.57 and 6.12.58 and forced an initramfs update just to be sure)

Official debian 6.17.8-1 and 6.12.57 both works.

Problem I did not change anything in my kernels. It looks like udev is failing either to load required modules
or to create the correct device path. But why only in my kernels and what else changed?

So there is a bad interaction between udev/kernel and mdadm and initramfs created by new dracut 109.

I have no clue where the culprit really is execpt that a kernel config option may fix it.

Note the DEVICE line pn my mdadm.conf:

 ls -l /dev/disk/by-id/
total 0
lrwxrwxrwx 1 root root  9 22 nov.  17:33 ata-ST3000DM001-1CH166_Z1F1N1R3 -> ../../sde
lrwxrwxrwx 1 root root 10 22 nov.  17:33 ata-ST3000DM001-1CH166_Z1F1N1R3-part1 -> ../../sde1
lrwxrwxrwx 1 root root  9 22 nov.  17:33 ata-ST3000DM001-9YN166_S1F0DYLW -> ../../sdb
lrwxrwxrwx 1 root root 10 22 nov.  17:33 ata-ST3000DM001-9YN166_S1F0DYLW-part1 -> ../../sdb1
lrwxrwxrwx 1 root root  9 22 nov.  17:33 ata-ST3000DM001-9YN166_W1F0LC4V -> ../../sdd
lrwxrwxrwx 1 root root 10 22 nov.  17:33 ata-ST3000DM001-9YN166_W1F0LC4V-part1 -> ../../sdd1
lrwxrwxrwx 1 root root  9 22 nov.  17:33 ata-ST4000DM004-2CV104_WFN1VEZS -> ../../sdf
lrwxrwxrwx 1 root root 10 22 nov.  17:33 ata-ST4000DM004-2CV104_WFN1VEZS-part1 -> ../../sdf1
lrwxrwxrwx 1 root root  9 22 nov.  17:33 ata-ST4000DM004-2CV104_ZFN117Q0 -> ../../sda
lrwxrwxrwx 1 root root 10 22 nov.  17:33 ata-ST4000DM004-2CV104_ZFN117Q0-part1 -> ../../sda1

cat /etc/mdadm/mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE /dev/sd[bcdef]1
DEVICE /dev/disk/by-id/ata-ST*part1
# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts

# definitions of existing MD arrays
ARRAY /dev/md0 metadata=1.2 spares=1 name=nas2:0 UUID=6abe1f20:90c629de:fadd8dc0:ca14c928
MAILADDR eric.valette@free.fr

Note that I did an /usr/bin/unmkinitramfs  /boot/initrd.img-6.12.58 /tmp

and checked /tmp/etc/mdadm/mdadm.conf there are both identical.

#1121209#10
Date:
2025-11-28 12:09:07 UTC
From:
To:
I tested two of my own generated kernel that did previously boot on my
headless nas: 6.12.57 and 6.12.58. Both failed.

So even in the eventuality, I wrongly modified kernel config for the
more recent one and did a rebuild, the previous one should have booted.

I cannot debug anything as I have no input when stuck on initramfs
prompt. I have input in BIOS and Grub stage but no more in the
initramfs. I have input once linux is booted with kernel that works...

I restarted the process I used in the past via linux-source 6.12 (this
time using 6.12.57+...).

Booted that 6.12.57 debian kernel, installed the corresponding sources,
followed
https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-building

used "make localmodconfig"

Found some bugs (I have no DRM_XE loaded, and no USB4) but the config
entry where still there. Some NEW debug information should also be
removed besides the one already mentioned in the page. Nothing really
annoying. Does not hurt.

used grub-reboot to be sure, even if it fails, to not have to plug
monitor and keyboard to debug...

And it booted.

I checked the kernel config to find differences between working and non
working kernel in MD related conf but nothing popped up (I do not use
raid456, ...)

In the meantime, I did systemd-hwdb update because probably my
/etc/database was obsolete (this hardware runs debian for more than 10
years and config was previously stored there). See bug #1120578.

So I will be unable to help anymore except I'm sure something related to
boot/initramfs did change and broke my working config.

Use this bug as a warning for people wanting to stay on longterm kernel
even if running unstable.