#1099655 initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs

Package:
initramfs-tools
Source:
initramfs-tools
Submitter:
eric
Date:
2025-07-02 11:43:02 UTC
Severity:
normal
Tags:
#1099655#5
Date:
2025-03-06 10:52:57 UTC
From:
To:
After updating and generating a new initramfs, it does not find the nvme
root fs. Booting previous kernel with previous generated initramfs (0.145)
works. Downgrading to 0.145 and rebuilding initramfs makes system work again.

Main disk is nvme disk.
#1099655#10
Date:
2025-03-07 16:10:45 UTC
From:
To:
I installed initramfs-tools 146 again, regenerated the buggy inird image
and compared the content of the 145 and 146 initrd.

After using lsinitramfs and sorting the files, it appears that in
*working* initrd, the kernel modules are not compressed while they are
in the non working one.

e.g

initrd 146:
usr/lib/modules/6.6.80/kernel/fs/ext4/ext4.ko.xz

initrd 145:
usr/lib/modules/6.6.80/kernel/fs/ext4/ext4.ko


Note that modules images are compressed in the regular fs (non initrd)
eg  : /lib/modules/6.6.80/kernel/fs/ext4/ext4.ko.xz
#1099655#15
Date:
2025-03-07 16:37:37 UTC
From:
To:
Just for the record, initramfs-tools 0.146 works on my Thinkpad T14
which also has a NVME disk.

Greetings
Marc
#1099655#18
Date:
2025-03-07 16:37:37 UTC
From:
To:
Just for the record, initramfs-tools 0.146 works on my Thinkpad T14
which also has a NVME disk.

Greetings
Marc
#1099655#23
Date:
2025-03-07 16:44:00 UTC
From:
To:
See additionnal info. What type of compression do you have for initrd
and modules?

  grep CONFIG_RD /boot/config-6.6.80
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y


# CONFIG_MODULE_COMPRESS_NONE is not set
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set

and that's rather strange as the working version, and allthougt the
config file says

COMPRESS=zstd

it is not respected! Iguess the working one is none and the non working
one is XZ in initrd.
#1099655#28
Date:
2025-03-07 16:44:00 UTC
From:
To:
See additionnal info. What type of compression do you have for initrd
and modules?

  grep CONFIG_RD /boot/config-6.6.80
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y


# CONFIG_MODULE_COMPRESS_NONE is not set
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set

and that's rather strange as the working version, and allthougt the
config file says

COMPRESS=zstd

it is not respected! Iguess the working one is none and the non working
one is XZ in initrd.
#1099655#31
Date:
2025-03-07 16:44:00 UTC
From:
To:
See additionnal info. What type of compression do you have for initrd
and modules?

  grep CONFIG_RD /boot/config-6.6.80
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y


# CONFIG_MODULE_COMPRESS_NONE is not set
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set

and that's rather strange as the working version, and allthougt the
config file says

COMPRESS=zstd

it is not respected! Iguess the working one is none and the non working
one is XZ in initrd.
#1099655#36
Date:
2025-03-07 19:43:08 UTC
From:
To:
2 [2/5411]mh@swivel:~ $ grep CONFIG_RD /boot/config-6.12.17-amd64
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_RDMA_RXE=m
# CONFIG_RDMA_SIW is not set
[3/5412]mh@swivel:~ $ grep CONFIG_MODULE /boot/config-6.12.17-amd64
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_DEBUG is not set
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
# CONFIG_MODULE_SIG_SHA1 is not set
CONFIG_MODULE_SIG_SHA256=y
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
# CONFIG_MODULE_SIG_SHA3_256 is not set
# CONFIG_MODULE_SIG_SHA3_384 is not set
# CONFIG_MODULE_SIG_SHA3_512 is not set
CONFIG_MODULE_SIG_HASH="sha256"
CONFIG_MODULE_COMPRESS=y
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set
CONFIG_MODULE_COMPRESS_ALL=y
CONFIG_MODULE_DECOMPRESS=y
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_MODULE_SIG_KEY_TYPE_RSA=y
CONFIG_MODULE_ALLOW_BTF_MISMATCH=y
[4/5413]mh@swivel:~ $

I am using the Debian kernel on unstable.

Greetings
Marc
#1099655#41
Date:
2025-03-07 19:43:08 UTC
From:
To:
2 [2/5411]mh@swivel:~ $ grep CONFIG_RD /boot/config-6.12.17-amd64
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_RDMA_RXE=m
# CONFIG_RDMA_SIW is not set
[3/5412]mh@swivel:~ $ grep CONFIG_MODULE /boot/config-6.12.17-amd64
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_DEBUG is not set
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
# CONFIG_MODULE_SIG_SHA1 is not set
CONFIG_MODULE_SIG_SHA256=y
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
# CONFIG_MODULE_SIG_SHA3_256 is not set
# CONFIG_MODULE_SIG_SHA3_384 is not set
# CONFIG_MODULE_SIG_SHA3_512 is not set
CONFIG_MODULE_SIG_HASH="sha256"
CONFIG_MODULE_COMPRESS=y
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set
CONFIG_MODULE_COMPRESS_ALL=y
CONFIG_MODULE_DECOMPRESS=y
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_MODULE_SIG_KEY_TYPE_RSA=y
CONFIG_MODULE_ALLOW_BTF_MISMATCH=y
[4/5413]mh@swivel:~ $

I am using the Debian kernel on unstable.

Greetings
Marc
#1099655#44
Date:
2025-03-07 19:43:08 UTC
From:
To:
2 [2/5411]mh@swivel:~ $ grep CONFIG_RD /boot/config-6.12.17-amd64
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_RDMA_RXE=m
# CONFIG_RDMA_SIW is not set
[3/5412]mh@swivel:~ $ grep CONFIG_MODULE /boot/config-6.12.17-amd64
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_DEBUG is not set
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
# CONFIG_MODULE_SIG_SHA1 is not set
CONFIG_MODULE_SIG_SHA256=y
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
# CONFIG_MODULE_SIG_SHA3_256 is not set
# CONFIG_MODULE_SIG_SHA3_384 is not set
# CONFIG_MODULE_SIG_SHA3_512 is not set
CONFIG_MODULE_SIG_HASH="sha256"
CONFIG_MODULE_COMPRESS=y
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set
CONFIG_MODULE_COMPRESS_ALL=y
CONFIG_MODULE_DECOMPRESS=y
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_MODULE_SIG_KEY_TYPE_RSA=y
CONFIG_MODULE_ALLOW_BTF_MISMATCH=y
[4/5413]mh@swivel:~ $

I am using the Debian kernel on unstable.

Greetings
Marc
#1099655#49
Date:
2025-03-08 11:29:06 UTC
From:
To:
On Fri, 7 Mar 2025 20:43:08 +0100 Marc Haber <mh+debian-bugs@zugschlus.de> wrote:
been so far although config has been created via the official
linux-source-6.6) and thus I cannot load compressed modules without user
space helpers that probably not exist in initrd busybox version. Of
course I can load compressed modules once the pivot_root/switch_root is
done.

So it remains a bug because the script store compressed modules in
initrd without checking this compilation flag and the kernel does not
boot. If was not doing this in 145. This is the reason for the bug I have.

A second bug is that the initrd does not seems to be compressed while
config says it should be with zstd (and the CONFIG_RD_* is ok). I can
change to xz.

A third one is that compressing compressed file is not efficient in term
of size. There has been bug opened exactly on this theme on other
distributions.

Additional note : enabling this compilation flags, cause the signature
of external modules to be incorrect as XZ with this flag *must* be used
with CRC32 checksum and not the xz default CRC64. I had to add
--check=crc32 to my signature scripts for external modules.

So I would recommend compressing the initrd image and not modules
themselves.
#1099655#59
Date:
2025-03-08 11:51:49 UTC
From:
To:
Not being a initramfs-tools maintainer, it works with the Debian kernel.
I'd recommend downgrading this bug to normal or minor. It doesn't
warrant a RC severity.

Greetings
Marc
#1099655#64
Date:
2025-03-08 13:10:09 UTC
From:
To:
I disagree. There are a lot of reasons why one can't use the default
kernels. To name a few:

	1) Version imposed by third party drivers (non free or not compiling yet),
	2) Missing config for a must use driver,
	3) or the reverse, an option that breaks boot on your config

For me the reasons are:

	1) I want to use longterm support kernel. 6.12 has not been there for a
long time after 6.6 and initial 6.12 was working rather bad with amd
gpu. I will switch to 6.12 at some point but maybe then debian will have
moved again to non long term support,
	2) I want NTFS3, not ntfs-3g
	3) I have external modules I need both free and non free,


And at least, if the initrd content depends on a feature, It should at
least check it.

But as said, regarding size or initrd, there are better ways than using
the compressed modules.
#1099655#69
Date:
2025-03-08 13:18:15 UTC
From:
To:
I understand the rationale for untouched compressed initrd kernel
modules for uncompressed initrd but at least it should check if kernel
has a chance to boot and keep old behavior if not!

And again if initrd size is important, compressing it entirely is the
best way and then size show you should have content uncompressed.

See this
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1932329
#1099655#74
Date:
2025-03-16 19:13:11 UTC
From:
To:
Control: tag -1 moreinfo unreproducible

The initramfs is supposed to use modprobe etc. from kmod, not from
busybox.  So there should be no difference in capabilities from the main
Debian system.

Does your initramfs image include the right version of modprobe?
Boot with "break=top" and run "modprobe --version" to check this.


[...]

As explained, it is intentional that the modules are not recompressed,
while other files are.  We made a trade-off between size and speed (of
mkinitramfs), and I don't intend to revisit that.

[...]
[...]

When does this signature script run?

I built a kernel from Linux 6.6.80 using the cloud-amd64 configuration
from linux-config-6.6, and an initramfs using initramfs-tools 0.146.
Module loading worked OK, so I don't believe this is a general problem.

Ben.
#1099655#81
Date:
2025-03-16 19:21:17 UTC
From:
To:
But the main system was able to laod compressed modules while the initrd
fs was not.

Just ebanbling the CONFIG_MODULE_DECOMPRESS=y made the problem vanish so
its shows somehow the module laoder was unable to decompress the modules.

It use wahever is put in. I did not make any chnage.
would be smaller. And then compressing compressed file is not a good
idea as shown in the link I have put in one of my message.

After I do a dkms. I do not want to put the signature key available
directly in the fs.

I do not use cloud config. Did you check if CONFIG_MODULE_DECOMPRESS=y
is set in this config? If yes then the bug goes untoticed as the kernel
itself decompress the modules not the module loader.
#1099655#86
Date:
2025-03-16 20:01:57 UTC
From:
To:
[...]

And yet module loading is going wrong in a way that can't be reproduced
elsewhere.  So please check.

[...]

Yes, that's a sensible policy.

I was wondering whether the modules installed in the main system are
properly signed but due to some mis-ordering the modules copied into the
initramfs are not.

Can you unpack the initramfs (with unmkinitramfs) and check that the
modules have the same contents as those installed in the main system?
(With the current version of unmkinitramfs, they will be unpacked under
a cpio<n> subdirectory.)

It is not set.

Ben.
#1099655#91
Date:
2025-03-16 22:53:35 UTC
From:
To:
kmod binaries are identical

kmod -V (system one)
kmod version 34.1
+ZSTD +XZ -ZLIB +OPENSSL
eric@pink-floyd3:~$ /tmp/cpio3/usr/bin/kmod -V (initrd one)
kmod version 34.1
+ZSTD +XZ -ZLIB +OPENSSL

But I was wondering what lib (via dlopen?), binary or build in code kmod
was using to decompress the xz modules. Because unxz binary differ
between normal fs and initrd one.

ldd /tmp/cpio3/usr/bin/kmod (paths are wrong but still)
         linux-vdso.so.1 (0x00007ffed0be3000)
         libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3
(0x00007faa89400000)
         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faa8920a000)
         libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007faa899dc000)
         libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1
(0x00007faa89140000)
         /lib64/ld-linux-x86-64.so.2 (0x00007faa89a57000)

ls -l /bin/unxz
lrwxrwxrwx 1 root root 2  1 mars  21:52 /bin/unxz -> xz
ls -l /bin/xz
-rwxr-xr-x 1 root root 92872  1 mars  21:52 /bin/xz

ls -l /tmp/cpio3/usr/bin/unxz
-rwxr-xr-x 269 eric eric 826128  6 oct.  13:30 /tmp/cpio3/usr/bin/unxz

checked manually one module decompression using both, they are identical.

more generally, the modules that exist in the CPIO and the one in the
rootfs are identical

diff -q -r /lib/modules/6.6.83 /tmp/cpio2/usr/lib/modules/6.6.83
Seulement dans /lib/modules/6.6.83: build
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto: aesni-intel.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto:
crc32-pclmul.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto:
crct10dif-pclmul.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto:
ghash-clmulni-intel.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto: sha1-ssse3.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto:
sha256-ssse3.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86/crypto:
sha512-ssse3.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/arch/x86: events
Seulement dans /lib/modules/6.6.83/kernel/arch/x86: kernel
Seulement dans /lib/modules/6.6.83/kernel/arch/x86: kvm
Seulement dans /lib/modules/6.6.83/kernel/crypto: af_alg.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: algif_hash.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: algif_skcipher.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: async_tx
Seulement dans /lib/modules/6.6.83/kernel/crypto: ccm.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: cmac.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: cryptd.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: crypto_null.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: crypto_simd.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: ecb.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: ecdh_generic.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: gcm.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: ghash-generic.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/crypto: xor.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers: acpi
Seulement dans /lib/modules/6.6.83/kernel/drivers: base
Seulement dans /lib/modules/6.6.83/kernel/drivers: bluetooth
Seulement dans /lib/modules/6.6.83/kernel/drivers: char
Seulement dans /lib/modules/6.6.83/kernel/drivers: cpufreq
Seulement dans /lib/modules/6.6.83/kernel/drivers: edac
Seulement dans /lib/modules/6.6.83/kernel/drivers: firmware
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm: amd
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm: display
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm: drm_buddy.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm: drm_exec.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm:
drm_kms_helper.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm:
drm_suballoc_helper.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm:
drm_ttm_helper.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm: scheduler
Seulement dans /lib/modules/6.6.83/kernel/drivers/gpu/drm: ttm
Seulement dans /lib/modules/6.6.83/kernel/drivers: hwmon
Seulement dans /lib/modules/6.6.83/kernel/drivers/i2c: algos
Seulement dans /lib/modules/6.6.83/kernel/drivers/i2c: i2c-mux.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers: infiniband
Seulement dans /lib/modules/6.6.83/kernel/drivers/input: evdev.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/input: joydev.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/input: misc
Seulement dans /lib/modules/6.6.83/kernel/drivers/input: sparse-keymap.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers: leds
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: linear.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: md-mod.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: multipath.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: raid0.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: raid10.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: raid1.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers/md: raid456.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/drivers: media
Seulement dans /lib/modules/6.6.83/kernel/drivers/net: wireless
Seulement dans /lib/modules/6.6.83/kernel/drivers: parport
Seulement dans /lib/modules/6.6.83/kernel/drivers: platform
Seulement dans /lib/modules/6.6.83/kernel/drivers: powercap
Seulement dans /lib/modules/6.6.83/kernel/fs: autofs
Seulement dans /lib/modules/6.6.83/kernel/fs: binfmt_misc.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/fs: configfs
Seulement dans /lib/modules/6.6.83/kernel/fs: efivarfs
Seulement dans /lib/modules/6.6.83/kernel/fs: exfat
Seulement dans /lib/modules/6.6.83/kernel/fs: fat
Seulement dans /lib/modules/6.6.83/kernel/fs: nls
Seulement dans /lib/modules/6.6.83/kernel/fs: ntfs3
Seulement dans /lib/modules/6.6.83/kernel/fs: squashfs
Seulement dans /lib/modules/6.6.83/kernel: kernel
Seulement dans /lib/modules/6.6.83/kernel/lib: crypto
Seulement dans /lib/modules/6.6.83/kernel/lib: libcrc32c.ko.xz
Seulement dans /lib/modules/6.6.83/kernel/lib: raid6
Seulement dans /lib/modules/6.6.83/kernel/net: bluetooth
Seulement dans /lib/modules/6.6.83/kernel/net: core
Seulement dans /lib/modules/6.6.83/kernel/net: ipv4
Seulement dans /lib/modules/6.6.83/kernel/net: mac80211
Seulement dans /lib/modules/6.6.83/kernel/net: netfilter
Seulement dans /lib/modules/6.6.83/kernel/net: rfkill
Seulement dans /lib/modules/6.6.83/kernel/net: wireless
Seulement dans /lib/modules/6.6.83/kernel: sound
Seulement dans /lib/modules/6.6.83/kernel: virt
Seulement dans /lib/modules/6.6.83: modules.alias
Seulement dans /lib/modules/6.6.83: modules.alias.bin
Seulement dans /lib/modules/6.6.83: modules.builtin
Seulement dans /lib/modules/6.6.83: modules.builtin.alias.bin
Seulement dans /lib/modules/6.6.83: modules.builtin.bin
Seulement dans /lib/modules/6.6.83: modules.builtin.modinfo
Seulement dans /lib/modules/6.6.83: modules.dep
Seulement dans /lib/modules/6.6.83: modules.dep.bin
Seulement dans /lib/modules/6.6.83: modules.devname
Seulement dans /lib/modules/6.6.83: modules.order
Seulement dans /lib/modules/6.6.83: modules.softdep
Seulement dans /lib/modules/6.6.83: modules.symbols
Seulement dans /lib/modules/6.6.83: modules.symbols.bin
Seulement dans /lib/modules/6.6.83: modules.weakdep
Seulement dans /lib/modules/6.6.83: updates

Strangely the "modules." files in initrd are in a diffrent CPIO (CPIO3)
than the modules themselves (CPIO2). If the cpios are extracted in
memory fs that should not change anything but worth noticing.

well I can do insmod of some modules when booted even the one that I
sign with my own key. I do not catch the order notion. The binaries are
identical so the signature is. The dependency, may be another problem
(e.g a modules is not copied but prevent another one needed to be
loaded. The depmod should take care of it but).

/tmp/cpio2/usr/lib/modules/6.6.83/kernel/fs/ext4:
total 360
-rw-r--r-- 1 eric eric 365588 13 mars  18:06 ext4.ko.xz

ls -l /lib/modules/6.6.83/kernel/fs/ext4/ext4.ko.xz
-rw-r--r-- 1 root root 365588 13 mars  18:06 
/lib/modules/6.6.83/kernel/fs/ext4/ext4.ko.xz

md5sum /tmp/cpio2/usr/lib/modules/6.6.83/kernel/fs/ext4/ext4.ko.xz
41686672fcda99eb20db4b2f7d1fa788
/tmp/cpio2/usr/lib/modules/6.6.83/kernel/fs/ext4/ext4.ko.xz
eric@pink-floyd3:~$ md5sum /lib/modules/6.6.83/kernel/fs/ext4/ext4.ko.xz
41686672fcda99eb20db4b2f7d1fa788
/lib/modules/6.6.83/kernel/fs/ext4/ext4.ko.xz

here is my complete module config if it helps (note that I keep the
DECOMPRESS so far).

grep CONFIG_MODULE /boot/config-6.6.83
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_DEBUG is not set
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
CONFIG_MODULE_SIG_ALL=y
# CONFIG_MODULE_SIG_SHA1 is not set
# CONFIG_MODULE_SIG_SHA224 is not set
CONFIG_MODULE_SIG_SHA256=y
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
CONFIG_MODULE_SIG_HASH="sha256"
# CONFIG_MODULE_COMPRESS_NONE is not set
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set
CONFIG_MODULE_DECOMPRESS=y
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"
# CONFIG_MODULE_SIG_KEY_TYPE_RSA is not set
CONFIG_MODULE_SIG_KEY_TYPE_ECDSA=y
#1099655#96
Date:
2025-03-18 17:15:06 UTC
From:
To:
Hi,

I would report something similar.

With the images built with 0.145 right now I get a pointless "Cannot open route device". And that's it, I cannot see any life signs from initrd among the long lines, at all. That means: one of my latest initrds was updated a couple of weeks ago, and that one is not booting. Similarly to one which I was created today with 0.145 (as far as I can tell).

With 0.146 it gets beyond this point, just to be stuck in the long "waiting for root fs" loop. The passwort query is NOT showing up. It throws me eventually to the rescue prompt, and I did a quick investigation of the kernel state. And all my block devices are there. It feels like it silently aborts the cryptosetup init sequence here.

Also I am pretty sure that it's not the compressor support (all relevant CONFIG_RD_... options are y).

And that all confuses me. The last working initrds which I found are build around mid of February. And I think this was already created with 0.145. So I unmkinitramfsed/diffed the initrd images for the same kernel built mid-February (with the environment from back then) and one built now (with 0.146). Looks like all the crypto-setup scripts are not present in the new version.

So it somehow misdetects the presense of a crypto-root now? Maybe because of some subtle change in the used tools?
#1099655#101
Date:
2025-03-19 12:58:39 UTC
From:
To:
For me 145 works while 146 does not without CONFIG_MODULE_DECOMPRESS
that is unneeded to load abcd.ko.xz when system is booted.
and older initramfs I downgraded to 145 version as I remembered to see
the 146 update, Thus it did a rebuild of most recent kernel initrd and
it worked when booting the same kernel that was not booting with the 146
initrd. Just to be sure I reinstalled 146 and I was stuck again in
initrd shell.

So for me, independently of other changes in various packages that may
have occurred in the mean time, with same kernel binary and package set,
145 works and 146 fails. As the difference in initrd 145 vs 146 files I
have are only compressed modules (did not make a complete md5 for all
files just checked via the name), I added CONFIG_MODULE_DECOMPRESS and I
worked again.

*But this may be a side effect* : imagine something randomly corrupts a
binary or library the initramfs in memory, the effect can be different
when having a different initramfs layout. *If* this is the root cause,
probably, in my case, it touch something that should do module
decompression and in your case something that touch crypto setup.

Now, I have no clue how to verify this hypothesis!
#1099655#106
Date:
2025-03-25 00:12:10 UTC
From:
To:
Hi Eric,

You wrote:
[...]
[...]
[...]

This all matches what I used in my local test (except for the
CONFIG_MODULE_DECOMPRESS which was disabled).  So I still don't
understand what's special about your configuration that results in the
failure.

Could you please send:

- The *complete* kernel config that you were using (before enabling
CONFIG_MODULE_DECOMPRESS).

- All the error messages from the initramfs when using this kernel and
initramfs-tools 0.146.  (A photograph of the screen is fine, if it's
readable.)

Ben.
#1099655#111
Date:
2025-03-25 23:44:46 UTC
From:
To:
As only one person has reported this and it's not been reproducible (so
far), I don't think this is critical.

Ben.
#1099655#118
Date:
2025-03-27 15:46:39 UTC
From:
To:
I switched tp 6.12.20 but same problem. Config attached.

Photo attached


No module loaded at all.
#1099655#123
Date:
2025-03-27 16:00:31 UTC
From:
To:
And just adding CONFIG_MODULE_DECOMPRESS=y makes it work on this kernel
also.
#1099655#128
Date:
2025-03-28 00:28:24 UTC
From:
To:
Indeed.  But the root device is there anyway, presumably because the
nvme driver is built-in.  So I think the "No such device" error is
actually because the *filesystem* module fails to load.

So I have more questions:

- Your kernel config has ext4 enabled as a module.  Is that the root
  filesystem type?
- What does the command "/usr/lib/klibc/bin/fstype /dev/nvme0n1p5"
  print?
- Using the bad initramfs, in the rescue shell:
  - What does "echo $ROOTFSTYPE" print?
  - What does "cat /proc/filesystems" print?
  - What does "modprobe fs-ext4" print, if anything?
  - If you then exit the rescue shell, does the boot process succeed?

Ben.
#1099655#133
Date:
2025-03-28 08:40:36 UTC
From:
To:
On 28/03/2025 01:28, Ben Hutchings wrote:

Hi Ben,

Thanks for your time and support.

I remember moving to builtin as a first try to fix the error. I can
switch back but anyway the module will be loaded so...

Or maybe the /dev is not correctly created/populated by kernel...
The no such device in the picture in the previous mail sound more like
an incorrect/impossible dev access (but I know mount error can be esoteric).

mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs
(rw,nosuid,relatime,size=7870260k,nr_inodes=1967565,mode=755,inode64)
devpts on /dev/pts type devpts
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs
(rw,nosuid,nodev,noexec,relatime,size=1576148k,mode=755,inode64)
/dev/nvme0n1p5 on / type ext4
(rw,noatime,nodiratime,discard,errors=remount-ro) <==========
securityfs on /sys/kernel/security type securityfs
(rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2
(rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs
(rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs
(rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4388)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs
(rw,nosuid,nodev,relatime,pagesize=2M)
tmpfs on /run/lock type tmpfs
(rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs
(rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/credentials/systemd-journald.service type tmpfs
(ro,nosuid,nodev,noexec,relatime,nosymfollow,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
fusectl on /sys/fs/fuse/connections type fusectl
(rw,nosuid,nodev,noexec,relatime)
systemd-1 on /boot/efi type autofs
(rw,relatime,fd=60,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7528)
systemd-1 on /multimedia type autofs
(rw,relatime,fd=61,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7530)
tmpfs on /run/credentials/systemd-networkd.service type tmpfs
(ro,nosuid,nodev,noexec,relatime,nosymfollow,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
/dev/nvme0n1p6 on /var type ext4 (rw,noatime,nodiratime,discard)
configfs on /sys/kernel/config type configfs
(rw,nosuid,nodev,noexec,relatime)
/dev/nvme0n1p7 on /usr/local type ext4 (rw,noatime,nodiratime,discard)
/dev/nvme0n1p9 on /tmp type ext4 (rw,noatime,nodiratime,discard)
/dev/nvme0n1p10 on /home type ext4 (rw,noatime,nodiratime,discard)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
/dev/sda1 on /multimedia type ntfs3
(rw,noatime,nodiratime,uid=1000,gid=1000,discard,iocharset=utf8,x-systemd.automount)
tmpfs on /run/user/1000 type tmpfs
(rw,nosuid,nodev,relatime,size=1576144k,nr_inodes=394036,mode=700,uid=1000,gid=1000,inode64)
portal on /run/user/1000/doc type fuse.portal
(rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
/dev/nvme0n1p1 on /boot/efi type vfat
(rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro,x-systemd.automount)
tmpfs on /run/user/0 type tmpfs
(rw,nosuid,nodev,relatime,size=1576144k,nr_inodes=394036,mode=700,inode64)

sfdisk -l /dev/nvme0n1
Disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: LITEON CL1-8D512
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: xxxxxxxxxxxxxxxxxxxxxxxxxxxxx <=edited

Device              Start       End   Sectors   Size Type
/dev/nvme0n1p1       2048    616447    614400   300M EFI System
/dev/nvme0n1p2     616448    878591    262144   128M Microsoft reserved
/dev/nvme0n1p3     878592 326473759 325595168 155.3G Microsoft basic data
/dev/nvme0n1p4  326475776 328153087   1677312   819M Windows recovery
environment
/dev/nvme0n1p5  328157184 406282239  78125056  37.3G Linux filesystem
/dev/nvme0n1p6  406282240 455110655  48828416  23.3G Linux filesystem
/dev/nvme0n1p7  455110656 474642431  19531776   9.3G Linux filesystem
/dev/nvme0n1p8  474642432 494174207  19531776   9.3G Linux swap
/dev/nvme0n1p9  494174208 498079743   3905536   1.9G Linux filesystem
/dev/nvme0n1p10 498079744 791048191 292968448 139.7G Linux filesystem
/dev/nvme0n1p11 791048192 893448191 102400000  48.8G Microsoft basic data
root@pink-floyd3:~#

/usr/lib/klibc/bin/fstype /dev/nvme0n1p5
FSTYPE=ext4
FSSIZE=40000028672

Will do later. I need to rebuild again the kernel.

lsmod | grep ext4
ext4                 1142784  5
crc16                  12288  3 bluetooth,amdgpu,ext4
mbcache                16384  1 ext4
jbd2                  200704  1 ext4
root@pink-floyd3:~# lsmod | grep fs
ntfs3                 335872  1
configfs               69632  1
efivarfs               28672  1
autofs4                57344  4
root@pink-floyd3:~#

No. I get first a long list of command and if I exit again a kernel
panic from memory.

More latter on.
#1099655#138
Date:
2025-03-28 09:06:35 UTC
From:
To:
Did it also from the initrd shell. Same result

Nothing!

See picture

Nothing. I tried insmod manually but maybe I should first decompress?

Confirmed.


Attached a picture of the above.
#1099655#143
Date:
2025-04-25 17:38:38 UTC
From:
To:
On Fri, 28 Mar 2025 10:06:35 +0100 Eric Valette <eric.valette@free.fr> wrote:

I think the bug #1099801 Marco d'Itri <md@Linux.IT>  closed is a
dupplicate from mine that must also be closed I guess.
#1099655#150
Date:
2025-04-30 01:19:09 UTC
From:
To:
On Fri, 2025-04-25 at 19:38 +0200, Eric Valette wrote:
[...]

Do you mean that the new version of kmod fixes this for you?

Ben.
#1099655#155
Date:
2025-04-30 08:38:25 UTC
From:
To:
If you read the changelog for #1099801 it was obvious that kmod was
unable to decompress modules as required shared lib used by dlopen was
missing and kernel was unable to do the job itself. This is exactly the
root cause of my problem and BTW if you read the comment I made in the
bug I suggested something like that.

Did the effort to validate the fix on my faulty computer by removing the
MODULES_DECOMPRESS option again (as it was not set originally) and yes
this time it boots.

So this bug can indeed be closed but it was truly due to the change in
initramfs-tool that was not tested with the old 6.1 config that was set
by linux-source-6.1
#1099655#176
Date:
2025-04-30 13:06:58 UTC
From:
To:
I read that, but I wanted to check that you had already verified the fix
because that wasn't clear to me.

I consider this a bug in kmod exposed by the change in initramfs-tools.
So I merged this with #1099801.

Ben.
#1099655#185
Date:
2025-05-22 19:27:39 UTC
From:
To:
I can reproduce this issue on Debian 12.11 (Bookworm) with kernel 6.1.0-35-amd64.

After upgrading from 12.9, my system hangs at "Loading initial ramdisk". The issue disappears when booting the fallback recovery mode or using an older kernel (6.1.0-30-amd64).

I verified that CONFIG_MODULE_DECOMPRESS is not set in the broken kernel.

The system is running on a Hyper-V Generation 2 virtual machine (EFI boot).

My version of initramfs-tools is 0.142+deb12u3, so the problem does not require 0.146 to manifest.

Let me know if I you need any other infos to reproduce the problem.

Guillaume