Dear Maintainer,

After upgrading to Trixie, btrfs-progs causes the following error to
be printed from the initramfs:

    modprobe: module btrfs not found in modules.dep

I believe this is due to
/usr/share/initramfs-tools/scripts/local-premount/btrfs calling
`modprobe btrfs` even though the btrfs module may not be present in
the initramfs.  (It is not present on this system due to not having
any btrfs partitions.  btrfs-progs is installed for working with
external media.)

A few potential solutions:

1. Only add the btrfs executable to the initramfs when there is a
   btrfs partition in use (i.e. cases when the btrfs module is added).
2. Send modprobe stderr to /dev/null.
3. Unconditionally add the btrfs module to initramfs from
   /usr/share/initramfs-tools/hooks/btrfs by calling
   manual_add_modules.

I'm sure there are other solutions as well.  I'd prefer solutions
which minimize the amount of unnecessary initramfs code on systems not
using btrfs for system partitions.

Thanks,
Kevin

Hello Kevin,

Kevin Locke <kevin@kevinlocke.name> writes:

What is the output of

  grep 'MODULES=' /etc/initramfs-tools/initramfs.conf

?

Why is the additional complexity needed to implement this justified, and
how do you propose to make it robust?  Is there a precedent for this
with any other FS?  Ie: XFS?

It sounds like you know that it's not an error, but a warning, and that
for your specific use case the warning is cosmetic and can be ignored.
Meanwhile, this proposal, if implemented, would hide legitimate errors.

If "btrfs is installed for working with external media" then a user may
also expect to be use the initramfs as a lightweight rescue system in
combination with external media (ie: GRUB can still load the kernel and
the initramfs, but the user modifies the kernel command line to pivot to
the rootfs on the external media).  Thus far, this looks like the best
solution to me, and the cost is:

  76K     kernel/lib/raid6/raid6_pq.ko.xz
  8.0K    kernel/crypto/xor.ko.xz
  4.0K    kernel/lib/libcrc32c.ko.xz
  760K    kernel/fs/btrfs/btrfs.ko.xz
  848K    total

for (popcon) 20486 users, only one of whom has complained about the 848K.

4. Locally implement option #2.

5. Do nothing.

Isn't this a non-btrfs-specific wishlist bug in initramfs?

Best,
Nicholas

Hi Nicholas,

Thanks for taking the time to consider and discuss!

MODULES=most

Warning messages place a burden on system administrators of both
expertise and effort to investigate and mitigate, which is duplicated
across the number of people and systems affected.  I advise weighing
it against the burden of additional complexity on the developer(s).

For this specific solution, perhaps considering a low-complexity
implementation like `if grep -q btrfs /proc/mounts ...` would be a
reasonable place to start?  If you think an approach like this might
be acceptable, lets discuss any robustness or complexity issues you
see.  If not, is there another approach you'd consider?  Otherwise,
lets focus on solution 3.

For reference, I have not observed warnings after installing
f2fs-tools, ntfs-3g, or xfsprogs.  It appears that none of these
packages, nor zfsutils-linux, provide initramfs hooks.

I know it now that I've paid the cost to investigate the error.  Other
users will have to pay the same cost and implement the same solution.

I agree about hiding legitimate errors and am not a fan of this
solution.  Lets set it aside.

That's acceptable to me.  I'd expect using btrfs rescue media while
booted from the system initramfs on a non-btrfs system to be uncommon,
but if you think it's important or common enough to justify the size
increase, or avoid the complexity of other solutions, I'd defer to
your judgement.  It's your call.

Although I'm the only user who has taken the time to investigate and
open an issue about the warning, it's unlikely I'm the only one who
has either seen or investigated the warning.  No complaints about the
848K for my current use cases though.

Sure.  With the caveat that it's only a solution for me and not for
other users.

I'm sorry, I don't understand.  Are you suggesting initramfs-tools
could determine which code is necessary and remove unnecessary code
generically?  Or provide an interface to make it easier for hook
developers to conditionally exclude their own code?  Or a common
system for determining which code is necessary?  How would you
summarize the wishlist bug?

Thanks again for considering and discussing this issue, and for
maintaining btrfs-progs in general.

Cheers,
Kevin

missed it.  It does not include fuse.ko in the initramfs itself[1] and
only calls `modprobe fuse` if $ROOTFSTYPE or $LOOPFSTYPE is ntfs or
ntfs-3g[2] (presumably relying on the initramfs-tools to include
fuse.ko when necessary).

What would you think about using a similar approach for btrfs?

Cheers,
Kevin

[1]: https://sources.debian.org/src/ntfs-3g/1%3A2022.10.3-5/debian/local/ntfs-3g.hook
[2]: https://sources.debian.org/src/ntfs-3g/1%3A2022.10.3-5/debian/local/ntfs-3g.local-premount

Hello Kevin,

Sorry for the long delay replying.

Kevin Locke <kevin@kevinlocke.name> writes:

The definition for this is "Add most filesystem and all harddrive
drivers."


Lets talk about the module for this bug.  The utility/executable is a
secondary or tertiary concern...but more on this later! :)

This suggestion appears to describe the MODULES=dep case because it is
an example of "Try and guess which modules to load."  Agreed?  I'm
guessing we both agree, and that this is worth supporting.

Thank you for your investigation!  The problem is "presumably relying on
the initramfs-tools to include fuse.ko when necessary" is that "when
necessary" is currently appears to be buggy for btrfs.

I dug around and found another precedent at
/usr/share/initramfs-tools/hooks/lvm2:

    manual_add_modules dm_mod dm_snapshot dm_mirror dm_raid raid0 raid1 raid10 raid456

it looks to me like lvm2 unconditionally adds its modules even when the
rootfs isn't on LVM.  That's how I planned to implement option 3.
[snip]

I'm happy to hear that.

[snip]

It feels to me like the primary question is what "Add most filesystem
and all harddrive drivers." means.  At this point I think that
MODULES=most should include btrfs drivers, because btrfs is an in-tree
filesystem unlike zfs or ntfs-3g, because of the lvm2 precedent, and
because of the user experience issues you've noted.  I seem to remember
that it's also the case that dkms insures that zfs drivers are always
present in the initramfs even when the rootfs isn't zfs.  Based on what
you're saying it sounds like there's also an initramfs bug, because
MODULES=most should arguably put btrfs.ko in the initramfs even when
btrfs-progs isn't installed.

I also think that it would be reasonable to clone this bug; the second
bug would be a request to support MODULES=dep.  MODULES=dep is defined
as a "best attempt" heuristic.

Meanwhile, I think it's wrong that btrfs-progs is stuffing btrfs-convert
into everyone's initramfs when that's not even upstream's recommended
migration protocol at this time, and when "convert" is an operation an
extreme minority of people will use at most once.  At the same time, we
don't yet ship a useful fsck.btrfs (which ought to call 'btrfs check
$DEVICE', then pause so the user can photograph the screen to include in
a bug report); that should be in the initramfs when rootfs=btrfs.  Then
there are the initramfs<->btrfs race issues like #964906.

[snip]

You're welcome!  'wish I had more free time and energy to make all the
improvements I planned to back in 2012...

Cheers,
Nicholas

control: tag -1 moreinfo unreproducible
Hi,

Nicholas D Steeves <sten@debian.org> writes:

I just examined a laptop (Debian 13 trixie) that has never had
btrfs-progs installed and that has never had a btrfs device attached:

mkdir ~/unpack-initramfs
cd $_
unmkinitramfs /boot/initrd.img-6.12.95+deb13-amd64 ./
find | grep btrfs.ko
./main/usr/lib/modules/6.12.95+deb13-amd64/kernel/fs/btrfs/btrfs.ko.xz

So initramfs-tools is working correctly.  I expect that if you verify
that "MODULES=most" is still enabled, then run 'update-initramfs -u -k
all', then unpack the initramfs, we'll see that btrfs.ko is correctly
included in the initramfs.

Hi Nicholas,

No worries.  This issue seems low priority to me, and we all have
limited time.  No pressure on my end.

My apologies; I've inadvertently mislead you.  After confirming that
btrfs.ko.xz is included in the initramfs on a fresh trixie install, as
you observed, I found that I had an additional configuration file in
/etc/initramfs-tools/conf.d/ which set MODULES=dep.  Doh!  My bad.

I believe this bug only appears in the MODULES=dep case and that
MODULES=most works as intended.

What do you think about option 1 (use a heuristic to detect whether
btrfs is in use - possibly already implemented by initramfs-tools to
determine whether to include btrfs.ko) vs option 3 (unconditionally
add the btrfs module) for the MODULES=dep case?  I'd lean toward
option 1, but I wouldn't be opposed to option 3 if it's more robust or
maintainable, especially given the lvm2 precedent you noted.

Good catch!  I completely agree about skipping btrfs-convert and would
love better handling of fsck for rootfs=btrfs and the race in #964906
(although I realize those last two are likely much more involved).

You and me both!

Cheers,
Kevin

Hi Kevin,

Kevin Locke <kevin@kevinlocke.name> writes:

If that's the case, this bug should be reassigned to initramfs-tools.
In addition to currently mounted fs, the heuristic should also include
unmounted btrfs fs in /etc/fstab.  If initramfs-tools has a plugin
system that I'm unaware of then I'd appreciate it if someone could save
me some time and spoons by pointing me in the right direction.

We can do this in btrfs-progs, like lvm2.

It's also a question of policy and user expectation.  Fsck had the
elegant interface of /forcefsck (per mount, IIRC) until (IIRC)
systemd-mount killed that.  I'm not sure if reimplementing that
somewhere would be the best approach.  Two user expectations are boot
without fsck unless fatal error vs fsck if uncleanly shutdown because
I'm using dangerous btrfs features.

Cheers,
Nicholas

Hi Nicholas,

Unfortunately, I'm not aware of a plugin system beyond the shell
functions exposed to hooks.  I had hoped initramfs-tools might expose
a suitable hook function we could use, but the best option I can find
to reuse the detection logic is to check for the presence of btrfs.ko
in the initramfs as it is being built, since modules are copied before
hooks are run[1], then include the btrfs scripts/tools only when
btrfs.ko is present (or built into the kernel).

[1]: https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/v0.151/mkinitramfs?ref_type=tags#L416

Unfortunately, I don't believe the heuristic used by initramfs-tools
matches your requirements.  Based on a quick investigation:
[dep_add_modules] calls [dep_add_modules_mount] for / and /usr, which
calls [manual_add_modules] for the detected fstype of the given
mountpoint.  In effect, btrfs is included only if / or /usr are active
btrfs mounts.

[dep_add_modules]: https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/v0.151/hook-functions?ref_type=tags#L598
[dep_add_modules_mount]: https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/v0.151/hook-functions?ref_type=tags#L489
[manual_add_modules]: https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/v0.151/hook-functions?ref_type=tags#L97

What are the use cases you have in mind where btrfs would be required
in the initramfs for inactive mounts, and mounts other than / and
/usr?

This seems like a simple, workable solution to me.  Although not ideal
for initramfs size, it seems like a pragmatic choice to me.

Cheers,
Kevin