Dear Maintainer,

*** Reporter, please consider answering these questions, where appropriate ***

   * What led up to the situation?

	I should have kept track of when I noticed the issue.

	Some fateful day in May 2024, I rebooted for the first time in an
	age, I was greeted with:

	> No pool imported. Manually import the root pool
	> at the command prompt and then exit.
	> Hint: Try: zpool import -N

   * What exactly did you do (or not do) that was effective (or
     ineffective)?

	 I've settled on the following (after I'm dumped to the command prompt with
     the above 'hint')

	 zpool export -a # be certain everything exported
	 zpool import -N -R /root rootpool
	 zpool import    # show all pools, of which there are many
	 zfs mount rootpool/ROOT/debian
	 zfs mount -a
	 exit

	 I do not know what zfs property (or what else) is wrong that is
	 causing the zfs initramfs script to fail to see/recognize the rootpool.
	 I've also tried regenerating the zpool.cache, followed by the initramfs (and
	 then validating the new initramfs _contains_ the updated zpool.cache)

	 For what it's worth, my kernel (grub) commandline is:
	 /BOOT/debian@/vmlinuz-6.12.16-amd64 root=ZFS=/ROOT/debian

	 zfs-initramfs has worked for over a decade without any issue, and I'm
	 unaware of any changes that have been made to the zpool.

	 I do have some observations that apply in my particular case:

	 - `zpool import -N` is _not_ helpful; it dies horribly for me, at
	   least, as it still doesn't have an `altroot` the import needs to
	   be mounted against.
	 - It's _easy_ to import a zpool. Or search for how to do so.
	   - What isn't documented, or hinted at, is that the standard
		 `zpool import` isn't going to work, and that additional options
		 are required to mount the pool to the aforementioned altroot (`-R`)
	   - The easy-to-find [Debian Root on ZFS
		 HowTo](https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Bookworm%20Root%20on%20ZFS.html)
		 has the user mount to `/mnt/ (ie. `-R /mnt`). The `initramfs`,
		 however, expects the pool to be imported to `/root`.
	 - My suggestion would be, for the hint:
		Try: `zpool import -N; zpool import <poolname> -R /root`
		... and maybe a shortened URL to help the poor sod who ever sees the
		message.
	 - It's taken me quite a while (of on-and-off effort) to decide whether the
	   issue was with ZFS-upstream, or with the Debian package; the 'hint' I'm
	   seeing in zfs-initramfs is why I'm reporting this, as it seems the
	   appropriate place.
	 - I'm at the point where I'm not sure where/how to add debug statements to
	   the initramfs zfs script and have them 'stick' so I can see exactly
	   where and why it's failing to see my pool automatically.

   * What was the outcome of this action?

	I'm able to get the system booted, but it's not possible to
	automatically reboot the system. Manual intervention is required
	every time.

Output of `zpool get rootpool`: (Note - I've just apt upgraded to zfs-*
2.3.0-1, but haven't rebooted yet)

NAME      PROPERTY                       VALUE                          SOURCE
rootpool  size                           3.59T                          -
rootpool  capacity                       34%                            -
rootpool  altroot                        /root                          local
rootpool  health                         ONLINE                         -
rootpool  guid                           1246700263549784553            -
rootpool  version                        -                              default
rootpool  bootfs                         -                              default
rootpool  delegation                     on                             default
rootpool  autoreplace                    off                            default
rootpool  cachefile                      none                           local
rootpool  failmode                       wait                           default
rootpool  listsnapshots                  off                            default
rootpool  autoexpand                     off                            default
rootpool  dedupratio                     1.00x                          -
rootpool  free                           2.34T                          -
rootpool  allocated                      1.25T                          -
rootpool  readonly                       off                            -
rootpool  ashift                         12                             local
rootpool  comment                        -                              default
rootpool  expandsize                     -                              -
rootpool  freeing                        0                              -
rootpool  fragmentation                  9%                             -
rootpool  leaked                         0                              -
rootpool  multihost                      off                            default
rootpool  checkpoint                     -                              -
rootpool  load_guid                      16122880430695053848           -
rootpool  autotrim                       on                             local
rootpool  compatibility                  off                            default
rootpool  bcloneused                     0                              -
rootpool  bclonesaved                    0                              -
rootpool  bcloneratio                    1.00x                          -
rootpool  feature@async_destroy          enabled                        local
rootpool  feature@empty_bpobj            active                         local
rootpool  feature@lz4_compress           active                         local
rootpool  feature@multi_vdev_crash_dump  enabled                        local
rootpool  feature@spacemap_histogram     active                         local
rootpool  feature@enabled_txg            active                         local
rootpool  feature@hole_birth             active                         local
rootpool  feature@extensible_dataset     active                         local
rootpool  feature@embedded_data          active                         local
rootpool  feature@bookmarks              enabled                        local
rootpool  feature@filesystem_limits      enabled                        local
rootpool  feature@large_blocks           enabled                        local
rootpool  feature@large_dnode            enabled                        local
rootpool  feature@sha512                 enabled                        local
rootpool  feature@skein                  enabled                        local
rootpool  feature@edonr                  enabled                        local
rootpool  feature@userobj_accounting     active                         local
rootpool  feature@encryption             enabled                        local
rootpool  feature@project_quota          active                         local
rootpool  feature@device_removal         enabled                        local
rootpool  feature@obsolete_counts        enabled                        local
rootpool  feature@zpool_checkpoint       enabled                        local
rootpool  feature@spacemap_v2            active                         local
rootpool  feature@allocation_classes     enabled                        local
rootpool  feature@resilver_defer         enabled                        local
rootpool  feature@bookmark_v2            enabled                        local
rootpool  feature@redaction_bookmarks    enabled                        local
rootpool  feature@redacted_datasets      enabled                        local
rootpool  feature@bookmark_written       enabled                        local
rootpool  feature@log_spacemap           active                         local
rootpool  feature@livelist               enabled                        local
rootpool  feature@device_rebuild         enabled                        local
rootpool  feature@zstd_compress          enabled                        local
rootpool  feature@draid                  enabled                        local
rootpool  feature@zilsaxattr             active                         local
rootpool  feature@head_errlog            active                         local
rootpool  feature@blake3                 enabled                        local
rootpool  feature@block_cloning          enabled                        local
rootpool  feature@vdev_zaps_v2           active                         local

Hi,

This seems to be the problem. It should be root=ZFS=rootpool/ROOT/debian
in your case. This is due to grub uses its own zfs implementation with limited
feature sets (and often with bugs [1][2]). It would refuse to probe your root dev
in its grub-mkconfig stage if some of your pool features make it unhappy.

You could try the following in order to fix your booting process:

1. Upgrade to grub 2.12 in bookworm-backports, and if not working,
2. Hardcode root=ZFS=... options in /etc/default/grub

Also, if possible, you can use a separate boot pool with compatibility=grub2 set, or
a /boot partition with other filesystems like ext4. This could totally get rid of
grub’s zfs driver at boot time, but you still need to take care of the erroneous
kernel cmdline.

Great! Actually it took me a while to learn it should be /root when I was hit by
the same issue some time ago. The hint in initrd stage comes from [3], you could
submit a PR to upstream if you want (and I’m happy to do that also).

[1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=848945
[2]: https://github.com/openzfs/zfs/issues/13873
[3]: https://github.com/openzfs/zfs/blob/27f3d94940490d891c70e0c148f80d0c0ce09ed4/contrib/initramfs/scripts/zfs#L901-L911

Thanks,
Shengqi Chen

I realize it’s been a while since you’ve responded, but, better late than never:

Interestingly enough, it appears some upgrade of grub changed the command line? The ‘backup’ file in /boot/grub/grub.cfg that dpkg created during the upgrade actually has the correct command line - but the one that was (apparently?) modified by dpkg removed all of the root pool references. (It renamed the old file to /boot/grub/grub-cfg)

I’m running `sid`; bookworm-back ports is a downgrade! I’ve had 2.12 since Tue Sep 12 10:53:21 2023 (according to etckeeper’s history). Oddly enough, that seems to line up fairly well with when /boot/grub/grub.cfg was copied to /boot/grub/grub-cfg. (I may have set apt-get to upgrade, and left it overnight - and had to come back and answer questions the next day).

 When I looked through my grub configuration, I can’t find any reference to my root partition anywhere in /etc.

From that, I’m left thinking the root pool is autodetected

 When I look at /etc/grub.d/10_linux:

    rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`
    bootfs="`make_system_path_relative_to_its_root / | sed -e "s,@$,,"`"
    LINUX_ROOT_DEVICE="ZFS=${rpool}${bootfs%/}”


Nevertheless, after hardcoding it in, It does get padded to the /end/ and is active.

    linux   /BOOT/debian@/vmlinuz-6.12.30-amd64 root=ZFS=/ROOT/debian ro root=ZFS=rootpool/ROOT/debian ro …

I’ve always had a boot pool, but it didn’t have a `zpool set compatibility=grub2 boot pool` set until now.