Dear Maintainer,

I am using Debian unstable and on May 1st, I rebooted into kernel 6.12.25:

2025-05-01T08:52:47.280039+02:00 nimbus kernel: Linux version 6.12.25-amd64
(debian-kernel@lists.debian.org) (x86_64-linux-gnu-gcc-14 (Debian 14.2.0-19)
14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 SMP PREEMPT_DYNAMIC Debian
6.12.25-1 (2025-04-25)

At that time I still had ZFS 2.3.1-1 installed:

2025-05-01T08:52:47.280708+02:00 nimbus zed[1584]: ZFS Event Daemon 2.3.1-1
(PID 1584)
[...]
2025-05-01T08:52:47.281218+02:00 nimbus kernel: ZFS: Loaded module v2.3.1-1,
ZFS pool version 5000, ZFS filesystem version 5

And the system was running just fine.

On May 2nd, the update to ZFS 2.3.2-1 was installed:

Aptitude 0.8.13: log report
Fri, May  2 2025 22:36:58 +0200
[...]
[UPGRADE] libnvpair3linux:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] libuutil3linux:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] libzfs6linux:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] libzpool6linux:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] zfs-dkms:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] zfs-initramfs:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] zfs-zed:amd64 2.3.1-1 -> 2.3.2-1
[UPGRADE] zfsutils-linux:amd64 2.3.1-1 -> 2.3.2-1

I did not immediately reboot. Today I rebooted and my system now exhibits
kernel panics like the following:

2025-05-05T10:42:49.790841+02:00 nimbus kernel: PANIC: zroot: blkptr at
ffffb9932045c080 has no valid DVAs
2025-05-05T10:42:49.790850+02:00 nimbus kernel: Showing stack for process 1931
2025-05-05T10:42:49.790851+02:00 nimbus kernel: CPU: 14 UID: 0 PID: 1931 Comm:
z_wr_iss Tainted: P           OE      6.12.25-amd64 #1  Debian 6.12.25-1
2025-05-05T10:42:49.790852+02:00 nimbus kernel: Tainted:
[P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
2025-05-05T10:42:49.790852+02:00 nimbus kernel: Hardware name: ASUS System
Product Name/ROG STRIX B760-I GAMING WIFI, BIOS 1205 06/14/2023
2025-05-05T10:42:49.790853+02:00 nimbus kernel: Call Trace:
2025-05-05T10:42:49.790853+02:00 nimbus kernel:  <TASK>
2025-05-05T10:42:49.790853+02:00 nimbus kernel:  dump_stack_lvl+0x5d/0x80
2025-05-05T10:42:49.790854+02:00 nimbus kernel:  vcmn_err.cold+0x54/0x7f [spl]
2025-05-05T10:42:49.790854+02:00 nimbus kernel:  zfs_panic_recover+0x79/0xa0
[zfs]
2025-05-05T10:42:49.790855+02:00 nimbus kernel:
zfs_blkptr_verify_log+0xba/0x190 [zfs]
2025-05-05T10:42:49.790855+02:00 nimbus kernel:  zfs_blkptr_verify+0x15a/0x5e0
[zfs]
2025-05-05T10:42:49.790855+02:00 nimbus kernel:  ?
bp_get_dsize_sync+0x124/0x160 [zfs]
2025-05-05T10:42:49.790856+02:00 nimbus kernel:  dbuf_write_ready+0xf5/0x410
[zfs]
2025-05-05T10:42:49.790856+02:00 nimbus kernel:  arc_write_ready+0xe9/0x560
[zfs]
2025-05-05T10:42:49.790856+02:00 nimbus kernel:  ? mutex_lock+0x12/0x30
2025-05-05T10:42:49.790857+02:00 nimbus kernel:  zio_ready+0x4b/0x400 [zfs]
2025-05-05T10:42:49.790857+02:00 nimbus kernel:  zio_execute+0x8f/0x130 [zfs]
2025-05-05T10:42:49.790857+02:00 nimbus kernel:  taskq_thread+0x352/0x6f0 [spl]
2025-05-05T10:42:49.790858+02:00 nimbus kernel:  ?
__pfx_default_wake_function+0x10/0x10
2025-05-05T10:42:49.790858+02:00 nimbus kernel:  ? __pfx_zio_execute+0x10/0x10
[zfs]
2025-05-05T10:42:49.790859+02:00 nimbus kernel:  ? __pfx_taskq_thread+0x10/0x10
[spl]
2025-05-05T10:42:49.790859+02:00 nimbus kernel:  kthread+0xcf/0x100
2025-05-05T10:42:49.790870+02:00 nimbus kernel:  ? __pfx_kthread+0x10/0x10
2025-05-05T10:42:49.790870+02:00 nimbus kernel:  ret_from_fork+0x31/0x50
2025-05-05T10:42:49.790871+02:00 nimbus kernel:  ? __pfx_kthread+0x10/0x10
2025-05-05T10:42:49.790871+02:00 nimbus kernel:  ret_from_fork_asm+0x1a/0x30
2025-05-05T10:42:49.790871+02:00 nimbus kernel:  </TASK>

or blocks like

2025-05-05T10:18:26.838905+02:00 nimbus kernel: INFO: task txg_sync:811 blocked
for more than 120 seconds.
2025-05-05T10:18:26.838942+02:00 nimbus kernel:       Tainted: P           OE
6.12.25-amd64 #1 Debian 6.12.25-1
2025-05-05T10:18:26.838944+02:00 nimbus kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
2025-05-05T10:18:26.838946+02:00 nimbus kernel: task:txg_sync        state:D
stack:0     pid:811   tgid:811   ppid:2      flags:0x00004000
2025-05-05T10:18:26.842221+02:00 nimbus kernel: Call Trace:
2025-05-05T10:18:26.842243+02:00 nimbus kernel:  <TASK>
2025-05-05T10:18:26.847107+02:00 nimbus kernel:  __schedule+0x505/0xbf0
2025-05-05T10:18:26.847126+02:00 nimbus kernel:  schedule+0x27/0xf0
2025-05-05T10:18:26.847128+02:00 nimbus kernel:  schedule_timeout+0x9e/0x160
2025-05-05T10:18:26.850319+02:00 nimbus kernel:  ?
__pfx_process_timeout+0x10/0x10
2025-05-05T10:18:26.850338+02:00 nimbus kernel:  io_schedule_timeout+0x51/0x70
2025-05-05T10:18:26.855185+02:00 nimbus kernel:
__cv_timedwait_common+0x138/0x170 [spl]
2025-05-05T10:18:26.855203+02:00 nimbus kernel:  ?
__pfx_autoremove_wake_function+0x10/0x10
2025-05-05T10:18:26.855205+02:00 nimbus kernel:  __cv_timedwait_io+0x19/0x20
[spl]
2025-05-05T10:18:26.857951+02:00 nimbus kernel:  zio_wait+0x14e/0x2f0 [zfs]
2025-05-05T10:18:26.862423+02:00 nimbus kernel:  dsl_pool_sync+0xf2/0x510 [zfs]
2025-05-05T10:18:26.862445+02:00 nimbus kernel:  spa_sync+0x577/0x1070 [zfs]
2025-05-05T10:18:26.866748+02:00 nimbus kernel:  ?
spa_txg_history_init_io+0x115/0x120 [zfs]
2025-05-05T10:18:26.866766+02:00 nimbus kernel:  txg_sync_thread+0x20a/0x3b0
[zfs]
2025-05-05T10:18:26.870416+02:00 nimbus kernel:  ?
__pfx_txg_sync_thread+0x10/0x10 [zfs]
2025-05-05T10:18:26.870434+02:00 nimbus kernel:  ?
__pfx_thread_generic_wrapper+0x10/0x10 [spl]
2025-05-05T10:18:26.870436+02:00 nimbus kernel:
thread_generic_wrapper+0x5a/0x70 [spl]
2025-05-05T10:18:26.874860+02:00 nimbus kernel:  kthread+0xcf/0x100
2025-05-05T10:18:26.874878+02:00 nimbus kernel:  ? __pfx_kthread+0x10/0x10
2025-05-05T10:18:26.874880+02:00 nimbus kernel:  ret_from_fork+0x31/0x50
2025-05-05T10:18:26.877755+02:00 nimbus kernel:  ? __pfx_kthread+0x10/0x10
2025-05-05T10:18:26.877773+02:00 nimbus kernel:  ret_from_fork_asm+0x1a/0x30
2025-05-05T10:18:26.882013+02:00 nimbus kernel:  </TASK>

or

2025-05-05T10:18:26.882032+02:00 nimbus kernel: INFO: task z_wr_iss:970 blocked
for more than 120 seconds.
2025-05-05T10:18:26.882034+02:00 nimbus kernel:       Tainted: P           OE
6.12.25-amd64 #1 Debian 6.12.25-1
2025-05-05T10:18:26.886399+02:00 nimbus kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
2025-05-05T10:18:26.886417+02:00 nimbus kernel: task:z_wr_iss        state:D
stack:0     pid:970   tgid:970   ppid:2      flags:0x00004000
2025-05-05T10:18:26.886420+02:00 nimbus kernel: Call Trace:
2025-05-05T10:18:26.890318+02:00 nimbus kernel:  <TASK>
2025-05-05T10:18:26.890336+02:00 nimbus kernel:  __schedule+0x505/0xbf0
2025-05-05T10:18:26.890338+02:00 nimbus kernel:  schedule+0x27/0xf0
2025-05-05T10:18:26.890340+02:00 nimbus kernel:  vcmn_err.cold+0x69/0x7f [spl]
2025-05-05T10:18:26.890341+02:00 nimbus kernel:  zfs_panic_recover+0x79/0xa0
[zfs]
2025-05-05T10:18:26.898558+02:00 nimbus kernel:
zfs_blkptr_verify_log+0xba/0x190 [zfs]
2025-05-05T10:18:26.898577+02:00 nimbus kernel:  zfs_blkptr_verify+0x15a/0x5e0
[zfs]
2025-05-05T10:18:26.902624+02:00 nimbus kernel:  ?
bp_get_dsize_sync+0x124/0x160 [zfs]
2025-05-05T10:18:26.902643+02:00 nimbus kernel:  dbuf_write_ready+0xf5/0x410
[zfs]
2025-05-05T10:18:26.906165+02:00 nimbus kernel:  arc_write_ready+0xe9/0x560
[zfs]
2025-05-05T10:18:26.906183+02:00 nimbus kernel:  ? mutex_lock+0x12/0x30
2025-05-05T10:18:26.910169+02:00 nimbus kernel:  zio_ready+0x4b/0x400 [zfs]
2025-05-05T10:18:26.910187+02:00 nimbus kernel:  zio_execute+0x8f/0x130 [zfs]
2025-05-05T10:18:26.910189+02:00 nimbus kernel:  taskq_thread+0x352/0x6f0 [spl]
2025-05-05T10:18:26.913818+02:00 nimbus kernel:  ?
__pfx_default_wake_function+0x10/0x10
2025-05-05T10:18:26.913836+02:00 nimbus kernel:  ? __pfx_zio_execute+0x10/0x10
[zfs]
2025-05-05T10:18:26.918401+02:00 nimbus kernel:  ? __pfx_taskq_thread+0x10/0x10
[spl]
2025-05-05T10:18:26.918419+02:00 nimbus kernel:  kthread+0xcf/0x100
2025-05-05T10:18:26.918421+02:00 nimbus kernel:  ? __pfx_kthread+0x10/0x10
2025-05-05T10:18:26.922779+02:00 nimbus kernel:  ret_from_fork+0x31/0x50
2025-05-05T10:18:26.922797+02:00 nimbus kernel:  ? __pfx_kthread+0x10/0x10
2025-05-05T10:18:26.922800+02:00 nimbus kernel:  ret_from_fork_asm+0x1a/0x30
2025-05-05T10:18:26.926833+02:00 nimbus kernel:  </TASK>

or

2025-05-05T10:18:26.926852+02:00 nimbus kernel: INFO: task exe:2620 blocked for
more than 120 seconds.
2025-05-05T10:18:26.926854+02:00 nimbus kernel:       Tainted: P           OE
6.12.25-amd64 #1 Debian 6.12.25-1
2025-05-05T10:18:26.930859+02:00 nimbus kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
2025-05-05T10:18:26.930877+02:00 nimbus kernel: task:exe             state:D
stack:0     pid:2620  tgid:2609  ppid:2489   flags:0x00000002
2025-05-05T10:18:26.930879+02:00 nimbus kernel: Call Trace:
2025-05-05T10:18:26.934801+02:00 nimbus kernel:  <TASK>
2025-05-05T10:18:26.934826+02:00 nimbus kernel:  __schedule+0x505/0xbf0
2025-05-05T10:18:26.937894+02:00 nimbus kernel:  ?
arc_buf_alloc_impl.isra.0+0x28e/0x300 [zfs]
2025-05-05T10:18:26.937912+02:00 nimbus kernel:  schedule+0x27/0xf0
2025-05-05T10:18:26.937915+02:00 nimbus kernel:
schedule_preempt_disabled+0x15/0x30
2025-05-05T10:18:26.942182+02:00 nimbus kernel:
__mutex_lock.constprop.0+0x3d0/0x6d0
2025-05-05T10:18:26.942200+02:00 nimbus kernel:  dbuf_dirty+0x4d/0x9b0 [zfs]
2025-05-05T10:18:26.945670+02:00 nimbus kernel:  dbuf_dirty+0x78b/0x9b0 [zfs]
2025-05-05T10:18:26.945689+02:00 nimbus kernel:  dnode_setdirty+0x96/0xf0 [zfs]
2025-05-05T10:18:26.949656+02:00 nimbus kernel:  dbuf_dirty+0x8b1/0x9b0 [zfs]
2025-05-05T10:18:26.949674+02:00 nimbus kernel:  sa_attr_op+0x27a/0x3d0 [zfs]
2025-05-05T10:18:26.949676+02:00 nimbus kernel:  sa_bulk_update_impl+0x62/0x100
[zfs]
2025-05-05T10:18:26.954273+02:00 nimbus kernel:  sa_bulk_update+0x50/0x90 [zfs]
2025-05-05T10:18:26.954290+02:00 nimbus kernel:  zfs_dirty_inode+0x2ab/0x3a0
[zfs]
2025-05-05T10:18:26.958278+02:00 nimbus kernel:  zpl_dirty_inode+0x2b/0x40
[zfs]
2025-05-05T10:18:26.958296+02:00 nimbus kernel:  __mark_inode_dirty+0x54/0x350
2025-05-05T10:18:26.958298+02:00 nimbus kernel:  generic_update_time+0x4e/0x60
2025-05-05T10:18:26.958300+02:00 nimbus kernel:  touch_atime+0xed/0x120
2025-05-05T10:18:26.962268+02:00 nimbus kernel:  zpl_iter_read+0x17b/0x190
[zfs]
2025-05-05T10:18:26.962286+02:00 nimbus kernel:  vfs_read+0x299/0x370
2025-05-05T10:18:26.962289+02:00 nimbus kernel:  __x64_sys_pread64+0x98/0xd0
2025-05-05T10:18:26.965801+02:00 nimbus kernel:  do_syscall_64+0x82/0x190
2025-05-05T10:18:26.965819+02:00 nimbus kernel:  ? eventfd_write+0xe2/0x210
2025-05-05T10:18:26.965821+02:00 nimbus kernel:  ? aa_file_perm+0x122/0x4d0
2025-05-05T10:18:26.970287+02:00 nimbus kernel:  ?
cgroup_rstat_updated+0x69/0x220
2025-05-05T10:18:26.970304+02:00 nimbus kernel:  ?
kmem_cache_alloc_noprof+0x106/0x2f0
2025-05-05T10:18:26.970306+02:00 nimbus kernel:  ? posix_lock_inode+0x516/0xa40
2025-05-05T10:18:26.970307+02:00 nimbus kernel:  ? fcntl_setlk+0x272/0x400
2025-05-05T10:18:26.974292+02:00 nimbus kernel:  ?
__lruvec_stat_mod_folio+0x83/0xd0
2025-05-05T10:18:26.974308+02:00 nimbus kernel:  ? do_fcntl+0x5e9/0x740
2025-05-05T10:18:26.974310+02:00 nimbus kernel:  ? __x64_sys_fcntl+0x87/0xe0
2025-05-05T10:18:26.974311+02:00 nimbus kernel:  ?
syscall_exit_to_user_mode+0x4d/0x210
2025-05-05T10:18:26.974312+02:00 nimbus kernel:  ? do_syscall_64+0x8e/0x190
2025-05-05T10:18:26.974313+02:00 nimbus kernel:  ?
__count_memcg_events+0x53/0xf0
2025-05-05T10:18:26.974314+02:00 nimbus kernel:  ?
count_memcg_events.constprop.0+0x1a/0x30
2025-05-05T10:18:26.974315+02:00 nimbus kernel:  ? handle_mm_fault+0x1bb/0x2c0
2025-05-05T10:18:26.974316+02:00 nimbus kernel:  ?
do_user_addr_fault+0x36c/0x620
2025-05-05T10:18:26.974316+02:00 nimbus kernel:  ? exc_page_fault+0x7e/0x180
2025-05-05T10:18:26.974317+02:00 nimbus kernel:
entry_SYSCALL_64_after_hwframe+0x76/0x7e
2025-05-05T10:18:26.981642+02:00 nimbus kernel: RIP: 0033:0x414cae
2025-05-05T10:18:26.981655+02:00 nimbus kernel: RSP: 002b:000000c000051750
EFLAGS: 00000212 ORIG_RAX: 0000000000000011
2025-05-05T10:18:26.981657+02:00 nimbus kernel: RAX: ffffffffffffffda RBX:
0000000000000003 RCX: 0000000000414cae
2025-05-05T10:18:26.981658+02:00 nimbus kernel: RDX: 0000000000000040 RSI:
000000c00031e680 RDI: 0000000000000003
2025-05-05T10:18:26.981659+02:00 nimbus kernel: RBP: 000000c000051790 R08:
0000000000000000 R09: 0000000000000000
2025-05-05T10:18:26.981660+02:00 nimbus kernel: R10: 0000000000000000 R11:
0000000000000212 R12: 000000c00031e680
2025-05-05T10:18:26.981660+02:00 nimbus kernel: R13: 0000000000000080 R14:
000000c000002380 R15: 0000000000000000
2025-05-05T10:18:26.981661+02:00 nimbus kernel:  </TASK>

or

2025-05-05T10:18:26.981662+02:00 nimbus kernel: INFO: task fish:2909 blocked
for more than 120 seconds.
2025-05-05T10:18:26.990200+02:00 nimbus kernel:       Tainted: P           OE
6.12.25-amd64 #1 Debian 6.12.25-1
2025-05-05T10:18:26.990218+02:00 nimbus kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
2025-05-05T10:18:26.990220+02:00 nimbus kernel: task:fish            state:D
stack:0     pid:2909  tgid:2909  ppid:1      flags:0x00000006
2025-05-05T10:18:26.990222+02:00 nimbus kernel: Call Trace:
2025-05-05T10:18:26.990223+02:00 nimbus kernel:  <TASK>
2025-05-05T10:18:26.993774+02:00 nimbus kernel:  __schedule+0x505/0xbf0
2025-05-05T10:18:26.993792+02:00 nimbus kernel:  schedule+0x27/0xf0
2025-05-05T10:18:26.993794+02:00 nimbus kernel:
schedule_preempt_disabled+0x15/0x30
2025-05-05T10:18:26.993796+02:00 nimbus kernel:
__mutex_lock.constprop.0+0x3d0/0x6d0
2025-05-05T10:18:26.998303+02:00 nimbus kernel:  dbuf_find+0xe1/0x250 [zfs]
2025-05-05T10:18:26.998321+02:00 nimbus kernel:  dbuf_hold_impl+0x6f/0x7e0
[zfs]
2025-05-05T10:18:26.998324+02:00 nimbus kernel:  ? dbuf_find+0x1b6/0x250 [zfs]
2025-05-05T10:18:26.998325+02:00 nimbus kernel:  dbuf_hold_impl+0x4d4/0x7e0
[zfs]
2025-05-05T10:18:27.001873+02:00 nimbus kernel:  dbuf_hold+0x31/0x60 [zfs]
2025-05-05T10:18:27.001892+02:00 nimbus kernel:  dnode_hold_impl+0x100/0x1310
[zfs]
2025-05-05T10:18:27.001894+02:00 nimbus kernel:  ?
zfs_znode_hold_enter+0x118/0x170 [zfs]
2025-05-05T10:18:27.001896+02:00 nimbus kernel:  dmu_bonus_hold+0x3c/0x90 [zfs]
2025-05-05T10:18:27.005814+02:00 nimbus kernel:  zfs_zget+0x70/0x290 [zfs]
2025-05-05T10:18:27.005832+02:00 nimbus kernel:  zfs_dirent_lock+0x42b/0x6c0
[zfs]
2025-05-05T10:18:27.005834+02:00 nimbus kernel:  zfs_dirlook+0xb4/0x320 [zfs]
2025-05-05T10:18:27.005836+02:00 nimbus kernel:  ? zfs_zaccess+0x26f/0x450
[zfs]
2025-05-05T10:18:27.009808+02:00 nimbus kernel:  zfs_lookup+0x264/0x410 [zfs]
2025-05-05T10:18:27.009826+02:00 nimbus kernel:  zpl_lookup+0xd9/0x2d0 [zfs]
2025-05-05T10:18:27.009829+02:00 nimbus kernel:  lookup_one_qstr_excl+0x6f/0xa0
2025-05-05T10:18:27.009830+02:00 nimbus kernel:  filename_create+0xc6/0x1a0
2025-05-05T10:18:27.009831+02:00 nimbus kernel:  do_mkdirat+0x61/0x180
2025-05-05T10:18:27.013921+02:00 nimbus kernel:  __x64_sys_mkdir+0x46/0x70
2025-05-05T10:18:27.013939+02:00 nimbus kernel:  do_syscall_64+0x82/0x190
2025-05-05T10:18:27.013941+02:00 nimbus kernel:  ?
syscall_exit_to_user_mode+0x4d/0x210
2025-05-05T10:18:27.013943+02:00 nimbus kernel:  ? do_syscall_64+0x8e/0x190
2025-05-05T10:18:27.013945+02:00 nimbus kernel:  ? exc_page_fault+0x7e/0x180
2025-05-05T10:18:27.013946+02:00 nimbus kernel:
entry_SYSCALL_64_after_hwframe+0x76/0x7e
2025-05-05T10:18:27.018264+02:00 nimbus kernel: RIP: 0033:0x7f36923ac687
2025-05-05T10:18:27.018283+02:00 nimbus kernel: RSP: 002b:00007ffff6008618
EFLAGS: 00000246 ORIG_RAX: 0000000000000053
2025-05-05T10:18:27.018285+02:00 nimbus kernel: RAX: ffffffffffffffda RBX:
00007ffff6008620 RCX: 00007f36923ac687
2025-05-05T10:18:27.018287+02:00 nimbus kernel: RDX: 0000000000000019 RSI:
00000000000001c0 RDI: 00007ffff6008620
2025-05-05T10:18:27.018288+02:00 nimbus kernel: RBP: 00007ffff60088e0 R08:
fffefffefffcfcff R09: 632e2f6e6f6c6c65
2025-05-05T10:18:27.018289+02:00 nimbus kernel: R10: 8080808080808080 R11:
0000000000000246 R12: 000055a4a068ef70
2025-05-05T10:18:27.021717+02:00 nimbus kernel: R13: 00007ffff6008910 R14:
000055a4a67cd110 R15: 0000000000000019
2025-05-05T10:18:27.021735+02:00 nimbus kernel:  </TASK>

This happens when accessing files in the ZFS. In my case, reproducibly,
~/.config in my main user is affected which leads to all kinds of system
misbehavior, as all applications reading configuration from there just block
(i.e. you can not even open a fish shell) and also the root pool cannot even be
unmounted at shutdown any more.

Luckily I can still boot into linux 6.12.17-1 with zfs 2.3.1-1 which works.

Hi,

This is due to stricter bptr checks added in 2.3.2 [1]. It means that your on-disk
zfs structure is somehow corrupted logically, but not a problem introduced by 2.3.2.
This might come from bugs in zfs code, or even cosmic ray (if you do not have ECC memory).

I would recommend running a scrub on your pool with a previous version, or even better,
recreate the dataset totally (by zfs send | zfs recv).

[1]: https://github.com/openzfs/zfs/commit/0433523ca2632a175546e0cc99d1812de2a90d55

Thanks,
Shengqi Chen

I just ran a "zpool scrub" on my pool with version 2.3.1-1 and got:

scan: scrub repaired 0B in 00:16:17 with 0 errors on Tue May  6
17:06:50 2025

Before I do "zfs send | zfs recv", I'd like to ask what the reasoning
is that this should be able to fix it, if a scrub cannot?

Isn't there any "fsck" I can do right now that actually finds the issue
before I invest time (and also money for the external storage) in the
"zfs send | zfs recv" attempt and in the worst case find out that it
didn't help either?

Hi,

That’s bad news :-(.
underlying blocks (of one specific snapshot). In the process the block
pointers would be rewritten (since DVA, i.e. Data Virtual Address, or
blocks of the received new dataset would definitely differ). I think
(but am not certain) in your case that one block has no valid DVA, the
error could be either corrected (e.g. dropping the blkptr), or at least
detected in an more elegant way.

Disclaimer: I have not checked the related code now.

Sorry but with my knowledge to zfs, no other tool is available.

I could give some personal suggestions, but please always make sure
you have backups of important files (at least snapshots!) before trying:

Since the problem originates from one single directory (~/.config),
try to copy all its content to another location without copy-on-write,
e.g.:

mv ~/.config ~/.config_old
cp -ar --reflink=never ~/.config_old ~/.config
# after check
rm -rf ~/.config_old

You can create a tar and untar to replace the cp.
If you encountered any errors, you can try zdb -r further.
This could “force” zfs to produce new blocks for these objects.
The corrupted blkptr will get removed if it is not referenced (but
be aware that there could be snapshots referring to it).

Thanks,
Shengqi Chen

I'm not sure this is actually true. It's just that I noticed that
at least ~/.config/fish/* is affected as I am unable to open a "fish"
shell as that user.

On the other hand, the stack traces I provided in my bug report also
show other processes besides "fish" to be affected (e.g. txg_sync,
z_wr_iss, ...), so I am not sure how "local" the issue is.

That's actually an interesting idea. So I could definitely - with
little effort - do that for ~/.config/fish/* and then verify whether a
"fish" shell successfully opens afterwards and whether at least that
stack trace for "fish" does not appear any more.

I think I would require some pointers to what to do with that, in order
not to make things worse.
all when accessing data in the active dataset. So, if I would get
that clean, I could still encounter the issue when accessing the same
affected files via a snapshot, but at least the active dataset would
be clean and if the (automatic) snapshots get dropped over time, this
would cure it for the whole pool?

And another question: When booting into the kernel 6.12.25 with ZFS
2.3.2, would it make sense to run "zpool scrub" from there, or could
this make things worse?

And final question for now: Would it make sense to open an issue at
OpenZFS on GitHub, or at least open a discussion topic there? I still
think, this is pretty unexpected for a mere ZFS user to end in a
situation where the system breaks if you upgrade but works "flawlessly"
on the older patch-level version.

Greetings,
Stefan

Hi,

My reply on 5/7 got consistently rejected by debian BTS with reason:

 550 Blacklisted URL in message. (openzfs . github . io) in [black]. See http://lookup.uribl.com.

So resending this message to the list.

They are zfs’s kernel threads, so it might still be related to your
recent I/O.
(Please remove the spacing which is used to avoid being rejected again)
It’s only used for copy objects without going through the vfs level.
E.g. zdb -r pool path/to/file /tmp/file

Correct. This is maybe the best situation we could hope for. But since
zfs literally uses blkptr everywhere, it is possible that the corrupted
data structure can not be easily fixed / recreated by copying files.

I do not see changes to scrub in 2.3.2. So maybe it’s better choice
not to run scrub on a version that would panic on access to your pool.

Sure, definitely it helps. One reason is what you mentioned - some abrupt
changes might shock users. And actually some of the IO hangs you got
do not seem normal either, e.g. blocking of user programs or kthreads.

Thanks,
Shengqi Chen