Dear Maintainer, Since the upgrade to 6.12.113-1, one of our KVM guests froze twice under page-cache reclaim. A second guest running 6.12.111-1 froze with similar symptoms during the same kind of workload, but that oops could not be captured; 6.12.111-1 is marked found because it contains the same upstream change (see below), not because of a captured trace. kswapd oopses with a NULL pointer dereference in list_lru_add() and dies while holding the mapping's xa_lock, so every subsequent reader of that file spins forever in __pv_queued_spin_lock_slowpath/kvm_wait and the guest becomes unresponsive. Oops (captured from guest memory, kernel otherwise untainted): BUG: kernel NULL pointer dereference, address: 0000000000000008 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI CPU: 1 UID: 0 PID: 67 Comm: kswapd0 Not tainted 6.12.113+deb13-amd64 #1 Debian 6.12.113-1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 RIP: 0010:list_lru_add+0x83/0x110 Call Trace: list_lru_add_obj+0x7c/0xa0 workingset_update_node+0x8a/0xb0 xas_store+0x24e/0x670 __filemap_remove_folio+0x116/0x1e0 __remove_mapping+0xdf/0x220 shrink_folio_list+0xb62/0xd00 evict_folios+0x24f/0x650 try_to_shrink_lruvec+0x21b/0x2b0 shrink_one+0x102/0x1f0 shrink_node+0xa08/0xc40 balance_pgdat+0x520/0x9a0 kswapd+0x1f1/0x380 kthread+0xcf/0x100 ret_from_fork+0x31/0x50 ret_from_fork_asm+0x1a/0x30 note: kswapd0[67] exited with irqs disabled note: kswapd0[67] exited with preempt_count 3 Workload: XFS root/data filesystem, many short-lived SSH sessions (each in its own systemd session scope / memcg) doing file deployments. No global memory pressure (about 60% of RAM available, swap unused). The freeze happened twice within 30 hours on 6.12.113-1. We never saw it on this guest with 6.12.107-1. Probable cause: stable 6.12.109 picked up ad812805fea7 "xarray: honor XA_FLAGS_ACCOUNT in xas_split_alloc()" (upstream 789763523fb4) which makes xa_nodes allocated during folio split memcg-charged. The upstream dependency that installs the shadow_nodes list_lru on the xa_state during split, aca1f2d5de17 "mm/huge_memory: initialise workingset state before folio split" does not appear to be in 6.12.y (checked ChangeLog-6.12.109 to 6.12.113). Without it, a memcg-charged xa_node can reach workingset_update_node() for a memcg whose shadow_nodes list_lru was never allocated, hence the NULL+0x8 dereference (list_add_tail on l->list.prev). The same crash and analysis were reported on Amazon Linux 6.12.110: https://github.com/awslabs/amazon-eks-ami/issues/2845 Amazon shipped a fixed kernel (6.12.111-136.204) on 2026-10-12. Note: the security tracker marks CVE-2026-80729 (the aca1f2d5de17 fix) as "vulnerable code not present" for trixie. That is true for xas_try_split(), but the 6.12.109 backport reintroduces the same defect via xas_split_alloc(), so trixie is effectively affected. Since an unprivileged process filling page cache in its own memcg seems able to trigger it, this is also a local DoS on shared hosts. Workaround confirmed in place: booting with cgroup.memory=nokmem. Suggested fix: backport aca1f2d5de17 to 6.12.y (and 6.18.y, where the stable backport also failed to apply), or revert ad812805fea7 in the Debian package until stable catches up. We are reporting the regression to stable@vger.kernel.org / regressions@lists.linux.dev as well and will link the thread here.
Two corrections to my initial report.
1. Suggested fix: aca1f2d5de17 does not apply as is to 6.12.y. Its
Fixes: target is the non-uniform __folio_split() (58729c04cf10, 6.15+).
In 6.12.y the page-cache split goes through
split_huge_page_to_list_to_order(), which calls xas_split_alloc()
without mapping_set_update() (mm/huge_memory.c in v6.12.113).
So the options for 6.12.y are rather:
- revert ad812805fea7 ("xarray: honor XA_FLAGS_ACCOUNT in
xas_split_alloc()"), or
- an adapted fix: mapping_set_update(&xas, mapping) before
xas_split_alloc() in split_huge_page_to_list_to_order(). Note that
in 6.12.y mapping_set_update() is still a static function in
mm/filemap.c (it only moved to mm/internal.h in 6.13), so this
also needs that move or an open-coded equivalent.
Neither has been tested on our side.
2. Upstream report: I wrote that we would report this to
stable@vger.kernel.org. We will not contact upstream ourselves; we
leave it to you to forward it if you confirm it is not Debian-specific,
or to ask us to do it (a draft for stable@ / regressions@ is ready).
We have an affected guest available to test a patched kernel (booted
without cgroup.memory=nokmem).
Hi Thomas, Please test with a pristine build of 6.12.113 (i.e. withouth Debian patches on top). If you can reproduce the issue, please forward it upstream (and report here where it has been reported). In case you can quickly trigger the behaviour, it would be helpful for the upstream report if you bisect the changes and confirm your suspected breaking change. Thank you already, Regards, Salvatore
Hi, guest running Debian's 6.12.113 (kswapd -> ... -> workingset_update_node -> list_lru_add+0x83, NULL+0x8), Debian bug https://bugs.debian.org/1151281. Amazon reported it on 6.12.110. In case it helps with the 6.12 backport Greg and Sasha asked for, we have a reproducer and ran it on one test guest (8 vCPU, 8 GB, XFS on a loop device, MGLRU, THP=always), pristine 6.12.113 from kernel.org built with a reduced Debian config: kernel result with background load Debian 6.12.113-1 crashed after ~13, ~2*, ~24 min pristine 6.12.113 crashed after ~14 min pristine 6.12.113, ad812805fea7 no crash in 90 min reverted pristine 6.12.113 + 62e72d2cf702 + no crash in 120 min aca1f2d5de17 adapted (below) * that run started just after a kernel build had finished Without the background load the reproducer alone did not crash (30 and 60 min runs). During the 120 min run on the backport the path was exercised heavily (~1.09M workingset_nodereclaim, ~37k thp_split_page) and the kernel log stayed clean (no BUG/WARNING). The backport we tested follows Sasha's description: 62e72d2cf702 ("mm, madvise: fix potential workingset node list_lru leaks") applies cleanly to 6.12.113, and the relevant part of aca1f2d5de17 becomes a single line in split_huge_page_to_list_to_order(): --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3529,6 +3529,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, goto out; } + mapping_set_update(&xas, mapping); xas_split_alloc(&xas, folio, folio_order(folio), gfp); if (xas_error(&xas)) { ret = xas_error(&xas); This is test feedback, not a formal submission: we are not in a position to carry these patches through review ourselves. If someone sends the 6.12.y backports, we are glad to run them through the same reproducer and reply with a Tested-by. The reproducer, roughly: on an XFS filesystem, a background loop that untars the kernel source and runs "make -j$(nproc) vmlinux" in its own cgroup, plus 4 workers each looping over (1) in a MemoryMax=128M scope, rewrite a 256 MiB file in 2 MiB writes and evict it by reading a 1 GiB file (cache dropped first), then (2) in a fresh MemoryMax=96M scope, read the 256 MiB file back with read_ahead_kb=16384, punch an 8 KiB hole in every 2 MiB block and read the 1 GiB file again to force memcg reclaim. The scripts are available on request. Thanks, Thomas Mischler
Hi Salvatore, It reproduces on a pristine 6.12.113 from kernel.org, and it turns out it was already reported upstream on 2 October in a stable@ thread. Our test results (pristine kernel, revert of ad812805fea7, and the 6.12 backport of aca1f2d5de17 + 62e72d2cf702 that the stable maintainers asked for) are in our reply there, also copied to this bug: https://lore.kernel.org/stable/PATP264MB48180D3DFF94FB287D09CE64E0662@PATP264MB4818.FRAP264.PROD.OUTLOOK.COM/ aca1f2d5de17 is in 6.18.56; 6.12.y, 6.6.y and 6.1.y are waiting for tested backports. Bookworm is likely affected too: 6.1.188+ carries the same xarray change without the fix (code reading, not tested). Thomas Mischler
Hi Thomas, Thank you for that. Regards, Salvatore
Thanks for the testing. I queued the backports for 6.12, 6.6 and 6.1.