#990464 Kdump is unable to use NVME

Package:
kdump-tools
Source:
kdump-tools
Description:
scripts and tools for automating kdump (Linux crash dumps)
Submitter:
Kody Barks
Date:
2023-04-12 03:00:02 UTC
Severity:
normal
Tags:
#990464#5
Date:
2021-06-29 23:25:48 UTC
From:
To:
When the kernel panics, it loads the kdump kernel correctly but this kernel
is unable to use NVME, making it unable to mount the root filesystem and
thus making the kdump kernel panic.
I am unable to generate a proper log of this error. The best I could do is
to take a picture of my screen when this happens. This is the attached
file.
I have tried using the boot parameter: "nvme_core.default_ps_max_latency_us=0"
but this made no changes in behavior.
I am using Debian 10 Buster with Linux 5.10.40 with libc6 2.28-10
My motherboard is the ASUS PRIME x399-a .
The SSD I'm trying to use as the root filesystem is the Intel Corporation
Optane SSD 900P Series [8086:2700].
It should be noted that outside of kdump, during normal operation, the
device works flawlessly. Only Kdump's kernel has an issue.

[image: IMG_20210321_122608.jpg]
#990464#10
Date:
2021-06-30 18:02:13 UTC
From:
To:
tag 990464 + moreinfo
thanks

hey Kody,

  It's likely that your crashkernel= parameter just doesn't provide
enough memory for the crashkernel to initialize all of your devices.
This is set in /etc/default/grub.d/kdump-tools.cfg. Try bumping that
up to see if it fixes things. Note that after changing it you'll need
to 1) sudo update-grub and 2) reboot before it will take effect for
the next crash dump. It's unfortunately not possible to provide
a static default crashkernel= that works for everyone, so tuning it is
often required.

 -dann
#990464#15
Date:
2021-06-30 18:02:13 UTC
From:
To:
tag 990464 + moreinfo
thanks

hey Kody,

  It's likely that your crashkernel= parameter just doesn't provide
enough memory for the crashkernel to initialize all of your devices.
This is set in /etc/default/grub.d/kdump-tools.cfg. Try bumping that
up to see if it fixes things. Note that after changing it you'll need
to 1) sudo update-grub and 2) reboot before it will take effect for
the next crash dump. It's unfortunately not possible to provide
a static default crashkernel= that works for everyone, so tuning it is
often required.

 -dann
#990464#20
Date:
2021-07-01 19:18:12 UTC
From:
To:
I tried increasing my crashkernel parameter to 8096M, and the problem still
persists.
However, I used that test as an opportunity to get more pictures of this
problem.
These images are attached.
#990464#25
Date:
2021-07-14 10:02:41 UTC
From:
To:
Hello Kody,
I had in the past also a working crash kernel setup.
But that started to fail with and after kernel 5.5.
Unfortunately I did not yet come to report this to debian.

If that is also the case for you, a workaround could be to
install the old 5.4.19 bpo kernel [1].

And modify /etc/default/kdump-tools like this:
-KDUMP_KERNEL=/var/lib/kdump/vmlinuz
-KDUMP_INITRD=/var/lib/kdump/initrd.img
+KDUMP_KERNEL=/boot/vmlinuz-5.4.0-4-amd64
+KDUMP_INITRD=/var/lib/kdump/initrd.img-5.4.0-4-amd64

This is a workaround for me with a "Asus-PRIME-B350M-A",
but for a plain SATA SSD, and running testing.


I just looked in upstream tracker and found this bug [2],
but that might show another issue.

Kind regards,
Bernhard

[1] https://snapshot.debian.org/package/linux/5.4.19-1%7Ebpo10%2B1/
[2] https://bugzilla.kernel.org/show_bug.cgi?id=209351
#990464#30
Date:
2021-07-14 16:55:00 UTC
From:
To:
reverting the kdump kernel to 5.4 did not make any difference.
#990464#35
Date:
2021-12-27 15:44:24 UTC
From:
To:
Kernel 5.10, and an upgrade to Bullseye. This issue is still not fixed.
#990464#40
Date:
2022-02-16 16:08:58 UTC
From:
To:
Hi Kody, can you please share the outputs of the following 3 commands?

lspci -nns 0000:09:00.0
lspci -nns 0000:41:00.0
lspci -nns 0000:42:00.0

Also, do you remember of a specific kernel version that ever worked with
these drivers on kdump?
Thanks,


Guilherme


P.S. Please don't forget to CC me when responding, I'm not sure if I'd
receive responses to the bug only, I'm not subscribed it seems.
#990464#45
Date:
2023-04-12 02:56:56 UTC
From:
To:
On Wed, 16 Feb 2022 13:08:58 -0300 "Guilherme G. Piccoli" < gpiccoli@igalia.com> wrote:
here ya go:
root@TESSA:~# lspci -nns 0000:09:00.0
09:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd
NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
root@TESSA:~# lspci -nns 0000:41:00.0
41:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd
NVMe SSD Controller SM961/PM961/SM963 [144d:a804]
root@TESSA:~# lspci -nns 0000:42:00.0
42:00.0 Non-Volatile memory controller [0108]: Intel Corporation Optane SSD
900P Series [8086:2700]

no, this has never worked. my system crashes frequently, and without kdump,
i have no means of figuring out why.