#1040759 installation-reports: "out of memory" message at boot, then unable to update processor microcode, and kernel panic

#1040759#5
Date:
2023-07-10 06:12:12 UTC
From:
To:
Boot method: USB
Image version: Debian 12.0.0-amd64-netist downloaded from July 3rd (i.e. a week old build)
Date: July 9, 2023

Machine: System76 serw10 laptop with an Intel Core i7-7700K CPU (CPU family 6, model 158, stepping 9)
Partitions:
Filesystem      Type     1K-blocks    Used Available Use% Mounted on
udev            devtmpfs  32815780       0  32815780   0% /dev
tmpfs           tmpfs      6567004    1732   6565272   1% /run
/dev/nvme1n1p3  ext4      28660644   10988  27168440   1% /
/dev/nvme1n1p4  ext4     114760836 6898996 101986080   7% /usr
tmpfs           tmpfs     32835012       0  32835012   0% /dev/shm
tmpfs           tmpfs         5120      12      5108   1% /run/lock
/dev/nvme1n1p2  ext4        989032  117276    804172  13% /boot
/dev/nvme1n1p8  ext4      23854928      24  22617812   1% /opt
/dev/nvme1n1p12 ext4       9510080     220   9005184   1% /tmp
/dev/nvme1n1p13 ext4      28660644     120  27179308   1% /usr/local
/dev/nvme1n1p14 ext4      47745772      24  45287944   1% /usr/src
/dev/nvme1n1p5  ext4     256921924   27736 243770604   1% /home
/dev/nvme1n1p7  ext4      81121736  433584  76521396   1% /var
/dev/nvme1n1p1  vfat       1021984    6112   1015872   1% /boot/efi
tmpfs           tmpfs      6567000     144   6566856   1% /run/user/1000

Base System Installation Checklist:
[O] = OK, [E] = Error (please elaborate below), [ ] = didn't try it

Initial boot:           [E]
Detect network card:    [O]
Configure network:      [O]
Detect media:           [O]
Load installer modules: [O]
Clock/timezone setup:   [O]
User/password setup:    [O]
Detect hard drives:     [O]
Partition hard drives:  [O]
Install base system:    [O]
Install tasks:          [O]
Install boot loader:    [O]
Overall install:        [E]

Comments/Problems:

During the first boot, the system displays an "out of memory" message (this
seems to be during the initramfs stage).  Nevertheless after pressing a key
it presents two successive screens of a failure to load microcode message.
After the second message, the boot stops and the power button has to be
pressed.  The OS doesn't reach login.

For some reason, I've have been able to reach login and to successfully login
on the second boot after my initial install and my re-install today.
(This current successful boot is running the 6.1.0-9-amd64 kernel but the
6.1.0-10-amd64 didn't boot minutes before.)

Here is the output from
# dmesg | grep microcode

~~~~~~~~~~~~~~~~~~~~

[    0.046049] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0x52 (or later)
[    0.113466] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.113467] TAA: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.113467] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.113468] SRBDS: Vulnerable: No microcode
[    1.023465] microcode: sig=0x906e9, pf=0x2, revision=0x48
[    1.023543] microcode: Microcode Update Driver: v2.2.

~~~~~~~~~~~~~~~~~~~~~

If I do
# cat /var/log/installer/syslog | grep microcode

then the block of lines above is followed by:

~~~~~~~~~~~~~~~~~~~~~

Jul 10 02:40:59 install-firmware: queuing intel-microcode installation (GenuineIntel)
Jul 10 03:12:25 hw-detect: installing intel-microcode
Jul 10 03:12:26 in-target:   intel-microcode iucode-tool
Jul 10 03:12:26 in-target: Get:2 cdrom://[Debian GNU/Linux 12.0.0 _Bookworm_ - Official amd64 NETINST with firmware 20230610-10:21] bookworm/non-free-firmware amd64 intel-microcode amd64 3.20230512.1 [6,124 kB]
Jul 10 03:12:26 in-target: Selecting previously unselected package intel-microcode.^M
Jul 10 03:12:26 in-target: Preparing to unpack .../intel-microcode_3.20230512.1_amd64.deb ...^M
Jul 10 03:12:26 in-target: Unpacking intel-microcode (3.20230512.1) ...^M
Jul 10 03:12:27 in-target: Setting up intel-microcode (3.20230512.1) ...^M
Jul 10 03:12:27 in-target: intel-microcode: microcode will be updated at next boot

~~~~~~~~~~~~~~~~~~~~~

However, on the (more-frequent) occasions when the kernel panics at boot and
the OS doesn't load the final output to my display has been:

~~~~~~~~~~~~~~~~~~~~~

[    0.047755] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0x52 (or later)
[    0.117586] RETBleed: WARNING: Spectre v2 mitigation leaves CPU vulnerable to
 RETBleed attacks, data leaks possible!
[    1.140611] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    1.140697] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 6.1.0-10-amd64 #1  Debian 6.1.37-1
[    1.140777] Hardware name: System76						Serval WS							/Serval WS
[    1.140902] Call Trace:
[    1.140929]  <TASK>
[    1.140952]  dump_stack_lvl+0x44/0x5c
[    1.140995]  panic+0x118/0x2ed
[    1.141031]  mount_block_root+0x1d3/0x1e6
[    1.141076]  prepare_namespace+0x136/0x165
[    1.141119]  kernel_init_freeable+0x25c/0x286
[    1.141165]  ? rest_init+0xd0/0xd0
[    1.141202]  kernel_init+0x16/0x130
[    1.141239]  ret_from_fork+0x22/0x30
[    1.141281]  </TASK>
[    1.141361] Kernel Offset: 0x33200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xff...
[    1.141465] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---

~~~~~~~~~~~~~~~~~~~~~

I'm concerned that when I try to reboot once again I won't be able to get the
system to boot (receiving a block of text like the last one mentioned).
I've only had a successful boot after installation twice (this current instance is
one of them).

My /lib/firmware/intel-ucode is not empty (the files inside it are timestamped
May 15 at 20:13 ).
From https://linuxfromscratch.org/blfs/view/stable/postlfs/firmware.html , I
think the particular microcode that my processor would need is 06-9e-09
which seems to match /lib/firmware/intel-ucode/06-9e-09.initramfs .

Also,
  $ dpkg -s intel-microcode

shows that that package is installed:

~~~~~~~~~~~~~~~~~~~~~
Package: intel-microcode
Status: install ok installed
Priority: standard
Section: non-free-firmware/admin
# etc.
~~~~~~~~~~~~~~~~~~~~~

I have tried purging and reinstalling intel-microcode via apt-get but that
didn't solve the boot problems.

I have been able to install and boot the base-system of Linux from Scratch and
install and use the current intel-cpu microcode within that distribution's
root.  I'd like to also tune and use Debian on this machine in other partitions
though.

I also updated System76's UEFI firmware recently and seemingly successfully
from within the extant (but old, clunky, and questionably-tuned Ubuntu 20.04
OS available on the system).

In the final analysis, this may be that I haven't figured out how to get
System76's firmware repository loaded on this system via Debian.  It could be
that some other firmware or chipset problem is affecting the boot and
producing the out-of-memory error that shows up as the first bit of bad news.

The laptop came originally installed with Ubuntu 16.04 and I was too much of
a newbie to keep the firmware and other parts of the system tuned over the few
years I've had this laptop.