Subject: linux-image-6.12.101+deb13-amd64: [REGRESSION] 6.12.101 breaks AMD SEV, SEV-ES, and SEV-SNP platform initialization
Package: src:linux
X-Debbugs-Cc: wizlonex@gmail.com
Version: 6.12.101-1
Severity: important

Hi Debian Kernel Team,

Reporting a regression introduced in linux-image-6.12.101+deb13-amd64 that breaks
AMD SEV/SEV-ES and SEV-SNP guest launching across different AMD EPYC platforms.

Disclaimer: I am a homelab enthusiast, not a kernel developer, so I may have limited
ability to answer deep technical debugging questions. However, I have tested and
confirmed this regression reproducibly across two different systems with concrete logs.
--- Platforms Tested ---
- System 1 (AMD EPYC Rome platform - Supermicro H11DSi-NT): Running SEV-ES guests.
- System 2 (AMD EPYC Milan platform): Running SEV-SNP guests.

--- Observed Failure on 6.12.101+deb13-amd64 ---
Host kernel command line:
  amd_iommu=on mem_encrypt=on kvm_amd.sev=1 kvm_amd.sev_es=1

1. Diagnostic probe via Virtee 'snphost ok' tool indicates uninitialized platform:
   [ PASS ] - Secure Encrypted Virtualization (SEV)
   [ FAIL ]   - SEV-ES initialized
   [ FAIL ] - SEV initialized: Uninitialized

2. All SEV-ES guests fail to start with QEMU errors:
   libvirtd[12953]: internal error: QEMU unexpectedly closed the monitor (vm='solo-pub-ws'):
   qemu-system-x86_64: -accel kvm: sev_common_kvm_init: guest policy requires SEV-ES, but host SEV-ES support unavailable
   qemu-system-x86_64: -accel kvm: failed to initialize kvm: Operation not permitted

3. Kernel dmesg comparison indicates CCP/SEV platform init is delayed/missing at boot:
   On 6.12.101, 'ccp ... SEV API:0.24 build:22' does not fire during boot init,
   leaving the platform uninitialized when libvirt attempts guest autostart.

--- Working Baseline on 6.12.95+deb13-amd64 ---
Rebooting back into kernel 6.12.95+deb13-amd64 immediately resolves the issue
across both platforms without any guest XML or QEMU changes:

1. 'snphost ok' reports ready:
   [ PASS ] - Secure Encrypted Virtualization (SEV)
   [ PASS ]   - SEV-ES initialized
   [ PASS ] - SEV initialized: Initialized, currently running a guest

2. All SEV-ES guests (solo-pvt-ws, solo-pub-ws, solo-gateway) autostart and run cleanly.

Hi,

As you can reliably confirm the regression, please bisect the issue
between 6.12.95 and 6.12.101 to identify which commit(s) break. The
procedure is as follows:

    git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
    cd linux-stable
    git checkout v6.12.95
    cp /boot/config-$(uname -r) .config
    yes '' | make localmodconfig
    make savedefconfig
    mv defconfig arch/x86/configs/my_defconfig

    # test 6.12.95 to ensure this is "good"
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm problem does not exist

    # test 6.12.101 to ensure this is "bad"
    git checkout v6.12.101
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm problem exists

With that confirmed, the bisection can start:

    git bisect start
    git bisect good v6.12.95
    git bisect bad v6.12.101

In each bisection step git checks out a state between the oldest
known-bad and the newest known-good commit. In each step test using:

    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install, verify if problem exists

and if the problem is hit run:

    git bisect bad

and if the problem doesn't trigger run:

    git bisect good

. Please pay attention to always select the just built kernel for
booting, it won't always be the default kernel picked up by grub.

Iterate until git announces to have identified the first bad commit.

Then provide the output of

    git bisect log

In the course of the bisection you might have to uninstall previous
kernels again to not exhaust the disk space in /boot. Also in the end
uninstall all self-built kernels again.

Regards,
Salvatore

Hi Debian Kernel Team,

With the help of Gemini to help me interpret my results, I have completed a
bisect on the upstream 6.12.y branch to pinpoint the exact commit causing
this SEV-ES launch failure regression.  My journey wasn't as clean and
surgical as I would have liked (so it took longer), but I THINK I got
there.

The failure was introduced by:
Commit: bb1c84647025 ("crypto: ccp - Move SEV/SNP Platform initialization
to KVM")
Upstream commit: 3f8f0133a5fc9b32d0c308530320c3f2430ba5ab

---
### Verification & Bisect Details:
* LAST WORKING COMMIT: 8a599f4f74d4
* FIRST BROKEN COMMIT: bb1c84647025 (which landed in 6.12.97+)
* Verified on 6.12.101: Cleanly reverting commit bb1c84647025 on top of
6.12.101 immediately fixes the issue and allows SEV-ES VMs to launch and
run without errors.

---
### Failure Mechanism:
Commit bb1c84647025 removed `sev_platform_init(&args)` from
`sev_pci_init()` in `drivers/crypto/ccp/sev-dev.c` under the assumption
that KVM would handle initialization on-demand.

In practice on bare metal, the PSP firmware is left uninitialized (never
transitions to SEV_STATE_INIT at probe time). When QEMU initializes the KVM
accelerator and probes host SEV-ES capabilities, the ioctl fails with
-EPERM, producing the error:

  qemu-system-x86_64: -accel kvm: sev_common_kvm_init: guest policy
requires SEV-ES, but host SEV-ES support unavailable
  qemu-system-x86_64: -accel kvm: failed to initialize kvm: Operation not
permitted

Reverting bb1c84647025 restores the probe-time initialization and resolves
the regression.

Please let me know if you need any additional hardware traces or logs.   I
have the time and resources to run tests if I can help - I have Milan, Rome
and even Naples (nothing newer) servers and I can try to do some trials if
it's helpful.

THANK YOU for your wonderful work.  I have never rebooted a supermicro
server as much in my life as I have on this.  LOL.  I HOPE this is useful.

Very Respectfully,

Andrew Wilson