Dear Maintainer,

Live migration of a guest with more than one CPU thread from a bookworm
host to a trixie host fails when the guest is started on the destination:

operation failed: guest CPU doesn't match specification: extra features: ht

This blocks migrating running guests from bookworm to trixie, affecting the
normal upgrade path between stable releases.

Versions

source:      bookworm, libvirt 9.0.0-4+deb12u2, QEMU
1:7.2+dfsg-7+deb12u18+b3
destination: trixie,   libvirt 11.3.0-3+deb13u3, QEMU 1:10.0.13+ds-0+deb13u1

Same-release migration is not affected.

How to reproduce

1.  On the bookworm host, run a guest with a custom/host-model CPU and
multiple threads, e.g.:
    <cpu mode='host-model'/> with <topology sockets='1' cores='1'
threads='2'/>
    (2 vCPUs; the guest CPU model in my test was Skylake-Server-IBRS).
2.  Live-migrate it to the trixie host using the libvirt API
(virDomainMigrateToURI3 with VIR_MIGRATE_LIVE | VIR_MIGRATE_PEER2PEER |
VIR_MIGRATE_PERSIST_DEST) or virsh migrate --live --p2p.
3.  The migration is rejected on the destination with the error above.

What is going on

On the source, a running guest has two copies of its CPU definition. The
live copy has check='full', while the migratable XML (virsh dumpxml
--migratable) and persistent config have check='partial':

virsh dumpxml <vm>               ->  <cpu mode='custom' match='exact'
check='full'>
virsh dumpxml --migratable <vm>  ->  <cpu mode='custom' match='exact'
check='partial'>

Neither lists an "ht" feature. During migration, the source sends its live
CPU (check='full') to the destination in the migration cookie, and the
destination replaces the guest CPU with it.

Destination debug log:

qemuProcessInit:5896 : ... migration=1
qemuDomainUpdateCPU:9446 : Replacing CPU definition
(cookie XML contains <cpu mode='custom' match='exact' check='full'>)

After QEMU starts, libvirt queries which CPU features are enabled. QEMU 10
answers "ht" = true:

qemuMonitorSend :
{"execute":"qom-get","arguments":{"path":"/machine/unattached/device[0]","property":"ht"},...}
reply: {"return": true, ...}
virCPUx86UpdateLive:3221 : Feature 'ht' enabled by the hypervisor
virCPUx86UpdateLive:3257 : operation failed: guest CPU doesn't match
specification: extra features: ht

"ht" is not in the guest CPU definition, so with check='full', libvirt
reports it as an extra feature. QEMU 10.0 does this since commit
c6bd2dd634208 ("i386/cpu: Set up CPUID_HT in x86_cpu_expand_features()
instead of cpu_x86_cpuid()"): it now reports "ht" as on whenever the
topology has more than one thread per core, whereas QEMU < 10 reported the
value given on the command line.

Changing the check attribute in the XML passed to the migration call does
not help; libvirt rejects a different value ("Target CPU check none does
not match source partial"), and the destination uses the CPU from the
cookie regardless.

Upstream

This is fixed upstream by two commits by Jiri Denemark (released in libvirt
11.10.0):

4d5c1bc241  qemu_monitor: Filter CPU features reported by QEMU
ba16113c76  qemu: Ignore "ht" CPU feature

Both commits are required. The second adds "ht" to ignoredFeatures[], and
the first ensures libvirt applies that list to the features QEMU reports
for a running guest.

Related reports:
https://gitlab.com/libvirt/libvirt/-/issues/821
https://issues.redhat.com/browse/RHEL-104216
https://gitlab.com/qemu-project/qemu/-/issues/3153

Proposed fix for trixie

Cherry-pick both commits for the next trixie stable update:

  - ba16113c applies cleanly to 11.3.0.
  - 4d5c1bc241 applies to 11.3.0 with a refresh.

Testing: With both patches applied to 11.3.0-3+deb13u3 on the destination,
the migration completes successfully.

The patches are attached
[qemu_monitor-Filter-CPU-features-reported-by-QEMU.patch,
qemu-Ignore-ht-CPU-feature.patch].

Please consider adding these patches to Debian Trixie to fix migration
functionality when upgrading from Debian 12 to 13.

Thanks,
Vasudev Kamath