Hi,

I’m deploying cloud images on a Libvirt/QEMU/KVM hypervisor using Ansible. After trixie hit stable, I replaced the bookworm image with a trixie one, but I am unable to get it to boot. The only change in the deployment process was switching from debian-12-generic-amd64.qcow2 to debian-13-generic-amd64.qcow2.
The process seems to hang during boot, and no errors are printed anywhere. I’m able to connect to the VM using virsh console, but I only see the first line "Booting Debian GNU/Linux" and nothing further happens.

QEMU log:

###2025-08-15 23:42:55.849+0000: starting up libvirt version: 11.5.0, qemu version: 10.0.2, kernel: 6.12.37-1-lts, hostname: ...LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin \
USER=root \
HOME=/var/lib/libvirt/qemu/domain-1-03 \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-03/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-03/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-03/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=03,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-03/master-key.aes"}' \
-machine pc-i440fx-10.0,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=on \
-accel kvm \
-cpu qemu64 \
-m size=4194304k \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":4294967296}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid 23ac7109-7d8d-4114-baf4-c23ca2df9869 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc \
-no-shutdown \
-global PIIX4_PM.disable_s3=1 \
-global PIIX4_PM.disable_s4=1 \
-boot strict=on \
-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \
-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x3"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x4"}' \
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/03/root.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x5","drive":"libvirt-2-format","id":"virtio-disk0","bootindex":1}' \
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/cidata.iso","node-name":"libvirt-1-storage","read-only":true}' \
-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-1-storage","id":"sata0-0-1"}' \
-netdev '{"type":"tap","fd":"34","vhost":true,"vhostfd":"36","id":"hostnet0"}' \
-device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:52:00:c6:2c:f6","bus":"pci.0","addr":"0x2"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=31,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x6"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
###
Please let me know if you need any additional information.

Cheers

Control: tags -1 + moreinfo

Can you share more about your ansible and/or libvirt configuration that
you're using to launch this VM?  I have no trouble launching VMs from
our trixie images, but you've got a whole bunch of extra configuration
passed by libvirt that I'm not using.  I suspect the issue is in there
somewhere.

noah

The options you see in the log are mostly defaults, the config I use is prettybare-bones. I copy the qcow2 image from the cache to the designated folder,
resize it, and use the Ansible Jinja2 template below to define the VM:

# --- #
<domain type='kvm'>

  <name>{{ vm_num }}</name>
  <title>{{ vm_fqdn }}</title>
  <os>
    <type>hvm</type>
    <boot dev='hd'/>
  </os>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <features>
    <acpi/>
  </features>

  <!-- cpu -->
  <vcpu>{{ vm_cpu }}</vcpu>

  <!-- ram -->
  <memory unit='GiB'>{{ vm_ram }}</memory>
  <currentMemory unit='GiB'>{{ vm_ram }}</currentMemory>

  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <!-- image -->>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='{{ vm_dir }}/{{ vm_num }}/{{ vm_img }}'/>
      <target dev='vda' bus='virtio'/>
    </disk>

    <!-- cloud-init -->>
    <disk type="file" device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='{{ vm_cloudinit_data }}'/>
      <target dev='vdb' bus='sata'/>
      <readonly/>
    </disk>

    <!-- network -->>
    <interface type='bridge'>
      <mac address='{{ vm_mac }}'/>
      <source bridge='br0'/>
      <target dev='vmif{{ vm_num }}'/>
      <model type='virtio'/>
    </interface>

    <!-- console -->>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>

    <!-- guest-agent -->>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
    </channel>

  </devices>
</domain>
# --- #

Once the VM is defined, I enable autostart, start the VM, and wait until it’s
reachable.


18 Aug 2025, 00:14 by noahm@debian.org:

Control: tags -1 - moreinfo
Control: reassign -1 src:grub2

I believe I've been able to reproduce the problem.  I'm not sure if the
issue is with qemu or with grub2, but we'll start with the latter.

I've created a script based on the qemu invocation from your log (see
attached) that reproduces this failure with our "nocloud" images and
doesn't rely on all the file descriptors that your ansible stuff sets
up.  When we run the script, we're shown a grub boot menu, but attempts
to boot a kernel fail and drop back to the menu.

If we then modify grub.cfg in the VM image to replace "terminal_output
gfxterm serial" with "terminal_output serial", things work as expected.

Use something like the following to edit the image's grub config.
sudo kpartx -av rootfs.raw
sudo mount /dev/mapper/loop0p1 /mnt
sudo vim /mnt/boot/grub/grub.cfg
sudo umount /mnt
sudo kpartx -dv rootfs.raw

The other thing I've observed is that if we remove most of the device
specification from your qemu invocation and stick with its default
device model, then the images boot as configured.  So it's posssible
that there's either a bug with qemu's emulation of the specific serial
devices you've configured or with grub's support of that hardware.
Whatever serial port qemu sets up when run without -nodefaults does not
trigger this failure.

noah

Thanks for looking into it! This helped me narrow down the problem and fix it
on my end.

It seems the issue is caused by the serial console. Changing the type from
"serial" to "virtio" fixes it (https://libvirt.org/formatdomain.html#console):

     <!-- console -->>
     <console type='pty'>
-      <target type='serial' port='0'/>
+      <target type='virtio' port='0'/>
     </console>

It's still a bit strange though, since serial console worked fine with all
previous cloud images.


Aug 20, 2025, 16:10 by noahm@debian.org: