Using virsh I tried to migrate a Squeeze guest live from
my desktop PC to another host (both running Testing and
kernel 2.6.34). Command line:

virsh migrate --live migrationtest qemu+ssh://fw02/system

There was no error message, but on the new guest I saw a
lot of filesystem errors (see attachment). After a reboot
the guest seems OK, but I don't trust it.


The storage pool ("/storage") is on NFSv4. /proc/mounts:

rootfs / rootfs rw 0 0
none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
none /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,relatime,size=10240k,mode=755 0 0
none /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0
/dev/disk/by-uuid/59ec3158-5555-4c40-987e-e7f7cf3f9f77 / reiserfs rw,relatime,notail 0 0
tmpfs /lib/init/rw tmpfs rw,nosuid,relatime,mode=755 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sdc3 /export/storage reiserfs rw,noatime 0 0
rpc_pipefs /var/lib/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfs-storage:/storage/ /storage nfs4 rw,relatime,vers=4,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.129,addr=192.168.1.213 0 0


Is it possible that some kind of sync operation is missing
before the guest disk is opened on the new host?


Regards

Harri

Hi Harald,
This is all kvm/qemu's business AFAIK. Could you attach the qemu log? Did
you try --live?

Could you also try to reproduce this with kvm/qemu (without libvirt) and
see if the problem persists?

Cheers,
 -- Guido

Migration on the command line never worked for me. I can use
kvm to start the guest, e.g.

/usr/bin/kvm \
         -enable-kvm -m 1024 -name migtest \
         -nodefaults \
         -boot c \
         -drive file=/storage/migtest/vda.img,if=none,id=drive-virtio-disk0,boot=on \
         -vga cirrus

but if I press ctrl-alt-2 as documented on http://www.linux-kvm.org/page/HOWTO
to switch to the monitor, then nothing happens.

Actually I am using libvirt to avoid this kind of problems.



Regards

Harri

Then lets turn this around: what happens without it?
It's not in /tmp see:

http://wiki.debian.org/libvirt#Bugreports.2BAC8-Debugging
Drop the -nodefaults  - then it should work.
True but to track this down we need to find out if it's kvm of libvirt
(I suspect the former).

Cheers,
 -- Guido

??? The whole bug report is about a live migration.

It seems that there is no problem using kvm. Here is the
procedure I tried:

on my desktop pc:

         /usr/bin/kvm \
         -enable-kvm -m 1024 -name migtest -vnc :14 \
         -boot c \
         -drive file=/storage/migtest/vda.img,id=drive-virtio-disk0,boot=on

(/storage is a NFS partition mounted on all virtualization hosts.)

on the new host:

         /usr/bin/kvm \
         -enable-kvm -m 1024 -name migtest -vnc :14 \
         -boot c \
         -drive file=/storage/migtest/vda.img,id=drive-virtio-disk0,boot=on \
         -incoming tcp:0:4444

When the guest completed booting I entered the qemu monitor
for this guest to run

         migrate -d tcp:fw02:4444

After that the kvm session on my desktop went away. I could
connect via vnc to the new host. There were no problems on
the guest.


After stopping all kvm sessions I tried the same procedure
using virsh and the same source and destination hosts:  The
file system problems were back :-(.

Do you think you could try to reproduce this.

Regards

Harri

Regards

Harri

Hi Harald,
Seems so. I've checked two of for combinations so far:

               | --live | no --live |
---------------------------------------
NFS-C -> NFS-S |   ?    |  works    |
NFS-S -> NFS-C | fails  |    ?      |

(NFS-S: NFS-Server, NFS-C: NFS-Client. The NFS server accesses the
images as a local fs (ext3)). The ?-cases are unchecked.

The image ends up as root:root instead of libvirt-qemu:kvm which could
be the problem. Next thing would be to check if setting

	user = root
	group = root

in /etc/libvirt/qemu.conf changes anything. I won't be able to work on
this during the next days so if anybody fills in the '?' that'd be
great.
Cheers,
 -- Guido

Hi Guido,

Sorry, my NFS server is not permitted to run kvm and libvirt.
I got the problem for migrating live from one NFS client
to another.

If I omit the "--live"

	virsh migrate migtest qemu+ssh://fw02/system

then I get these I/O errors for the guest on the new host,
too.

I got the "root:root", too.

If I set user and group to "root" on both NFS clients,
then the migration seems to work.

Actually NFSv4 is supposed to hide these details, but
libvirt-qemu:kvm is 200:200 on all my hosts.


Hope this helps. Regards

Harri

Regards

Harri

Hi Harald,
On Mon, Jul 05, 2010 at 02:03:57PM +0200, Harald Dunkel wrote:
[..snip..]
Thanks for testing! So we at least have a workaround. And it indeed
seems to be related to the fact, that the client can't access the image.
I'll recheck with 0.8.2 and if it still fails there I'll try to find the
time to dig deeper into this.
 -- Guido