Error report for a bug with qemu on arm host and xubuntu/debian.
- guest host arm-architecture chroot with qemu and guest x86_64/amd64 not working list of files and apt-get


****************************************************
Setup
****************************************************

Host is cubietruck with arm-architecture with Armbian based on Debian Jessie:
$ uname -a
Linux cubietruck 4.9.12-sunxi #4 SMP Thu Feb 23 19:46:51 CET 2017 armv7l GNU/Linux

Guests, all x86_64/amd64, with debian/xubuntu 17.04, 16.04 and lubuntu 14.04 tested.
# uname -a
setup_frame: not implemented
Linux cubietruck 4.9.12-sunxi #4 SMP Thu Feb 23 19:46:51 CET 2017 x86_64 x86_64 x86_64 GNU/Linux


Setup similar to working i386 environment.
- Files extracted from iso-file included squash-file.
- copied all qemu-files to /usr/bin
- mounted  /proc, /sys, /dev/pts
- copied /run/.../resolvconf for internet access


$ time sudo chroot mmcblk0p6/xubuntu1704amd64/ /usr/bin/qemu-x86_64-static /bin/bash
$ sudo chroot mmcblk0p6/xubuntu1704amd64/ /usr/bin/qemu-x86_64-static /bin/bash


Additional experiments later done with following, too:
dbus-uuidgen > /var/lib/dbus/machine-id
dbus-uuidgen >/etc/machine-id

Intresting difference with mount:
root@cubietruck:/# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)

With mounted squash-file and "aufs" overlay for read/write there is one line more:
none on /  type ....



****************************************************
Errors / Error Messages
****************************************************

root@cubietruck:/# ls
/bin/ls: cannot access ''$'\f': No such file or directory
?

# /bin/dir
setup_frame: not implemented
\002

# dir
setup_frame: not implemented
\002

$ sudo chroot mmcblk0p6/xubuntu1704amd64/ /usr/bin/qemu-x86_64-static /usr/bin/find
.
/usr/bin/find: ‘./\002’: No such file or directory
./?


/# apt-get upgrade
Reading package lists... Done
^Cilding dependency tree... 66%
root@cubietruck:/# apt-get update
setup_frame: not implemented
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
...
Fetched 1,182 kB in 6min 18s (3,123 B/s)
setup_frame: not implemented
setup_frame: not implemented
Reading package lists... Done
N: Ignoring 'r-ubuntu' in directory '/etc/apt/apt.conf.d/' as it is not a regular file
N: Ignoring '
             ' in directory '/etc/apt/sources.list.d/' as it is not a regular file
root@cubietruck:/# apt-get upgrade
Reading package lists... Done


root@cubietruck:/# apt-get upgrade
...
125 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
Need to get 0 B/246 MB of archives.
After this operation, 6,307 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
/bin/rm: cannot remove '/var/lib/dpkg/tmp.ci': Directory not empty
dpkg: error processing archive /var/cache/apt/archives/init-system-helpers_1.29ubuntu4_all.deb (--unpack):
 subprocess rm command for cleanup returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/init-system-helpers_1.29ubuntu4_all.deb
N: Ignoring 'r-ubuntu' in directory '/etc/apt/apt.conf.d/' as it is not a regular file
N: Ignoring '
             ' in directory '/etc/apt/sources.list.d/' as it is not a regular file
N: Ignoring '
             ' in directory '/etc/apt/preferences.d/' as it is not a regular file
E: Sub-process /usr/bin/dpkg returned an error code (1)

# dpkg -l | grep apt
setup_frame: not implemented
setup_frame: not implemented
/bin/grep: memory exhausted

************************************************
Remarks and perhaps similar bugs
************************************************

qemu on arm and chroot i386 debian/xubuntu works.
But chrooting in amd64 debian/xubuntu is not working. Errors are:  setup_frame: not implemented;  /bin/ls not working; /bin/dir "\002" and no listing;  /bin/nano is working; apt-get ends in N: Ignoring 'r-ubuntu' in directory '/etc/apt/apt.conf.d/' as it is not a regular file  and then you are lost.


Perhaps similar bug, but different architecture used, here ppc64:
https://bugs.launchpad.net/qemu/+bug/1591611

*****************************************************************************
An Attempt of a work around - Cascades of qemu
*****************************************************************************
Done for trying to get some more error messages!
*****************************************************************************

#!/bin/bash

# Another test with cascades:
# qemu-arm host --> qemu-i368 guest (as host for next cascade) --> qemu-amd64/x86_64 guest

# Change to start folder.
cd ~/CHROOT

# Create Folders/Directories.
mkdir linux_i386

# First mount file system with all folders.
sudo mount /dev/mmcblk0p5 mmcblk0p5
sudo mount -o bind mmcblk0p5/chroot_xubuntu16p04i386 linux_i386

sudo mount -o bind,ro mmcblk0p5/qemu_arm_host_2.8/ mmcblk0p5/chroot_xubuntu16p04i386/home/qemu_bind_and_link
sudo mount -o bind,remount,ro mmcblk0p5/qemu_arm_host_2.8/ mmcblk0p5/chroot_xubuntu16p04i386/home/qemu_bind_and_link

sudo mount -o bind mmcblk0p5/chroot_xubuntu16p04iamd64  mmcblk0p5/chroot_xubuntu16p04i386/home/dieterd/CHROOT/linux_amd64

sudo mount -o bind,ro mmcblk0p5/qemu_i386_host_2.5/ mmcblk0p5/chroot_xubuntu16p04iamd64/home/qemu_bind_and_link
sudo mount -o bind,remount,ro mmcblk0p5/qemu_i386_host_2.5/ mmcblk0p5/chroot_xubuntu16p04iamd64/home/qemu_bind_and_link

sudo mount -o bind /dev/pts linux_i386/dev/pts
sudo mount -o bind /proc linux_i386/proc
sudo mount -o bind /sys linux_i386/sys

sudo chroot linux_i386 /home/qemu_bind_and_link/qemu-i386-static /bin/bash

sudo mount -o bind /dev/pts linux_amd64/dev/pts
sudo mount -o bind /proc linux_amd64/proc
sudo mount -o bind /sys linux_amd64/sys

chroot linux_amd64  /home/qemu_bind_and_link/qemu-x86_64-static  /bin/bash

# Output was following Error Message:
#warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
#qemu: uncaught target signal 11 (Segmentation fault) - core dumped
#qemu: uncaught target signal 11 (Segmentation fault) - core dumped
#Segmentation fault

# This work around does not work either.

******************************************************************************
Result: Work around failed.
******************************************************************************



A1) Cascading and calling of qemu-i386 for arm host:
# chroot linux_amd64  /usr/bin/qemu-i386-static  /bin/bash
/bin/bash: Invalid ELF image for this architecture

A2) Cascading and calling of qemu-arm for arm host:
# chroot linux_amd64  /usr/bin/qemu-arm-static  /bin/bash
/bin/bash: Invalid ELF image for this architecture


A3) Cascading and calling of qemu-x86_64 for arm host:
# chroot linux_amd64  /usr/bin/qemu-x86_64-static  /bin/bash
setup_frame: not implemented
setup_frame: not implemented
setup_frame: not implemented
setup_frame: not implemented
setup_frame: not implemented
# ls
/bin/ls: cannot access ''$'\f': No such file or directory
?
# dir
\f


B1) Cascading and calling of qemu-x86_64 for i386 host:
# chroot linux_amd64  /home/qemu_bind_and_link/qemu-x86_64-static  /bin/bash
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
'

B2) Cascading and calling of qemu-i386 for i386 host:
# chroot linux_amd64  /home/qemu_bind_and_link/qemu-i386-static  /bin/bash
/bin/bash: Invalid ELF image for this architecture

B3) Cascading and calling of qemu-arm for i386 host:
# chroot linux_amd64  /home/qemu_bind_and_link/qemu-arm-static  /bin/bash
/bin/bash: Invalid ELF image for this architecture



Calling from arm host chroot amd64/x84_64 different qemu for arm or i386 host
C1)
sudo chroot linux_i386/home/dieterd/CHROOT/linux_amd64/ /home/qemu_bind_and_link/qemu-x86_64-static /bin/bash
[sudo] password for dieterd:
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]

qemu: uncaught target signal 11 (Segmentation fault) - core dumped
**
ERROR:/build/qemu-Hq3Lm4/qemu-2.5+dfsg/cpu-exec.c:584:cpu_x86_exec: assertion failed: (cpu == current_cpu)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
'
C2)
$ sudo chroot linux_i386/home/dieterd/CHROOT/linux_amd64/ /home/qemu_bind_and_link/qemu-i386-static /bin/bash
/bin/bash: Invalid ELF image for this architecture

C3)
$ sudo chroot linux_i386/home/dieterd/CHROOT/linux_amd64/ /home/qemu_bind_and_link/qemu-arm-static /bin/bash
/bin/bash: Invalid ELF image for this architecture

D1)
$ sudo chroot linux_i386/home/dieterd/CHROOT/linux_amd64/ /usr/bin/qemu-arm-static /bin/bash
/bin/bash: Invalid ELF image for this architecture

D2)
$ sudo chroot linux_i386/home/dieterd/CHROOT/linux_amd64/ /usr/bin/qemu-i386-static /bin/bash
/bin/bash: Invalid ELF image for this architecture

D3)
$ sudo chroot linux_i386/home/dieterd/CHROOT/linux_amd64/ /usr/bin/qemu-x86_64-static /bin/bash
setup_frame: not implemented
setup_frame: not implemented
setup_frame: not implemented
setup_frame: not implemented
setup_frame: not implemented
root@cubietruck:/# ls
/bin/ls: cannot access ''$'\f': No such file or directory
?
root@cubietruck:/# dir
\f

Some more experiments:

root@cubietruck:/# nano
Error in /etc/nanorc on line 236: Error expanding /usr/share/nano/*.nanorc: Success
Press Enter to continue starting nano.
# But nano works
# File was possible to create
nano /home/dieterd/CHROOT/nano_Versuch
# And was able to save file. When reopened the changed text was ok.

$ thunar
No protocol specified
Thunar: Cannot open display:
$ DISPLAY=:0.0 thunar
No protocol specified
Thunar: Cannot open display:

ping -c 2 www.web.de
ping: prctl: Invalid argument
ifconfig
: error fetching interface information: Device not found
# But apt-get has connection to the internet.
apt-get update
setup_frame: not implemented
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
0% [Connecting to archive.ubuntu.com (91.189.88.162)] [1 InRelease 6,129 B/102 kB 6%]^Csetup_frame: not implemented
# But apt-get hungs because of already shown bug.


But with pressing Tab/--> you jump out of chroot and you have to restart again.



******************************************************************************
Basic of this bug or trouble is, that many funktions of the file system of this
kind of guest (amd64/x64_86) is not working properly.

Bug makes it impossible to work in this chroot environment.
******************************************************************************

Version: 1:7.2+dfsg-1
...
...

This appears to be fixed in qemu for a long time already,
and various arm chroots are working fine.

Closing this bug report with bookworm version of qemu.
Please feel free to reopen it if you think it is incorrect
and the issue is still present.

Thanks,

/mjt