#877710 /usr/sbin/sbuild-createchroot: fails to create stretch arm64 schroot on unstable

#877710#5
Date:
2017-10-04 17:27:43 UTC
From:
To:
$ sudo sbuild-createchroot --arch=arm64 --foreign stretch /srv/chroot/arm64 http://mirror.bytemark.co.uk/debian

$ sudo cp /usr/bin/qemu-aarch64-static /srv/chroot/arm64/usr/bin/
$ sudo chroot /srv/chroot/arm64/
I have no name!@sylvester:/# /debootstrap/debootstrap --second-stage
....
I: Configuring e2fsprogs...
I: Configuring libblkid1:arm64...
I: Configuring sysvinit-utils...
I: Configuring libmount1:arm64...
I: Configuring util-linux...
I: Configuring libc-bin...
W: Failure while configuring required packages.
W: See //debootstrap/debootstrap.log for details (possibly the package passwd is at fault)
I have no name!@sylvester:/# exit


From the debootstrap.log:

Setting up gzip (1.6-5+b1) ...
Setting up bsdutils (1:2.29.2-1) ...
Setting up dash (0.5.8-2.4) ...
Setting up init-system-helpers (1.48) ...
Setting up libpam0g:arm64 (1.1.8-3.6) ...
Setting up libpam-modules-bin (1.1.8-3.6) ...
Setting up bash (4.4-5) ...
update-alternatives: using /usr/share/man/man7/bash-builtins.7.gz to provide /usr/share/man/man7/builtins.7.gz (builtins.7.gz) in auto mode
Setting up libpam-modules:arm64 (1.1.8-3.6) ...
Setting up libpam-runtime (1.1.8-3.6) ...
Setting up passwd (1:4.4-4.1) ...
duplicate group entry
delete line 'sbuild:x:121:neil'? No
group sbuild: no user neil
delete member 'neil'? No
duplicate group entry
delete line 'sbuild:x:121:neil'? No
group sbuild: no user neil
delete member 'neil'? No
grpck: no changes
Multiple entries named 'sbuild' in /etc/group. Please fix this with pwck or grpck.
grpconv: failed to prepare the new /etc/group entry 'sbuild'
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
/bin/chown: cannot access '/etc/gshadow': No such file or directory
/bin/chmod: cannot access '/etc/gshadow': No such file or directory
Please correct the error and rerun `/sbin/shadowconfig on'
dpkg: error processing package passwd (--configure):
 subprocess installed post-installation script returned error exit status 1
Setting up login (1:4.4-4.1) ...
dpkg: libuuid1:arm64: dependency problems, but configuring anyway as you requested:
 libuuid1:arm64 depends on passwd; however:
  Package passwd is not configured yet.

The line:
Multiple entries named 'sbuild' in /etc/group. Please fix this with pwck or grpck.
would seem to indicate a problem within sbuild rather than debootstrap.

#877710#10
Date:
2017-10-04 19:04:05 UTC
From:
To:
Hi Neil,

Quoting Neil Williams (2017-10-04 19:27:43)

This segfault also looks fishy...

Your conclusion might be correct. sbuild-createchroot fills /etc/group by
running "getent group sbuild" on the parent system. The following output would
be interesting to debug this further:

 - what you get when you run "getent group sbuild" yourself

 - the actual content of /etc/group inside the chroot

Might it also be possible that chroot directory was not empty before you
executed sbuild-createchroot?

I'm just puzzled that this problem occurs when you use qemu-static and
--foreign. It's a weird effect...

Thanks!

cheers, josch

#877710#15
Date:
2017-10-04 20:21:45 UTC
From:
To:
I tried the recommended fix using grpck but that got into a whole new
world of trouble. apt had not been installed at this point and although
the .deb existed in /var/cache/apt/archives inside the chroot,
installing it (with manually identified dependencies) didn't result in
a usable system due to some other problem with dpkg.

neil@sylvester:~$ getent group sbuild
sbuild:x:121:neil

# cat /etc/group
sbuild:x:121:neil
sbuild:x:121:neil
root:*:0:
daemon:*:1:
bin:*:2:
sys:*:3:
adm:*:4:
tty:*:5:
disk:*:6:
lp:*:7:
mail:*:8:
news:*:9:
uucp:*:10:
man:*:12:
proxy:*:13:
kmem:*:15:
dialout:*:20:
fax:*:21:
voice:*:22:
cdrom:*:24:
floppy:*:25:
tape:*:26:
sudo:*:27:
audio:*:29:
dip:*:30:
www-data:*:33:
backup:*:34:
operator:*:37:
list:*:38:
irc:*:39:
src:*:40:
gnats:*:41:
shadow:*:42:
utmp:*:43:
video:*:44:
sasl:*:45:
plugdev:*:46:
staff:*:50:
games:*:60:
users:*:100:
nogroup:*:65534:

It was definitely empty. It was a new directory and after
investigating, I removed it and re-ran the same command before filing
the bug report.
the bin format handler to be added, then the second stage handled
manually. sbuild could borrow from other debootstrap wrappers and
support a --bin-fmt option which takes the path to qemu-$ARCH-static
and copies it into the chroot itself.

Note the presence of the sbuild user at the top of /etc/group with the
bug. With a native schroot (freshly created using the same version
using a tmp suffix), /etc/group contains:

# cat /etc/group
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:
tty:x:5:
disk:x:6:
lp:x:7:
mail:x:8:
news:x:9:
uucp:x:10:
man:x:12:
proxy:x:13:
kmem:x:15:
dialout:x:20:
fax:x:21:
voice:x:22:
cdrom:x:24:
floppy:x:25:
tape:x:26:
sudo:x:27:
audio:x:29:
dip:x:30:
www-data:x:33:
backup:x:34:
operator:x:37:
list:x:38:
irc:x:39:
src:x:40:
gnats:x:41:
shadow:x:42:
utmp:x:43:
video:x:44:
sasl:x:45:
plugdev:x:46:
staff:x:50:
games:x:60:
users:x:100:
nogroup:x:65534:
sbuild:x:121:neil

Is this related to sbuild appending to what is initially an empty file?

Also, when using --foreign, sbuild-createchroot isn't particularly helpful when it stops. I tried to replicate the bug with an armhf stretch chroot:

W: The selected architecture and the current architecture do not match
W: (armhf versus amd64).
I: You probably need to add a personality option (see schroot(1)).
I: You may want to report your use case to the sbuild developers so that
I: the appropriate option gets automatically added in the future.

I: sudo chroot configuration linked as /etc/sbuild/chroot/stretch-armhf-sbuild.
/usr/sbin/chroot: failed to run command ‘/bin/sh’: No such file or directory
/usr/sbin/chroot: failed to run command ‘/bin/sh’: No such file or directory
E: Failed to create build directory /build
Failed to set up chroot
E: Error creating chroot session: skipping apt update
I: Successfully set up stretch chroot.

First, if --foreign is supplied, sbuild-createchroot should already
know that chroot will fail without a bin format handler and it hasn't
asked for one, so why does it try? Then when it does try and fails, it
then emits the final message: Successfully set up chroot.

This needs fixing at the same time - this is not a successfully set up
chroot, this is a chroot which currently needs manual intervention
(which is mentioned in the manpage but should also be in the message).

Also exiting zero in this situation is not helpful:
neil@sylvester:~$ echo $?
0

sbuild-createchroot can Recommends: qemu-static (or alternatively
Suggests:) and fail early (as soon as it parses --foreign has been
used) if a bin format handler isn't specified or the one specified
isn't found.

$ sudo cp /usr/bin/qemu-arm-static /srv/chroot/armhf-tmp/usr/bin/
neil@sylvester:~$ sudo chroot /srv/chroot/armhf-tmp/
I have no name!@sylvester:/# /debootstrap/debootstrap --second-stage

This doesn't seem to be arch dependent, the armhf schroot fails in
exactly the same way:

I: Configuring passwd...
I: Configuring libuuid1:armhf...
I: Configuring mount...
I: Configuring libfdisk1:armhf...
I: Configuring e2fsprogs...
I: Configuring libblkid1:armhf...
I: Configuring sysvinit-utils...
I: Configuring libmount1:armhf...
I: Configuring util-linux...
I: Configuring libc-bin...
W: Failure while configuring required packages.
W: See //debootstrap/debootstrap.log for details (possibly the package passwd is at fault)

Setting up passwd (1:4.4-4.1) ...
duplicate group entry
delete line 'sbuild:x:121:neil'? No
group sbuild: no user neil
delete member 'neil'? No
duplicate group entry
delete line 'sbuild:x:121:neil'? No
group sbuild: no user neil
delete member 'neil'? No
grpck: no changes
Multiple entries named 'sbuild' in /etc/group. Please fix this with pwck or grpck.
grpconv: failed to prepare the new /etc/group entry 'sbuild'
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
/bin/chown: cannot access '/etc/gshadow': No such file or directory
/bin/chmod: cannot access '/etc/gshadow': No such file or directory
Please correct the error and rerun `/sbin/shadowconfig on'

I already have a jessie armhf schroot, so this is new behaviour. Sadly,
I don't know of a way to determine which version of sbuild created the
chroot and I don't build these often.

vmdebootstrap has some useful options for this, including the ability
to have "slow build tests" which actually run test bootstraps and check
things like exit values and the presence of certain files in the
chroot. The test language itself is interpreter agnostic, it will work
as well with perl scripts as it does with python. A local mirror is
advisable when running the slow build tests.