Hi!

While working on the “reproducible builds” effort [1], we have noticed
that debian-installer could not be built in some configurations.
It could be an effect of any of the commands in [2], but it is likely to
be the locale.

The attached file contains the full build logs.

 [1]: https://wiki.debian.org/ReproducibleBuilds
 [2]:
https://alioth.debian.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=reproducible/misc.git;a=blob;f=prebuilder/pbuilderhooks/A02_user;h=a3c8ceb9a4e5b0aa069e146bc50d3757d89a2e1f;hb=b012c16d7349a30b3c906851c171670abbced407

Regards,
Valentin

Hi,

Val Lorentz <progval@progval.net> (2015-12-03):

Focusing on that part:

This list of packages is identical when building locally (with a regular
locale):

$ locale
LANG=en_GB.UTF-8
LANGUAGE=
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=

(except mine is about NEW packages.)

This loop seems to be missing quite a bunch of packages for reasons I
don't know yet. I'm attaching a diff (beware of some version skews
because the builds happened at different times, so stretch had different
packages), between ok-unpacking (my successful build) and ko-depaquetage
(yours), after selecting with awk the “package (version)” part of the
unpack messages, just in case someone figures out what could be
responsible for such a diff:

kibi@chloe:~$ colordiff -u ok-unpacking ko-depaquetage
--- ok-unpacking	2015-12-04 05:25:15.454347994 +0100
+++ ko-depaquetage	2015-12-04 05:25:55.108452336 +0100
@@ -1,18 +1,16 @@
 acpi-modules-4.2.0-1-amd64-di (4.2.6-1)
-alsa-utils-udeb (1.0.29-1+b1)
 anna (1.54)
 archdetect (1.114)
-ata-modules-4.2.0-1-amd64-di (4.2.6-1)
 bogl-bterm-udeb (0.1.18-10)
 brltty-udeb (5.2~20141018-5+b2)
-busybox-udeb (1:1.22.0-15)
+busybox-udeb (1:1.22.0-15+b1)
 cdebconf-gtk-terminal (0.23)
-cdebconf-gtk-udeb (0.197)
+cdebconf-gtk-udeb (0.199)
 cdebconf-newt-terminal (0.23)
-cdebconf-newt-udeb (0.197)
-cdebconf-priority (0.197)
-cdebconf-text-udeb (0.197)
-cdebconf-udeb (0.197)
+cdebconf-newt-udeb (0.199)
+cdebconf-priority (0.199)
+cdebconf-text-udeb (0.199)
+cdebconf-udeb (0.199)
 cdrom-checker (1.31)
 cdrom-core-modules-4.2.0-1-amd64-di (4.2.6-1)
 cdrom-detect (1.59)
@@ -20,21 +18,16 @@
 console-setup-linux-fonts-udeb (1.134)
 console-setup-pc-ekmap (1.134)
 console-setup-udeb (1.134)
-core-modules-4.2.0-1-amd64-di (4.2.6-1)
-crc-modules-4.2.0-1-amd64-di (4.2.6-1)
 di-utils-reboot (1.112)
 di-utils-shell (1.112)
 di-utils-terminfo (1.112)
 di-utils (1.112)
 env-preseed (1.68)
-espeak-data-udeb (1.48.04+dfsg-1)
 espeakup-udeb (1:0.71-19)
-event-modules-4.2.0-1-amd64-di (4.2.6-1)
 fat-modules-4.2.0-1-amd64-di (4.2.6-1)
 fb-modules-4.2.0-1-amd64-di (4.2.6-1)
 file-preseed (1.68)
 firewire-core-modules-4.2.0-1-amd64-di (4.2.6-1)
-fontconfig-udeb (2.11.0-6.3)
 fonts-android-udeb (1:4.4.4r2-7)
 fonts-farsiweb-udeb (0.4.dfsg-12)
 fonts-khmeros-udeb (5.0-7)
@@ -53,63 +46,22 @@
 gtk2-engines-udeb (1:2.20.2-3)
 hw-detect (1.114)
 hyperv-modules-4.2.0-1-amd64-di (4.2.6-1)
-i2c-modules-4.2.0-1-amd64-di (4.2.6-1)
 initrd-preseed (1.68)
 input-modules-4.2.0-1-amd64-di (4.2.6-1)
 installation-locale (1.5)
-isofs-modules-4.2.0-1-amd64-di (4.2.6-1)
 kbd-udeb (1.15.5-2)
 kernel-image-4.2.0-1-amd64-di (4.2.6-1)
-libasound2-udeb (1.0.29-1)
-libatk1.0-udeb (2.18.0-1)
-libblkid1-udeb (2.27.1-1)
-libcairo2-udeb (1.14.4-1)
-libdebconfclient0-udeb (0.197)
+libdebconfclient0-udeb (0.199)
 libdebian-installer4-udeb (0.102)
-libdrm2-udeb (2.4.65-3)
-libevdev2-udeb (1.4.5+dfsg-1)
-libexpat1-udeb (2.1.0-7)
-libffi6-udeb (3.2.1-3)
-libfontenc1-udeb (1:1.1.3-1)
-libfreetype6-udeb (2.6.1-0.1)
 libfribidi0-udeb (0.19.7-1)
-libgdk-pixbuf2.0-0-udeb (2.32.2-1)
-libglib2.0-udeb (2.46.2-1)
-libgtk2.0-0-udeb (2.24.28-1)
-libharfbuzz0-udeb (1.0.1-1+b1)
 libkmod2-udeb (21-1)
-libmtdev1-udeb (1.1.5-1)
-libnss-dns-udeb (2.19-22)
-libnss-files-udeb (2.19-22)
-libpango1.0-udeb (1.38.1-1)
-libpciaccess0-udeb (0.13.4-1)
-libpcre3-udeb (2:8.35-8)
-libpixman-1-0-udeb (0.33.4-1)
-libpng12-0-udeb (1.2.54-1)
-libtextwrap1-udeb (0.1-14)
-libudev1-udeb (228-2)
-libuuid1-udeb (2.27.1-1)
-libvte9-udeb (1:0.28.2-5)
-libx11-6-udeb (2:1.6.3-1)
-libxau6-udeb (1:1.0.8-1)
-libxcb1-udeb (1.10-3+b1)
-libxcursor1-udeb (1:1.1.14-1+b1)
-libxdmcp6-udeb (1:1.1.2-1)
-libxext6-udeb (2:1.3.3-1)
-libxfixes3-udeb (1:5.0.1-2+b2)
-libxfont1-udeb (1:1.5.1-1)
-libxft2-udeb (2.3.2-1)
-libxi6-udeb (2:1.7.5-1)
-libxinerama1-udeb (2:1.1.3-1+b1)
-libxkbfile1-udeb (1:1.0.9-2)
-libxrender1-udeb (1:0.9.9-2)
-libxshmfence1-udeb (1.2-1)
+libnss-dns-udeb (2.21-1)
+libnss-files-udeb (2.21-1)
 load-cdrom (1.37)
 localechooser (2.67)
 lowmemcheck (1.43)
 main-menu (1.47)
 media-retriever (1.38)
-mmc-core-modules-4.2.0-1-amd64-di (4.2.6-1)
 mmc-modules-4.2.0-1-amd64-di (4.2.6-1)
 mountmedia (0.23)
 mouse-modules-4.2.0-1-amd64-di (4.2.6-1)
@@ -131,7 +83,6 @@
 serial-modules-4.2.0-1-amd64-di (4.2.6-1)
 sound-modules-4.2.0-1-amd64-di (4.2.6-1)
 speakup-modules-4.2.0-1-amd64-di (4.2.6-1)
-ttf-dejavu-mono-udeb (2.33-3)
 ttf-dejavu-udeb (2.33-3)
 ttf-freefont-udeb (20100919-1)
 udev-udeb (228-2)
@@ -140,10 +91,6 @@
 usb-modules-4.2.0-1-amd64-di (4.2.6-1)
 usb-serial-modules-4.2.0-1-amd64-di (4.2.6-1)
 usb-storage-modules-4.2.0-1-amd64-di (4.2.6-1)
-util-linux-udeb (2.27.1-1)
-x11-xkb-utils-udeb (7.7+2)
-xkb-data-udeb (2.16-1)
 xserver-xorg-core-udeb (2:1.17.3-2)
 xserver-xorg-input-evdev-udeb (1:2.9.2-1)
 xserver-xorg-video-fbdev-udeb (1:0.4.4-1+b3)
-zlib1g-udeb (1:1.2.8.dfsg-2+b1)


Mraw,
KiBi.

--
Sent from myMail app for Android

Hi,

Val Lorentz <progval@progval.net> (2015-12-03):

Looking at your A02_user hook, I don't see anything locale-related (now or
in previous commits). I've tried setting LANG=fr_CH.UTF-8 and I don't see
debian-installer's master fail to build in a sid chroot.

Can you please tell us whether this issue is still seen in your setup, and
with which exact settings? I can't replicate it with my devel chroots,
it's not seen on buildds, so I'm lowering the severity for the time being.
It can be upgraded again if a relevant reproducer is found.

Thanks for your time.


KiBi.

Hi,

when possible we don't modify the environment with pbuilder hooks but
rather directly with our build script
https://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/bin/reproducible_build.sh

have a look at lines 591-600 for the 1st build and 637-660 for the 2nd
build.

Upon replying I've scheduled rebuilds of src:debian-installer for
(amd64|i386|armhf) on unstable+testing and the rebuilds have all already
happened, all ftbfs…

the results are all be linked from https://reproducible.debian.net/debian-installer

thanks!

likewise! :)

Hi,

Holger Levsen <holger@layer-acht.org> (2016-11-20):
| I: pbuilder: network access will be disabled during build
[…]
| WARNING: mirror 'http://ftp.de.debian.org/debian' appears to be invalid; skipping
| WARNING: mirror 'http://ftp.de.debian.org/debian' appears to be invalid; skipping
| Using generated sources.list.udeb:
|    deb [trusted=yes] copy:/build/1st/debian-installer-20161031/build/ localudebs/

FTBFS due to 4.7 vs. 4.8 kernel udebs is expected to be an issue (fixed
in master where the ABI bump happened; but failing to download any udebs
is a no-go, d-i needs to access a mirror during its build.


KiBi.

retitle -1 debian-installer: ftbfs because d-i needs network to build…
thanks
[...]

ah, ic, retitling the bug accordingly. Thanks.

Holger Levsen <holger@layer-acht.org> (2016-11-20):

Let's try again without forgetting the “Control:” bit.


KiBi.