#1052942 Upstream testsuite fails to produce deterministic results

Package:
src:insserv
Source:
src:insserv
Submitter:
Lucas Nussbaum
Date:
2024-05-12 17:12:04 UTC
Severity:
normal
Tags:
#1052942#5
Date:
2023-09-26 13:45:26 UTC
From:
To:
Hi,

During a rebuild of all packages in sid, your package failed to build
on amd64.


Relevant part (hopefully):
The full build log is available from:
http://qa-logs.debian.net/2023/09/25/insserv_1.24.0-1_unstable.log

All bugs filed during this archive rebuild are listed at:
https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=ftbfs-20230925;users=lucas@debian.org
or:
https://udd.debian.org/bugs/?release=na&merged=ign&fnewerval=7&flastmodval=7&fusertag=only&fusertagtag=ftbfs-20230925&fusertaguser=lucas@debian.org&allbugs=1&cseverity=1&ctags=1&caffected=1#results

A list of current common problems and possible solutions is available at
http://wiki.debian.org/qa.debian.org/FTBFS . You're welcome to contribute!

If you reassign this bug to another package, please mark it as 'affects'-ing
this package. See https://www.debian.org/Bugs/server-control#affects

If you fail to reproduce this, please provide a build log and diff it with mine
so that we can identify if something relevant changed in the meantime.

#1052942#8
Date:
2023-09-26 16:29:17 UTC
From:
To:
Lucas,

Thanks for this. However, I am currently unable to repoduce this failure in my
customary pbuilder setup. And it doesn't appear at reproducible builds either[1]

My successful pbuilder log is attached.

Mark


[1]  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/insserv.html

#1052942#15
Date:
2023-09-27 21:33:32 UTC
From:
To:
Mark Hindley writes ("Bug#1052942: insserv: FTBFS: insserv: Could not read script nolsbheader: No such file or directory"):

I just tried this myself with my usual sbuild setup and it succeeded
there too[1].  Lucas, I think something from your rebuild environment
(a chroot of some kind I presume) must be triggering this failure.  Is
there some way we can reproduce it more precisely (eg, a buildinfo
file?)

I looked at the build log
http://qa-logs.debian.net/2023/09/25/insserv_1.24.0-1_unstable.log
and compared it to the one from my sbuild, using diff.  There are a
lot of changes to the "furniture" but also there are noise changes to
the output of the insserv test suite, including ordring changes of
passing tests.  This seemed surprising to me.

Mark, is the insserv test suite supposed to produce deterministic
output ?

Ian.

[1] just-updated sid chroot, "dgit sbuild -wgf build -c build".

#1052942#20
Date:
2023-10-04 20:25:25 UTC
From:
To:
Ian


Thanks for your confirmation.

Yes, I agree a buildinfo file might give a hint.

I have never had the need to look at the testsuite since I started looking after
the package. A quick look now doesn't immediately reveal something that would
obviously change the order of the tests.

I tried again locally with make -j8 and still could not reproduce any failure.

Mark

#1052942#27
Date:
2023-10-26 08:01:05 UTC
From:
To:
Hi,
attached log.

Lucas

#1052942#32
Date:
2023-10-29 13:20:15 UTC
From:
To:
Lucas,

I am afraid I still cannot reproduce this.

I attach my successful .buildinfo. What are the differences to yours?

Thanks

Mark

#1052942#37
Date:
2023-10-29 13:39:44 UTC
From:
To:
El 29/10/23 a las 14:20, Mark Hindley escribió:

Hello. The problem is that failed builds do not produce .buildinfo files,
so we can't compare buildinfo files in this case.

However, I can create a machine for you to reproduce the problem.

Please contact me privately for details.

Thanks.

#1052942#42
Date:
2023-10-31 15:17:38 UTC
From:
To:
Santiago,

Thanks. I have reproduced on your provided machine, but still not locally and I
can't identify the underlying difference between the builds.

In the failure case the problem is in the upstream testsuite, specifically the
test for #491391 in tests/run-testsuite which produces

init.d:
bootchart
four
one
rmnologin
three
two

insserv:
override

rc0.d:

rc1.d:

rc2.d:
S01one
S01three
S01two
S02four
S98rmnologin
S99bootchart

rc3.d:
S01one
S01three
S01two
S02four
S98rmnologin
S99bootchart

rc4.d:
S01one
S01three
S01two
S02four
S98rmnologin
S99bootchart

rc5.d:
S01one
S01three
S01two
S02four
S98rmnologin
S99bootchart

rc6.d:

rcS.d:
error: incorrect 5 sequence bootchart not before rmnologin

The same failure mode appears to be responsible for armel and armhf autopkgtest
failures logged on ci.debian.net[1]

As Ian pointed out[2], there are significant and surprising changes in looping
order and behaviour between the successful and failing testsuites. The diff is attached.

Having said that, I still can't reproduce locally or determine a good fix.
Hopefully Jesse will have a useful contribution

Mark

[1]  https://ci.debian.net/data/autopkgtest/unstable/armel/i/insserv/38435862/log.gz

[2]  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1052942#15

#1052942#51
Date:
2023-11-01 19:09:41 UTC
From:
To:
Could this be related to #989284 where we also haven’t figured out
yet why this happens on that machine?

Meow,
//mirabilos

#1052942#56
Date:
2024-01-10 17:22:50 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
insserv, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1052942@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Mark Hindley <leepen@debian.org> (supplier of updated insserv package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Wed, 10 Jan 2024 16:35:21 +0000
Source: insserv
Architecture: source
Version: 1.24.0-2
Distribution: unstable
Urgency: medium
Maintainer: Debian sysvinit maintainers <debian-init-diversity@chiark.greenend.org.uk>
Changed-By: Mark Hindley <leepen@debian.org>
Closes: 1052942 1056176
Changes:
 insserv (1.24.0-2) unstable; urgency=medium
 .
   [ Mark Hindley ]
   * Drop Petter Reinholdtsen from uploaders with thanks.
   * Update standards version to 4.6.2 (no changes).
   * d/postinst: add patch from josch@debian.org to support DPKG_ROOT.
     (Closes: #1056176)
   * d/t/control: mark upstream tests as flaky (See #1052942).
   * Ignore flaky upstream testsuite failures, pending upstream fix.
     (Closes: #1052942)
 .
   [ Debian Janitor ]
   * Remove 1 obsolete maintscript entry.
   * Bump debhelper from old 12 to 13.
   * Set upstream metadata fields: Bug-Database, Repository-Browse.
Checksums-Sha1:
 50579e15dc668189967e79e204b4b944d4b15bad 1957 insserv_1.24.0-2.dsc
 87f46a4bdb1acf72e8835f9f4745f7c92b255cf7 36424 insserv_1.24.0-2.debian.tar.xz
 890379b8d0d825c60c07d6eb2d642e1d43530f82 6730 insserv_1.24.0-2_amd64.buildinfo
Checksums-Sha256:
 9ca07a513c8c8a7faa2ea0afc468197bb58b952676f45efba662c21971b3ee8e 1957 insserv_1.24.0-2.dsc
 484976d02f71797a80c4e1b8f1a5c3c81120b4d84c98cbc627ad0b80f8c33c7f 36424 insserv_1.24.0-2.debian.tar.xz
 2dd4a402895220d25154407659d9d67058c5f3669bac8ce18598ab630539bf81 6730 insserv_1.24.0-2_amd64.buildinfo
Files:
 9214efcdc5171003a1b5d5bd2edbfc4f 1957 misc optional insserv_1.24.0-2.dsc
 f455b0339757753b05b414dbe7cb4a83 36424 misc optional insserv_1.24.0-2.debian.tar.xz
 71db373749732d8ba076604421c4cf28 6730 misc optional insserv_1.24.0-2_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEUGwVpCsK9aCoVCPu0opFvzKH1kkFAmWeyLAACgkQ0opFvzKH
1klmbQ//eCtrNnd11FI5LYyVZsGZ4m+afrMo0GVe6tNrPolESZ9NYyQ5MC3fsE9n
nFeCegGsNAb81oXYrf+i4kBEKIfVrHaGhZaBS2hV7S6FRVITRPaNr5sJxx/38N0X
UMP5HJIILpAkqUSvmnRoGZu0ju1NnWA4VlARQkxPf/NokCLlcANB/EvpxwusQ4n8
b/glTGYHUAkoF56HD/od9Jor2Lmi1XNJ6C+L7Z/CIZ+kRMrmEnUdihhcVlwq3Hz8
Kdfr5pbFmZTNdahKmXJzp92oFK2gqyaY+S2kMOCfKeNONXhAR/9mlXIlPLR26sL3
jOJ5MPqrDMNBJUCQMGp1H7em2cF5x3I4MmJxiFYhd6Sya6yWRvmmRy6l9Hpomplp
bVeoe63C5KVCHUwli0L6Ttext4NhOIOEw/HkKhbsEAQdlCC7ILPAkhBYrMJF0kGy
Ere2PjkplMYeAXZBSudBlTHDj2oti/u8KS2nYh8RdKg++cZrvYkqPN2Tb2QGsrbi
b8p+Jws7f/jt0OBr2xTZqSdfbrOZFftfEet4CIk4FWYjeFB7PmvyO8exnsqKbq2C
BG1PxAagXEmqMJBPwYkIQQuB/YwA8nWpQuq6GCdFGyVtsOllyvOekSquHiT+UK6+
BAjNNDGJ3JRHonzkTkzFEJ+ysOZptDFmS89+cOsI8YaFqjYs0Bc=
=cqxF
-----END PGP SIGNATURE-----