flatpak_1.18.0-1 compiled successfully on the first attempt on all of
the official buildds (all architectures) and on reproduce.debian.net
(all architectures except amd64), but its build-time tests failed on
reproduce.debian.net on amd64:

https://reproduce.debian.net/amd64/api/v1/builds/261317/log

The log sections for those two tests don't show anything obviously
wrong, until SIGTERM is received and cleanup starts.

On the official amd64 buildd, the whole of test-auth.sh took 8.31
seconds. On reproduce.debian.net, test-auth.sh started at 7:13:09 and
successfully completed 1 out of 3 sub-tests ("installed build-exported
token-type app") shortly after 07:13:51, meaning about 42 seconds to
complete 1 out of 3 sub-tests - that seems quite dramatically slower for
a test that ought to be mostly I/O-bound. Based on this, I suspect the
test might have passed if it had been given an arbitrarily large
timeout.

Similarly test-preinstall.sh seems to have still been making forward
progress on the first of 12 sub-tests about a minute after it started
(that's the last timestamp I can immediately see in its log), compared
with having only taken 12.13 seconds in total for all 12 sub-tests on
the official amd64 buildd.

Obviously I can increase arbitrary timeouts if I have to (and non-x86
architectures already have a much longer timeout), but it seems
unexpected that test-auth.sh took more than 10x as long as it did on the
official amd64 buildd. For comparison, test-auth.sh and
test-preinstall.sh both took about a minute on an official *riscv64*
buildd, and the riscv64 buildds have been noted to be slow enough to be
causing practical problems.

The upstream default (from Meson) is that "most" tests are allowed 30
seconds to run and then killed, with a few exceptions where the timeout
is extended for particularly slow tests. On x86 the Debian packaging
multiplies timeouts by 3 (90s for most tests, scaling up appropriately
for particularly slow ones), and on non-x86 it multiplies timeouts by 20
(10 minutes for most tests).

I'm reluctant to increase the timeout excessively on x86, because
Flatpak's test suite (like many test-suites) is rather complicated, and
if one of the tests genuinely gets stuck (no longer making forward
progress), having it block for multiple minutes before giving up is an
unwelcome debugging experience.

Can reproduce.debian.net jobs be retried, to see whether this might be a
one-off glitch?

Are reproduce.debian.net amd64 machines known to be significantly
(orders of magnitude!) slower than official amd64 buildds, or my laptop,
or even riscv64 buildds?

Or does reproduce.debian.net perhaps parallelize builds of unrelated
packages on the same machine, which might have caused the tests to be
extra-slow?

I also wonder whether running the build-time test-suite is necessary for
reproduce.debian.net's goals: if we're trying to prove that the binaries
in the archive are genuine, it seems to me that a build with the nocheck
option would be sufficient for demonstrating that. Obviously test-suites
that don't always work are a bug, but perhaps that's orthogonal to
whether an existing package can be reproduced?

Thanks,
    smcv

Hi Simon,

answering here for more visibility.

* Simon McVittie <smcv@debian.org> [2026-06-19 11:32]:

Non reproducible packages are retried 3 times automatically and you can
always ask in #debian-reproducible. A self service is in the works.
flatpak is in the queue but it is currently rather long.

No, they are reasonably fast from my experience and for sure faster then
riscv64.

No, they are KVM on top of Cascadelake Intel Xeon Processor with one
build per VM.

That is surely desirable but currently the goal of reproduce.d.n is to
match the Debian buildds as close as possible and not to add more
variation. After all this should be a QA system and not pose more work
for the maintainers. reproducible-builds.org is using nocheck and once
that is well established in the archive we can also think about enabling
that on reproduce.d.n.

Cheers Jochen