Dear Maintainer,

diffoscope crashes when comparing the build results of src:dasel. To make it
more fun, src:dasel is only unreproducible on i386 (out of our four tested
archs, amd64/i386/arm64/armhf) and only *sometimes*.

vagrant added the following note to reproducible-notes.git, visible at
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/dasel.html
---begin-note---
timezone-dependent date in manpages triggered when building with
reprotest but not reproducible builds test infrastructure.
dasel itself is used to generate the manpage.
https://sources.debian.org/src/dasel/2.7.0-1/internal/command/man.go/
.
Something non-deterministic, possibly GO BUILDID only on i386.
---end-note---

several build artifacts at available at
https://tests.reproducible-builds.org/debian/artifacts/r00t-me/
and only the i386 ones are sometimes unreproducible and then
crashing diffoscope. (Please download them for investigations,
they will vanish after 48h but I can easily and quickly recreate
them anytime.)

[...]

I'm attaching the crashing artifacts now to this bug report, however minus
the orig.tar.gz files, though I suppose that the .deb files are enough to
crash diffoscope anyway...

[...]

I'm attaching the crashing artifacts now to this bug report, however minus
the orig.tar.gz files, though I suppose that the .deb files are enough to
crash diffoscope anyway...

Holger Levsen wrote:

Hm, I can't seem to reproduce the crash with these files. In the first
instance, can you paste a traceback or similar of the crash in
question? Maybe it is fixable just from that without having to find
and upload more files, etc.


Best wishes,

processes...

May 13 20:45:41 jenkins kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=/,mems_allowed=0,oom_memcg=/user.slice/user-103.slice/user@103.service/rb.slice,task_memcg=/user.slice/user-103.slice/user@103.service/rb.slice/rb-build.slice/rb-build-diffoscope.slice/rb-diffoscope-i386_1-26686.service,task=diffoscope,pid=4117661,uid=103
May 13 20:45:41 jenkins kernel: Memory cgroup out of memory: Killed process 4117661 (diffoscope) total-vm:218023820kB, anon-rss:86849224kB, file-rss:40448kB, shmem-rss:0kB, UID:103 pgtables:425080kB oom_score_adj:200
May 13 20:45:41 jenkins systemd[1]: user@103.service: A process of this unit has been killed by the OOM killer.
May 13 20:45:41 jenkins systemd[1620]: rb-diffoscope-i386_1-26686.service: A process of this unit has been killed by the OOM killer.

is basically all I see.

https://tests.reproducible-builds.org/debian/artifacts/r00t-me/trixie_i386_dasel_tmp-kqFaQ/
is maybe working as in crashing for you?

Holger Levsen wrote:

Ah, I was hoping that the systemd slice apparatus would be able to
contain any traceback, but now that I think of it, being OOM-killed is
not quite the same as CPython-level crash (and thus traceback).

Alas, this works for me and does not crash. I suppose the next thing
might be to try and run with --debug? That way, we might be able to
determine which file, comparator or external tool was being run when
diffoscope invoked the ire of the oom-killer.


Regards,

:)

I'm not sure how --debug output should survive, but you mean just running
diffoscope with an added --debug option?

Holger Levsen wrote:

Ah, yeah. It won't survive from Jenkins' log perspective, huh?

Hmm, the --debug output could perhaps to be directed straight to an
on-disk file. Given that that should be flushed after every line, that
should survive an OOM kill.

If not, hmm, I'll have a think. Either way, apologies that I'm not
more familiar with all the abstraction layers in our setup and thus
which might survive an OOM.


Regards,