- Package:
- golang-github-google-goexpect
- Source:
- golang-github-google-goexpect
- Submitter:
- Simon Josefsson
- Date:
- 2026-09-26 08:23:01 UTC
- Severity:
- normal
- Tags:
Hi! This package seems to have flaky tests which triggers testing migration blockers, see what I believe are random build failures: https://tests.reproducible-builds.org/debian/rb-pkg/forky/arm64/golang-github-google-goexpect.html https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/golang-github-google-goexpect.html https://ci.debian.net/packages/g/golang-github-google-goexpect/testing/i386/64964676/ https://salsa.debian.org/go-team/packages/golang-github-google-goexpect/-/jobs/8113152 Looking at the error messages, this looks timing related so there is probably a race condition somehwere. Maybe the 20s timeout is not sufficient on slow machines. For flaky tests, I usually report these upstreams and mask it out of the Debian build. I initially thought keeping flaky tests around was acceptable, but I've realized it causes serious Debian testing migration pains so I believe we must not have any flaky tests in Debian. Here is an idea: override_dh_auto_test: dh_auto_test $(DH_BUILD_OPTS) -- -skip=ExampleGExpect_SendSignal -dh_auto_test $(DH_BUILD_OPTS) This allows the failure message to still become available in build logs, for future references in bug reports. If it only has happened on a particular arch, here is an idea: ifneq (,$(filter $(DEB_TARGET_ARCH), ppc64el)) override_dh_auto_test: dh_auto_test $(DH_BUILD_OPTS) -- -skip=ExampleGExpect_SendSignal -dh_auto_test $(DH_BUILD_OPTS) endif For complex workarounds I have used this: ifneq (,$(filter $(DEB_TARGET_ARCH), i386 ppc64el riscv64 s390x)) override_dh_auto_test: ifneq (,$(filter $(DEB_TARGET_ARCH), i386)) dh_auto_test $(DH_BUILD_OPT) -- --skip=TestAllocate endif ifneq (,$(filter $(DEB_TARGET_ARCH), ppc64el riscv64 s390x)) dh_auto_test $(DH_BUILD_OPT) -- --skip='TestTreePersistent|TestBuffer' endif -dh_auto_test $(DH_BUILD_OPT) -- --timeout=20m endif /Simon
Failures seen again: https://ci.debian.net/packages/g/golang-github-google-goexpect/testing/riscv64/71982686/#L1655 1178s === RUN TestSendSignal 1178s === RUN TestSendSignal/Sig_USR1 1179s 2026/06/12 10:32:13 Match for RE: "Waiting for signal" found: ["Waiting for signal"] Buffer: "Waiting for signal\n" 1179s 2026/06/12 10:32:13 Match for RE: "USR1" found: ["USR1"] Buffer: "Got the USR1 Signal\n" 1179s === RUN TestSendSignal/Sig_HUP 1179s 2026/06/12 10:32:13 Match for RE: "Waiting for signal" found: ["Waiting for signal"] Buffer: "Waiting for signal\n" 1181s expect_test.go:1341: Sig HUP: exp.Expect("HUP", 2s) failed: expect: timer expired after 2 seconds, match: , buf: [] 1181s --- FAIL: TestSendSignal (2.11s) 1181s --- PASS: TestSendSignal/Sig_USR1 (0.07s) 1181s --- FAIL: TestSendSignal/Sig_HUP (2.04s) ... 1183s === RUN ExampleGExpect_SendSignal 1203s --- FAIL: ExampleGExpect_SendSignal (20.04s) 1203s got: 1203s exp.Expect failed, match: , buf: [], err: expect: timer expired after 20 seconds 1203s want: 1203s Got the USR1 Signal 1203s === RUN ExampleGExpect_SendSignal_Batch 1233s --- FAIL: ExampleGExpect_SendSignal_Batch (30.04s) 1233s got: 1233s ExpectBatch failed: expect: timer expired after 30 seconds 1233s want: 1233s Signal received 1233s FAIL It is not clear if this is the same problem as in the earlier report, the only link that still worked contains this snippet: === RUN ExampleGExpect_SendSignal --- FAIL: ExampleGExpect_SendSignal (20.01s) got: exp.Expect failed, match: , buf: [], err: expect: timer expired after 20 seconds want: Got the USR1 Signal /Simon
More flaky failures: https://ci.debian.net/packages/g/golang-github-google-goexpect/testing/arm64/71981794/#L1654 110s === RUN TestSpawnWithArgs 110s expect_test.go:1110: Expect(a b) failed: expect: Process not running 110s --- FAIL: TestSpawnWithArgs (0.00s) /Simon
Another example: https://ci.debian.net/packages/g/golang-github-google-goexpect/testing/armhf/74274207/#L1733 151s === RUN TestSpawnWithArgs 151s expect_test.go:1110: Expect(a b) failed: expect: Process not running 151s --- FAIL: TestSpawnWithArgs (0.00s) /Simon
severity 1117471 serious thanks release managers announced that flaky tests should be considered RC. So, if you see a flaky test somewhere, please don't be shy and report it as "important" at least :-) For this package, I'm getting a failure rate of 40% on machines with 1 CPU and also 40% on machines with 2 CPUs (after trying 25 times in each case), neither of which I would consider acceptable for a stable release, so I'm raising the severity accordingly. I've put a small collection of failed build logs here: https://people.debian.org/~sanvila/build-logs/202609/ I see that last year Otto tried to improve the status of some tests. I'm not sure if doing that is worth the effort for this particular package, because the upstream repository is archived: https://github.com/google/goexpect So: what are your current thoughts about this? Note that the failing tests are not always the same, but the following two happen quite often: TestSendSignal TestSpawnWithArgs My preferred course of action here would be to disable the above two tests. I volunteer to do that, because those tests are noise in my own building environment. However, if somebody still wants to spend time fixing them, they are welcome of course. I see the idea of "preserving information" behind that, but it has a price: it makes debian/rules a little bit more longer and therefore a little bit more difficult to understand and maintain. (So, personally, I don't particularly like such construct and would prefer the simple approach of skipping flaky tests where necessary) Note that this can be tricky. It may be the case that the test has only failed *so far* on a particular architecture, but that may happen for a lot of different reasons. If the failure is not *inherent* to the architecture, and it happens just because the test is wrongly designed, skipping the test for everybody might be a better option. As before, the more complex this becomes, the more complex it will be to read it, understand it, or maintaing it. In my opinion, we are already good enough citizens in the FOSS world when we forward the bug upstream. After that, the issue is in the hands of the authors, and we have the "right", so to speak, not to make things more complex than required for ourselves. [ sorry for the long email... ] Thanks.
What is we simply remove golang-github-google-goexpect from Debian? It won't prevent anyone from installing anything, only the builds of usql and spike will fail. Both of them will however build on next upload, as both upstreams stopped using goexpect 5 days ago: https://github.com/spiffe/spike/commit/fc56e0fd0686e020629e7db78e4031399449b8e9#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6 https://github.com/xo/usql/commit/df18e3d2fae2286105ac7daa3a5b14669b0f3fe7 Therefore, the next import and upload of those packages will no longer require goexpect.
LGTM!