#1102308 neovim-qt: FTBFS: flaky tests

Package:
src:neovim-qt
Source:
src:neovim-qt
Submitter:
Santiago Vila
Date:
2026-06-29 21:29:02 UTC
Severity:
normal
Tags:
#1102308#5
Date:
2025-04-07 13:21:03 UTC
From:
To:
Dear maintainer:

During a rebuild of all packages in unstable, your package failed to build:

[ please read the notes at the end ]
--------------------------------------------------------------------------------
[...]
 4/20 Test  #4: tst_encoding .....................   Passed    0.05 sec
      Start  5: tst_msgpackiodevice
 5/20 Test  #5: tst_msgpackiodevice ..............   Passed    0.04 sec
      Start  6: tst_shell
 6/20 Test  #6: tst_shell ........................***Failed    4.39 sec
********* Start testing of NeovimQt::TestShell *********
Config: Using QtTest library 5.15.15, Qt 5.15.15 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 14.2.0), debian unknown
PASS   : NeovimQt::TestShell::initTestCase()
QWARN  : NeovimQt::TestShell::benchStart() QProcess: Destroyed while process ("nvim") is still running.
[...]
Total Test time (real) =  36.51 sec

The following tests FAILED:
	  6 - tst_shell (Failed)
Errors while running CTest
FAILED: CMakeFiles/test.util
cd /<<PKGBUILDDIR>>/build && /usr/bin/ctest --force-new-ctest-process
ninja: build stopped: subcommand failed.
dh_auto_test: error: cd build && LC_ALL=C.UTF-8 MESON_TESTTHREADS=1 ninja test returned exit code 1
make[1]: *** [debian/rules:39: override_dh_auto_test] Error 25
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:21: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
--------------------------------------------------------------------------------

The above is just how the build ends and not necessarily the most relevant part.
If required, the full build log is available here:

https://people.debian.org/~sanvila/build-logs/202503/

About the archive rebuild: The build was made on virtual machines from AWS,
using sbuild and a reduced chroot with only build-essential packages.

In this particular case, the build was made on AWS machines of type
m7a.medium and r7a.medium. Incidentally, those machines have a single CPU,
but at this point it's not clear yet if that's the condition which triggers
the build failure or, for example, the fact that they are overall slower
than other instance types, or maybe some other reason.

If you could not reproduce the bug using GRUB_CMDLINE_LINUX="nr_cpus=1"
please contact me privately, as I am willing to provide ssh access to a
virtual machine where the bug is fully reproducible.

Disclaimer: Please note that this is probably a violation of
Debian Policy 4.2, and the only reason I'm initially reporting
it as "important" is to avoid discussing about the severity.

If this is really a bug in one of the build-depends, please use
reassign and add an affects on src:neovim-qt, so that this is still
visible in the BTS web page for this package.

Thanks.

#1102308#10
Date:
2025-05-11 20:02:56 UTC
From:
To:
severity 1102308 normal
thanks

Update:

When using the kernel of trixie, the failure rate
(on the systems where it used to fail all the time)
becomes a lot smaller (around 5%).

(So I can remove the package from my list of "always fails"
and put it in the "randomly fails").

Thanks.

#1102308#19
Date:
2026-06-29 18:48:16 UTC
From:
To:
severity 1102308 serious
thanks

Hello. This issue has switched recently from being a random thing to
being an always thing. To illustrate, this is my build history on
machines with 1 CPU:

Status: successful  neovim-qt_0.2.19-1_amd64-20260531T055336.795Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260602T045417.108Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260603T045029.231Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260604T044910.167Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260605T044435.516Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260606T044112.574Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260606T044800.292Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260607T043704.447Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260608T043259.594Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260609T044146.994Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260610T044242.412Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260611T043647.678Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260612T044657.122Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260613T043045.126Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260614T043756.958Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260615T043456.978Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260616T020854.506Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260617T043820.170Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260618T044515.779Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260619T164018.542Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260620T092214.329Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260621T044732.609Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260625T222855.416Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260626T214512.500Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260628T045639.181Z

and this is my build history on machines with 2 CPUs:

Status: successful  neovim-qt_0.2.19-1_amd64-20260602T043815.881Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260603T043707.808Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260604T043248.780Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260605T043448.298Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260606T044636.348Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260606T045956.552Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260607T043156.359Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260608T042210.807Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260608T154404.351Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260609T042908.681Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260610T043602.401Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260611T042755.835Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260612T043130.034Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260613T045344.618Z
Status: successful  neovim-qt_0.2.19-1_amd64-20260614T043044.517Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260615T042432.584Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260617T042947.598Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260618T043240.828Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260619T164957.183Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260620T094732.958Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260621T042705.425Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260625T222907.396Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260626T214526.711Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260627T100850.611Z
Status: failed      neovim-qt_0.2.19-1_amd64-20260628T050529.683Z

It looks like something happened around 2026-06-15 which makes this
failure to happen a lot more than before, to the point that I can't
build the package at all anymore.

The failing tests are now these ones:

The following tests FAILED:
          6 - tst_shell (Failed)
          8 - tst_qsettings (Failed)

and this also happens here:

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/neovim-qt.html

My recommendation would be to disable those two tests as "flaky".

As always, if you prefer to debug this, I can provide a VM for testing
(please contact me privately for that).

Thanks.

#1102308#26
Date:
2026-06-29 21:26:29 UTC
From:
To:
tags 1102308 patch
thanks

Proposed patch. I've actually tested and it works for me. Thanks.