#1074781 request-tracker5/4: autopkgtests don't work directly with new MariaDB 11.4

#1074781#5
Date:
2024-07-03 01:31:56 UTC
From:
To:
MariaDB 1:11.4.2-1 was recently uploaded to Debian. MariaDB 11.4 and MySQL 8.4
are slowly diverging more and more.

In MariaDB 11.4 the 'mysql' naming in filenames has shifted towards 'mariadb'.
Hence the request-tracker5 (both version 5 and 4) autopkgtests are currently
failing, but the reason is a bit unclear.

Could you please assist in reading what the actual error in the logs is?

https://ci.debian.net/packages/r/request-tracker5/testing/armel/48409176/
https://ci.debian.net/packages/r/request-tracker4/testing/armel/48409175/

There are 60 tests that say 'not ok' but it is hard to figure out which one of
them is new, and why they are failing with MariaDB 11.4.

#1074781#12
Date:
2024-07-03 06:54:55 UTC
From:
To:
Hi Otto, sure thing, I'll have a look.

Cheers,
Andrew

#1074781#17
Date:
2024-07-28 12:10:42 UTC
From:
To:
The issue isn't related to the mariadb package changing (the tests use
SQLite), but rather that occasionally some of the tests fail. Looking at the
failing tests in https://ci.debian.net/packages/r/request-tracker5 across
various different triggers, they appear to be the same (or a similar) set of
tests. But they fail intermittently.

I have a feeling there is a race condition in the tests, but, they should be
running independently with their own server and database.

More investigation required.

#1074781#26
Date:
2024-09-06 12:19:42 UTC
From:
To:
Hi,
This happens often enough on some architectures on ci.d.n that the
Release Team considers this RC. Because the unstable-to-testing
migration software now blocks on regressions in testing, flaky tests,
i.e. tests that flip between passing and failing without changes to the
list of installed packages, are causing people unrelated to your package
to spend time on these tests. Just like Otto did.

Paul

#1074781#31
Date:
2024-09-11 12:37:33 UTC
From:
To:
Re: Paul Gevers

Fwiw, I stumbled over this last week when it was blocking
postgresql-17, and now again for blocking postgresql-common.

If you cannot fix the tests reliably, mark them as "flaky".

Thanks,
Christoph

#1074781#38
Date:
2024-09-27 11:59:39 UTC
From:
To:
Hey,

I believe that I've fixed the intermittent issue with the tests failing in 5.0.7+dfsg-2 with [0]. However as I'm not 100% sure yet, I've set the tests as flaky for now.

I'll keep this bug open until I'm 100% confident the tests are reliable.

My apologies for causing people to waste their time looking into these tests. I hadn't realised how flaky they were.

Cheers,
Andrew

[0] https://salsa.debian.org/request-tracker-team/request-tracker5/-/commit/44497367f88096a9f91439682b3c0460bf905eb0

#1074781#47
Date:
2024-10-02 21:08:37 UTC
From:
To:
Control: tags -1 - serious

Dropping severity from Serious as the the tests are now flagged as flaky, so will stop impacting other packages.

There is at least one other test[0] that is failing intermittently, but looking at how it failed, it might affect other tests. Yes, seeing the same behaviour elsewhere[1].

167s t/web/installer.t ......................................
167s ok 1 - started plack server ok
167s ok 2 - GET http://localhost:2335/__test_warnings
167s ok 3 - Got startup warning
167s ok 4 - GET http://localhost:2335
167s ok 5 - at installer
167s # Testing language change
167s ok 6 - change language to french
167s ok 7 - Content is like "(?^:Pour commencer)"
167s ok 8 - change language to english
167s ok 9 - Content is like "(?^:Getting started)"
167s # Walking through install screens setting defaults
167s ok 10
167s ok 11 - Content contains "DatabaseType"
167s ok 12 - found database MySQL
167s ok 13 - found database PostgreSQL
167s ok 14 - found database Oracle
167s ok 15 - found database SQLite
167s ok 16 - Content contains "DatabaseName"
167s ok 17 - Content contains "Connection succeeded"
167s ok 18
167s ok 19 - set root password
167s ok 20 - set admin email
167s ok 21 - set addresses
167s ok 22 - Content contains "database"
167s ok 23 - Content contains "/RT_SiteConfig.pm"
167s ok 24 - Content contains "Finish"
167s ok 25 - Content contains "Login"
167s DBD::SQLite::st execute failed: attempt to write a readonly database at /usr/share/perl5/DBIx/SearchBuilder/Handle.pm line 634.
... and then many many errors due to failed INSERTs ...

Cheers,
Andrew

[0] https://ci.debian.net/packages/r/request-tracker5/unstable/amd64/52549710/
[1] https://ci.debian.net/packages/r/request-tracker5/unstable/armel/52427660/