Dear Maintainer,
The Cassandane integration tests run by debian/tests do not test this
cyrus-imapd package, and have not for some years. I would like to rewrite
that setup, and I am filing this first so the approach can be agreed before
I send a merge request.
## The suite is fetched from an archived repository
debian/tests clones https://github.com/cyrusimap/cassandane.git. Its last
commit was on 2021-11-22, and the repo was archived on 2024-12-10. Its
description reads "DEFUNCT: The test suite for Cyrus IMAP, now merged into
Cyrus's main repository". _cassandane() in debian/tests/docker/functions.sh:94
checks out origin/master from that frozen tree.
The upstream-current Cassandane suite ships in our own source package: 1718
files under cassandane/, including Makefile, testrunner.pl and
cassandane.ini.example. However, nothing in debian/ uses it.
## The test never reaches Cassandane at all
In today's (2026-08-25) run on testing/amd64:
unit-test PASS
cyrus-no-docker SKIP exit status 77 and marked as skippable
https://ci.debian.net/data/autopkgtest/testing/amd64/c/cyrus-imapd/74660819/log.gz
All eight git clones and all twenty cpan builds succeed. But then, at t=1000s:
+ cd /srv/dovecot.git
+ git fetch
+ git checkout -q 6264b51bcce8ae98efdcda3e55a765d7a13d15ed
error: The following untracked working tree files would be overwritten by
checkout:
src/lib-fts
[... repeated ...]
Please move or remove them before you switch branches.
Aborting
The cause is a collision between two parts of install-env.sh. Line 11
clones Dovecot's main branch. Lines 14-16 then copy Debian's packaged
Unicode data into that clone:
cp /usr/share/unicode/auxiliary/WordBreakProperty.txt dovecot.git/src/lib-fts
cp /usr/share/unicode/PropList.txt dovecot.git/src/lib-fts
Current Dovecot has no src/lib-fts directory, so those cp calls create
a plain file at that path. Line 63 then checks out commit 6264b51b,
dated 2016-11-04, where src/lib-fts is a directory of 34 files. And each
one of them gets blocked by the untracked file just created. The result
is that git aborts, and eventually this all gets skipped.
So, run.sh never executes and Cassandane never starts. The ancient
Dovecot is not merely outdated; it is never built. That is a good thirteen
minutes of building lost (on each architecture).
The Unicode copy was added on purpose: The comment reads "Use packaged
UnicodeData.txt as it often 503's when downloading on arm64 armhf in
Ubuntu's autopkgtest infrastructure". So, a deliberate workaround is
now what breaks the test, because upstream Dovecot changed.
Besides, there is another config defect: cyrus-no-docker:7 sets
CASSANDANEOPTS="Cyrus::ImapTest", and the configured path does not match
where install-env.sh puts it:
install-env.sh:21 clones to /srv/imaptest.git
cassandane.ini.dockertests [imaptest] basedir = /srv/imaptest
functions.sh:107 copies that ini into place. ImapTest.pm:62 does
"$basedir = abs_path($basedir)", which yields undef for a path that does
not exist, so list_tests takes the "!defined $basedir" branch and returns
a single test_warning_imaptest_is_not_installed. Even with the checkout
repaired, the selected suite would report nothing.
## Consequence
Tests that upstream added since 2021 are never run; and tests that
upstream has since changed, run in their 2021 form. Concretely:
The nine CVE fixes in 3.12.3 (CVE-2026-47081 through CVE-2026-47089,
#1142925) each ship an upstream Cassandane regression test in
cassandane/, and none of them can be reached by the current setup.
## Why this needs a rewrite rather than a patch
The obvious minimal change (point _cassandane() at the in-tree cassandane/
instead of the clone) is not sufficient:
* cassandane/Makefile's default target runs "perl -c" over every .pm and
.pl in the tree, around 160 files. That fails unless every CPAN
dependency is installed, including seven that Debian does not package.
Only "make -C utils", which builds the C helpers testrunner.pl checks
for, is wanted.
* CASSANDANEOPTS=Cyrus::ImapTest would still need the imaptest binary, so
install-env.sh's clones and cpan invocations would still be required and
needs-internet could not be dropped.
* testrunner.pl requires cwd == cassandane/, and writes reports/ and
utils/*.o into it, so running in place dirties the source tree.
* debian/tests/no-docker/{entrypoint,functions,testscript}.sh are
symlinks into debian/tests/docker/, so the two paths share their logic
and cannot be changed independently. Those scripts are a November 2019
fork of upstream's cyrus-docker, which upstream has since rewritten;
they cannot be resynced piecemeal.
## What I propose
The goal is to drop needs-internet and skippable; needs-root has to stay.
One self-contained autopkgtest that runs the in-tree suite against the
installed packages:
1. Copy cassandane/ to a scratch directory so the source tree stays clean
and UNIX socket paths stay short;
2. "make -C utils";
3. Create the cyrus user;
4. Generate a cassandane.ini with
[cyrus default] prefix = /usr/lib
;
5. Run ./testrunner.pl. Suites would be selected by checking which ones
compile, so coverage grows by itself as missing dependencies get
packaged, rather than from a hardcoded list that goes stale.
The dependency list would be kept in step with cassandane/doc/README.deps,
which is upstream's own list, ships in our tarball, and already names
Debian packages.
Running every suite additionally needs seven Perl modules packaged, as well
as one C programme (imaptest), built against Dovecot:
- Net::DAVTalk
- Net::CalDAVTalk
- Net::CardDAVTalk
- Mail::JMAPTalk
- Text::VCardFast
- XML::Spice
- Convert::Base64
- imaptest
All are DFSG-free and I am willing to file the ITPs. Without it, the
pure-IMAP suites still run, which already includes all nine CVE regression
tests above. However, I would like to get the JMAP tests running, too, and
anything upstream thinks important.
debian/tests/cyrus-no-docker, cyrus-docker and the docker/ and no-docker/
directories would all go. (Note that debian/tests/cyrus-docker has no Tests:
entry in debian/tests/control, so it does not run under autopkgtest at all.)
## Reference: Fedora
Fedora runs Cassandane in their cyrus-imapd package and has solved most of
the problems this raises, including one that affects Debian specifically:
Cassandane disables its syslog replacement when Cyrus is built with
_FORTIFY_SOURCE, which Debian always is. I intend to model the Debian test
on theirs and to credit it explicitly, in d/changelog, in comments in the
test script, and in DEP-3 Origin:/Author: headers on any patch adapted from
theirs. The relevant Fedora work is by Martin Osvald <mosvald@redhat.com>,
Jason L Tibbitts III <tibbs@math.uh.edu> and Petr Pisar <ppisar@redhat.com>.
I am happy to prepare a merge request along these lines. This might take
some time, and I will run tests to make sure it is all working before filing
an MR.
Please let me know your thoughts on this proposal.
Kind regards,
Edmund Lodewijks