#1111856 compat-el: autopkgtest regression: dh_elpa_test: command not found

#1111856#5
Date:
2025-08-22 20:59:55 UTC
From:
To:
Dear maintainer(s),

Your package has an autopkgtest, great. However, it fails. Can you
please investigate the situation and fix it? I copied some of the output
at the bottom of this report.

The release team has announced [1] that failing autopkgtest on amd64 and
arm64 are considered RC in testing.

More information about this bug and the reason for filing it can be
found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation

Paul

[1] https://lists.debian.org/debian-devel-announce/2019/07/msg00002.html

https://ci.debian.net/packages/c/compat-el/testing/amd64/63554754/

  24s autopkgtest [20:50:45]: test dh-elpa-test-autopkgtest:
[-----------------------
  25s bash: line 1: dh_elpa_test: command not found
  25s autopkgtest [20:50:46]: test dh-elpa-test-autopkgtest:
-----------------------]

#1111856#10
Date:
2025-08-23 05:59:49 UTC
From:
To:
Paul Gevers <elbrus@debian.org> writes:
breaks/replaces/provides for built-in packages which includes compat.
Looking at the "test dh-elpa-test-autopkgtest: preparing testbed" step,
both dh-elpa and emacs were removed as a result, causing the
dh_elpa_test command not found.  This is working as intended.

So this is how this works now: when compat-el upgrades to a version
higher than the one provided by Emacs (e.g. 30.2.x.x), it can enter
Forky.  And if later a newer Emacs version provides that higher version
of compat, compat-el should be kept out of Forky again.

#1111856#15
Date:
2025-08-23 05:59:49 UTC
From:
To:
Paul Gevers <elbrus@debian.org> writes:
breaks/replaces/provides for built-in packages which includes compat.
Looking at the "test dh-elpa-test-autopkgtest: preparing testbed" step,
both dh-elpa and emacs were removed as a result, causing the
dh_elpa_test command not found.  This is working as intended.

So this is how this works now: when compat-el upgrades to a version
higher than the one provided by Emacs (e.g. 30.2.x.x), it can enter
Forky.  And if later a newer Emacs version provides that higher version
of compat, compat-el should be kept out of Forky again.

#1111856#20
Date:
2025-08-23 06:57:45 UTC
From:
To:
Hi,


I think it would then be good to ensure compat-el is (and similar other
packages are) tested for uploads of emacs, such that this issue is
spotted automatically and bugs will be filed at the right moment emacs
tries to migrate. See hint-testsuite-triggers in the autopkgtest spec [1].

Paul

[1]
https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst

#1111856#25
Date:
2025-08-23 07:56:42 UTC
From:
To:
Paul Gevers <elbrus@debian.org> writes:

compat-el does have autopkgtest enabled[1], but there is no direct
dependency on emacs, which I suppose is why it didn't block emacs from
migrating.

From the user point of view, the installation of uninstallation of
elpa-compat is done automatically, so it's not causing any user facing
issues.

[1] https://salsa.debian.org/emacsen-team/compat-el/-/blob/master/debian/control?ref_type=heads#L11

#1111856#30
Date:
2025-08-23 08:09:36 UTC
From:
To:
Hi Xiyue


Sure, I'm well aware of that (as this bug is about failing autopkgtest).
I referenced hint-testsuite-triggers to *add* a pseudo direct dependency
on emacs, such that the machinery triggers the test.


Sure, maybe not user facing, but without this the automated machinery
doesn't do what we want it to do, with this we're informed and warned.

Paul

#1111856#35
Date:
2025-08-23 08:30:24 UTC
From:
To:
Paul Gevers <elbrus@debian.org> writes:

Ah apparently I misread this part.  Is there a way to trigger
hint-testsuite-triggers with autopkgtest-pkg-elpa or it needs to be used
in debian/tests/control?

#1111856#40
Date:
2025-08-23 08:44:34 UTC
From:
To:
Hi,


The hint-testsuite-triggers stanze in d/t/control is there such that
dpkg adds these pseudo test dependencies to the Testsuite-Triggers field
of the source in question. I assume autopkgtest-pkg-elpa isn't involved
during source building. In that case, it should go into d/t/control. I'm
thinking, you *might* have tooling in the lisp eco system that could
generate the required Testsuite-Triggers field and not rely on dpkg, but
that seems quite a bit more involved and duplication of logic that's
already there.

Paul

#1111856#45
Date:
2025-08-23 10:35:41 UTC
From:
To:
Hi Paul,

Paul Gevers <elbrus@debian.org> writes:

Thanks for the explanation!  Looks like to use hint-testsuite-triggers
it needs to convert from autopkgtest-pkg-elpa to a real d/t/control
setting.  Currently there is no newer compat-el release that is higher
than the one provided by Emacs so we cannot test it either.  So this
will need to wait for a new upstream version to be able to test.

On the other hand, as there is no real user facing issues, I wonder
whether it is desirable to block Emacs migration due to this.  As a
matter of fact, there are other packages in the same situation
(e.g. use-package, org, etc.), and I wonder whether all of them should
block the migration of Emacs.  While admittedly this is good to fix, I
wonder whether relying on some periodic package rebuild service
(automatically or manually) to find such issue and file RC bug for
removing from testing is an acceptable compromise?

And of course, there is also the option to just remove the package.  In
the case of compat-el, this is usually used for providing functions in
newer Emacs to be available for use in older Emacs releases, so its
version will next be higher than the version of Emacs.  If we keep Emacs
up-to-date in Debian, compat-el will be less useful for Debian user, so
this sounds like a good reason for its removal.

#1111856#50
Date:
2025-08-23 11:30:26 UTC
From:
To:
Hi,


That's exactly what I'm doing here, which is of course why I rather want
it solved. I'll not remember this case in a year from now, and I'd be
spending time on the investigation. Having it tied to a fresh upload of
emacs will make that easier.

Paul

#1111856#55
Date:
2025-08-23 22:26:00 UTC
From:
To:
Hi Paul,

Paul Gevers <elbrus@debian.org> writes:

The status quo means extra work from you, and I understand that you'd
like this to be fixed.  On the other hand, as I mentioned in my previous
email, blocking Emacs migration due to this means that the Emacsen team
will probably need to file 3-4 RM bugs for packages like this to let
Emacs migrate each time this happens, which is also some extra efforts
that IMHO not well spent as there is no user facing issues.

As a result, IMHO the best action to deal with this situation that saves
the most energy from everyone is probably to just remove compat-el (or
just keeping this bug open as-is).  I'm CCing spwhitton and bremner to
hear their thoughts (FTR, my reason for removing compat-el can be found
in the last paragraph at [1]).

P.S. I think compat-el is a particular package that it's more useful for
older versions of Emacs when backported, e.g. Emacs 28.2 in Bookworm
would benefit from the utilities it provides.  However, IIUC, the
backporting policy requires that the package exists in testing to be a
candidate for backporting to stable, and this won't happen due to this
bug, so it's still moot.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1111856#45

#1111856#60
Date:
2025-08-28 08:14:40 UTC
From:
To:
Hello,

ISTM that doing the extra work to file those bugs is okay, and better
than potentially having to go through NEW again.  I.e., it doesn't seem
to me that RMing compat-el would be a good idea.

#1111856#65
Date:
2025-08-29 01:07:00 UTC
From:
To:
Sean Whitton <spwhitton@spwhitton.name> writes:

Specifically for compat-el, I don't think it's useful in Debian anymore.
In upstream Emacs, its `package--builtin-versions' always contains a
dummy entry of compat that is strictly higher than its own version (see
code point at [1]), e.g. for Emacs 30.1, it has `(30 1 9999)' as compat
version.  And for compat, it will never have a version higher than the
latest released version of Emacs (which is what compat for, provide
utility from newer Emacs versions to older Emacs versions).  This means
that if we ensure that the latest Emacs is packaged in Debian, compat-el
will never be installable with the breaks/replaces/provides generated
for built-in packages,

Well, strictly speaking (as I mentioned in a previous email), compat is
more useful when backported, e.g. for Emacs 28.2 in bookworm.  But as it
cannot enter testing because of this bug, this is also moot.

[1] https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/compat.el?h=emacs-30.2#n83

#1111856#70
Date:
2025-08-29 10:31:51 UTC
From:
To:
Hello,

This makes sense to me, but do we know why it was ever useful?

I guess it's only useful when the version of Emacs in Debian is a major
version behind upstream?  Which is generally only a short lived
situation?

#1111856#75
Date:
2025-08-29 22:08:22 UTC
From:
To:
Hi Sean,

Sean Whitton <spwhitton@spwhitton.name> writes:

It was useful before we had the breaks/replaces/provides for built-in
packages: when a package explicitly specifies `compat' in
`Package-Requires', dh-elpa will generate an explicit dependency on
elpa-compat.  Well, the explicit dependency is still there, but now we
have Emacs providing that so it's not needed anymore.

Mostly yes.  Maybe also for minor revision, but now we are at Emacs 30.2
but compat is still at 30.1.0.1.

But I agree removing compat-el too soon may cause unnecessary work,
e.g. NEW queue, etc.  I guess maybe we can just keep this bug open and
keep compat-el out of Forky for now, and see how it will progress with
future Emacs releases.

#1111856#80
Date:
2025-08-30 05:11:46 UTC
From:
To:
Xiyue Deng <manphiz@gmail.com> writes:

Apparently I missed the fact that removing compat-el from testing will
also remove all its reverse dependencies, which is already happening[1].

I think one easy way to solve this is to make compat-el a dummy package
that depends on Emacs.  This is slightly more than a workaround as it's
function is indeed fulfilled by Emacs already.  The same can be done to
other packages that has lower versions than provided by Emacs.

An alternative is to let dh-elpa generate the dependencies on compat
(and other packages) that are provided by Emacs to have emacs as an
alternative.  This has the downside that all emacs packages need to be
rebuilt and IIRC we don't have binNMU for arch:all packages, and it can
be tricky to handle version bumps, e.g. when the required package
version is higher than the one provided by Emacs, and when an updated
Emacs provides a package higher than required version.

I'd prefer the first easier solution.  Would like to hear the opinions
from others.

[1] https://tracker.debian.org/pkg/compat-el

#1111856#85
Date:
2025-08-30 05:44:18 UTC
From:
To:
Hi,


emacs-common Provides compat-el (versioned), so I my opinion this
removal (thread?) is a bug in the autoremoval tooling. (Versioned)
Provides is relatively new in Debian, not all tools deal with it in all
corner cases. Either somebody finds a fix for the autoremoval tooling
or, if we let the removal happen, I expect all dependent packages to
migrate straight back after removal.

Paul

#1111856#90
Date:
2025-08-30 05:59:04 UTC
From:
To:
Hi Paul,

Paul Gevers <elbrus@debian.org> writes:

Thanks for the insights!  I hope that can work one way or another.

#1111856#95
Date:
2025-08-30 10:01:37 UTC
From:
To:
Hello,

Then I think we should remove it.

#1111856#100
Date:
2025-09-01 11:09:06 UTC
From:
To:
Hello everyone,

Le vendredi 29 août 2025 à 11:31, Sean Whitton
<spwhitton@spwhitton.name> a écrit :

Some upstream maintainers started pulling it as a dependency, in
order to not have to condition their code to emacs versions, while
also using the latest functions provided by the most recent emacs.
This library made sense to them insofar as it allowed them not to
have to update their code down the line.

Since the upstream maintainers of some third-party elisp libraries
started using it as a dependency, we had to package it to package
those libraries as well.

Now, since a stub version of compat is included in emacs since
version 30, compat is only going to be useful to us if :
- a new function appears in the 31 branch.
- AND the maintainer of compat decides to implement it, and
  release some version of compat 31.x.x.x.
- AND some third-party elisp maintainer (of a package we package
  in debian) decides to pull this compat 31.x.x.x version as
  dependency in order to use this brand new function.

If this happens and we have removed compat, we will have to wait
for emacs 31 to be released (and packaged by us) to be able to
package this new version of the third party elisp library.

I am not sure how likely this situation is, and whether this is
really a problem : we could perfectly decide to delay packaging a
new version of a third-party elisp library until after the needed
emacs version is released by upstream and packaged.

Best,

Aymeric

#1111856#105
Date:
2025-09-02 08:31:56 UTC
From:
To:
Hello,

Thanks for the analysis.  I don't think the compat-el maintainers would
ever do that, so I still think we should remove compat-el.

#1111856#110
Date:
2025-10-09 04:16:57 UTC
From:
To:
Xiyue Deng <manphiz@gmail.com> writes:

So as Paul predicted, the reverse dependencies of compat-el got removed
from testing around Oct 6 and then re-entered around Oct 7, and since
then no remove notice is showing anymore.  Thanks!

#1111856#117
Date:
2026-05-12 04:56:16 UTC
From:
To:
A new upstream version of compat-el is available tracking the unreleased
Emacs 31 branch. So for the time being compat-el is installable again,
before Emacs 31 is released.

Paul Gevers <elbrus@debian.org> writes: