I have modified and adapted the integration tests from dh-make-elpa to
work in dh-elpa. The tests work in a similar fashion as dh-make-elpa: in
a template repo that provides a minimal-ish set of files required for
dh_elpa command, run the command, and compare the output with an
expected repository, and check whether there are any diff. I hope this
provides a way to easily verify changes to the dh_elpa (and dh-elpa.el)
utility, especially debian/*.substvars to check whether the generated
"misc:Depends" and "elpa:Depends" are expected.

During implementation, I found some limitation of how the "dh_elpa"
command works and used some workarounds to make it work.

* The "dh_elpa" perl script uses global path that assumes a installed
  dh-elpa is available. However, the build time test needs to run from
  the source tree. As a result I extended the dh_elpa script to allow
  referencing files from the source tree (as well as other tweaks) to
  make it work during build time. The changes are guarded by a
  "--test-source-root" parameter.

* For multiple file packages, dh_elpa will create a tarball which cannot
  be compared with "diff" directly (which bremner had warned me before).
  For the time being, I just skip the tarball files as the unpacked
  directory is also available in the generated templates so their
  contents are still part of the comparisons.

I hope the diff tests can give more confidence when modifying dh_elpa,
especially for testing ideas like in Bug#1138884.

The changes are in an "integration-test" branch[1], and the diff against
master branch can be seen here[2]. An overview of how the tests work can
be found at the README.org[3].

Reviews and suggestions welcome!

[1] https://salsa.debian.org/emacsen-team/dh-elpa/-/tree/integration-test?ref_type=heads
[2] https://salsa.debian.org/emacsen-team/dh-elpa/-/compare/master...integration-test?from_project_id=18920
[3] https://salsa.debian.org/emacsen-team/dh-elpa/-/blob/integration-test/t/README.org?ref_type=heads

Xiyue Deng [08/Jun  6:36pm -07] wrote:

I think you should try to cut down the Perl deps.  Most of those aren't
really necessary, I used them for the dh-make-elpa test suite because I
didn't know much Perl at the time.

Hi Sean,

Sean Whitton <spwhitton@spwhitton.name> writes:

Makes sense. I have reduced the lists of Build-Depends and Depends.
PTAL.

Xiyue Deng <manphiz@gmail.com> writes:

Friendly ping.

Hello,

If David would like to review and merge this I won't object.  At the
present time however I'm sceptical as to whether integration tests of
this form will help us all that much with dh_elpa bugs.

Hi Sean,

Sean Whitton <spwhitton@spwhitton.name> writes:

Please allow me to try to show the usefulness of the tests using the
filtering change as an example. With the proposed integration tests, we
can be sure that the change that removes the built-in addon filtering is
actually working by looking at the diff of the output files that the
built-in addons are showing up in generated files after the change, e.g.
"elpa-cl-lib (>= 0.1)". And it also helps us notice potential
regressions caused by future changes if we see them getting removed
accidentally.

Xiyue Deng <manphiz@debian.org> writes:
2.2.0 release, and you can be sure that dh-elpa stops filtering built-in
addon info correctly.

[1] https://salsa.debian.org/emacsen-team/dh-elpa/-/commit/2fd65da7adc51e6baba14b507380a6dc096b4f53