Whenever I upload a package, I use my build automation tool, vectis[1] to
build it in sbuild and run autopkgtests, piuparts and lintian, because I'm
doing my best to avoid uploading something that has serious regressions.

In previous releases this took about 2h35, which is already inconveniently
long. For the 5.26 release, it took 5h31, which is just unsustainable - I
can't do releases if they're going to take most of a day per attempt.

A breakdown of the time to run autopkgtest's autopkgtests in a qemu VM,
excluding time taken to install dependencies:

- autopkgtest (NullRunner, ChrootRunner, etc.): 3 minutes
- installed (a smoke-test): < 1 minute
- docker: 32 minutes
- lxc: 16 minutes
- lxd: 45 minutes
- podman: 41 minutes
- podman-init: 20 minutes
- schroot: 96 minutes (!)
- unshare: 9 minutes

I don't know why schroot was so painfully slow. I already made changes
during this release cycle to try to speed it up.

I want to see whether we can run the unshare test (currently marked with
isolation-machine) in lxc containers like the ones used by debci. If we
can, then I'm inclined to use UnshareRunner for all the test coverage
that can't be done by NullRunner or ChrootRunner, use lxc or podman-init
for all the test coverage that needs an init system, and limit docker,
lxd, podman and schroot to a relatively simple smoke-test (plus targeted
tests for things that are known to have regressed for those backends).

    smcv

[1] https://salsa.debian.org/smcv/vectis

Updated for 5.44, on similar hardware although perhaps not entirely
apples-to-apples:

4 minutes

< 1 minute

20 minutes

53 minutes (!)

18 minutes

20 minutes

13 minutes

7 minutes (schroot used to be notably slow, but it seems that using
eatmydata throughout has made a huge difference to this one)

There might be some randomness here: I don't know why lxc was so much
slower now than before, while most of the others were significantly faster.

I also don't understand why podman is still slower than podman-init, when
podman-init does more work (podman just runs sleep(1) or a minimal reaper
or something as pid 1, similar to docker, whereas podman-init runs a full
operating system from systemd upwards).

Paride's work on applying eatmydata to schroot seems to have been a
huge win. Can someone perhaps give lxc a similar treatment?

Merge requests very welcome for anything moving in this direction.

    smcv

Currently the slow tests are "lxc" and "lxc-old-testbed". Many of those
tests do not really need a container: a chroot-like environment would be
enough, so that running a test does not need to wait for boot to
complete and similar. We could/should move many of those under
UnshareRunner.