Hi,

since version 1.20.0, dpkg no longer creates /var/lib/dpkg/available
(see #647911).

But on Debian releases until and including buster, dpkg fails to
operate with e.g. `dpkg --set-selections`, if
/var/lib/dpkg/available doesn't exist:

| The following NEW packages will be installed:
| [...]
| debconf: delaying package configuration, since apt-utils is not installed
| dpkg: error: failed to open package info file '/var/lib/dpkg/available' for reading: No such file or directory

My workaround to work around this is to run:

  chroot "${TARGET}" /usr/lib/dpkg/methods/apt/update /var/lib/dpkg

As mmdebstrap seems to rely on dpkg of the host system, it would be
nice if mmdebstrap could ensure to create an environment where
apt/dpkg of the installed system doesn't stumble upon. :)

Thanks for mmdebstrap!

regards
-mika-

Hi,

Quoting Michael Prokop (2023-06-14 17:19:37)

I see that the bug talks about 1.20.0 but I've had code creating
/var/lib/dpkg/available in mmdebstrap for a while and determined that it wasn't
necessary anymore since dpkg 1.17.11. Did you do practical tests or are you
only basing your analysis on that bug?

I'm quite certain about the version requirement because I did tests using
debbisect and using the suggestion below I'm able to create chroots for all of
unstable since 2006-08-10.

this already exists as a hook. :)

Try running mmdebstrap with

Hi,

* Johannes Schauer Marin Rodrigues [Wed Jun 14, 2023 at 07:09:35PM +0200]:

This showed up in an actual environment, where we automatically
deploy systems on a daily base. The deployment of a buster based
system failed as soon as we upgraded the host system (a Grml based
live system) to bookworm, which provides dpkg v1.21.22 and
mmdebstrap v1.3.5-7. (Deploying bullseye where dpkg v1.20.12 exists
works fine, FTR)

mmdebstrap itself does *not* fail, but the installation of e.g. the
`nullmailer` package inside the target buster environment then fails:

| Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install nullmailer' returned 100: Reading package lists...
| Building dependency tree...
| Reading state information...
| The following packages were automatically installed and are no longer required:
|   libevent-2.1-6 libgnutls-dane0 libunbound8
| Use 'apt autoremove' to remove them.
| The following packages will be REMOVED:
|   exim4-base exim4-config exim4-daemon-light
| The following NEW packages will be installed:
|   nullmailer
| 0 upgraded, 1 newly installed, 3 to remove and 0 not upgraded.
| Need to get 109 kB of archives.
| After this operation, 3560 kB disk space will be freed.
| [...]
| [master dbda30d] saving uncommitted changes in /etc prior to apt run
|  6 files changed, 115 insertions(+), 575 deletions(-)
|  rewrite locale.gen (99%)
|  create mode 100644 needrestart/conf.d/ignore.conf
| debconf: delaying package configuration, since apt-utils is not installed
| dpkg: error: failed to open package info file '/var/lib/dpkg/available' for reading: No such file or directory
| Fetched 109 kB in 0s (542 kB/s)
| E: Sub-process dpkg --set-selections returned an error code (2)
| E: Couldn't record the approved state changes as dpkg selection states[0m

(FTR, this is part of an automated puppet run)

I can manually reproduce the problem with:

| # mmdebstrap buster /tmp/buster
| [...]
| # chroot /tmp/buster apt update
| [...]
| # chroot /tmp/buster apt install nullmailer
| [...]
| # chroot /tmp/buster apt purge nullmailer
| [...]
| perl: warning: Falling back to the standard locale ("C").
| locale: Cannot set LC_CTYPE to default locale: No such file or directory
| locale: Cannot set LC_MESSAGES to default locale: No such file or directory
| locale: Cannot set LC_ALL to default locale: No such file or directory
| dpkg: error: failed to open package info file '/var/lib/dpkg/available' for reading: No such file or directory
| E: Sub-process dpkg --set-selections returned an error code (2)
| E: Couldn't record the approved state changes as dpkg selection states

This indeed solves the problem, /var/lib/dpkg/available exists in
the target system then (as empty file, FTR).

This does *not* help/work, though, FYI.

(FTR, we already have a local workaround, I just thought that it
would make sense to have a decent and nice out-of-the-box experience
also for other users. :))

Thanks! :)

regards
-mika-

Hi!

So just to clarify, old and new dpkg in normal conditions works fine
for a long time when the available file is not present (since 1.17.11 as
you point out). The problem here as hinted at below is that apt tries to
set the package selections via «dpkg --set-selections» after having
refreshed the available file via «dpkg --merge-avail» where both these
commands fail if the available file is missing with versions earlier
than 1.20.0.

Yeah I mentioned this also to Mika, but also encouraged him to file
this report as I think it might make sense for these hooks to be
adapted to cover up to buster out of the box.

Initially I thought that perhaps only the variants that end up with
apt installed so that it can work correctly, but given that with older
dpkg versions some of these actions will always fail, then perhaps it
makes sense to still add those unconditionally so that users do not
need to create the file manually if they use the other dpkg actions.

(Hope this clarifies things! :)

Thanks,
Guillem

Hello,

Quoting Guillem Jover (2023-06-14 20:29:17)

aha, then I tested the wrong thing! I'm now able to verify the correct dpkg
version required for dpkg --set-selections without /var/lib/apt/available
using:

debbisect --depends=dpkg --cache=./cache 20200627T025456Z 20200627T085709Z '! echo | /usr/sbin/chroot "$1" dpkg --set-selections'
...
bisection finished successfully
  last good timestamp: 20200627T025456Z
  first bad timestamp: 20200627T085709Z
only one package differs: dpkg 1.19.7 -> 1.20.1

yes, thank you for finding and reporting this bug!

I'm unsure how to fix it... maybe by introducing a maybe-buster-or-older hook
which automatically pulls in the jessie-or-older hook?

Thanks!

cheers, josch