Hi,

since the upload of debconf 1.5.88 a few days ago, building chrootless Debian
chroot tarballs is broken if not using fakeroot. Specifically, since that
upload, /var/cache/debconf/ and /var/cache/debconf/tmp.ci/ outside the chroot
will get modified or created, respectively. I tried to come up with a patch but
am left with more questions than answers and am thus reaching out to the
debconf maintainers as well as the readers of the deity list. There is a bit to
unpack, lets start with a patch that works around the issue in our CI [1]:
--- a/dpkg-preconfigure
+++ b/dpkg-preconfigure
@@ -140,7 +140,10 @@ elsif (! @debs) {
 }

 my $apt_extracttemplates;
-if (-x '/usr/lib/apt/apt-extracttemplates') {
+if (($ENV{USER} // '') ne 'user') {
+	warn gettext("delaying package configuration, since apt-extracttemplates does not support DPKG_ROOT");
+	exit;
+} elsif (-x '/usr/lib/apt/apt-extracttemplates') {
 	$apt_extracttemplates = '/usr/lib/apt/apt-extracttemplates';
 } elsif (Debconf::Path::find('apt-extracttemplates')) {
 	$apt_extracttemplates = 'apt-extracttemplates';
EOF

Before debconf 1.5.88, dpkg-preconfigure did not find apt-extracttemplates
(because apt-utils was not installed) and thus bailed out early. But since apt
2.9.11, apt-extracttemplates is part of the "apt" package and thus always
available. Thus, dpkg-preconfigure does not bail out and ends up touching files
that it should not.  Notably, this only happens when running mmdebstrap in
chrootless mode *without* fakeroot around it. With fakeroot we are lacking
privileges and we get:

debconf: DbDriver "passwords" warning: could not open /var/cache/debconf/passwords.dat: Permission denied

Which means once again, dpkg-preconfigure bails out early (but for a different
reason this time) and nothing outside of the chroot gets modified. This is also
part of the reason why we check the USER environment variable in the workaround
patch above. It ensures that when fakeroot is *not* used, that is, if it is not
the CI user named "user" that is running dpkg-preconfigure, we bail out early.

But why this clunky check? Other patches to enable chrootless support just
check for the DPKG_ROOT environment variable. But that variable is not set for
the dpkg-preconfigure script. The DPKG_ROOT environment variable is set for
maintainer scripts when apt is started by mmdebstrap with these options:

    -oDPkg::Chroot-Directory=
    -oDPkg::Options::=--force-not-root
    -oDPkg::Options::=--force-script-chrootless
    -oDPkg::Options::=--root=...

But dpkg-preconfigure is called by /etc/apt/apt.conf.d/70debconf via
DPkg::Pre-Install-Pkgs and thus does *not* have the DPKG_ROOT variable set as
it is not executed by dpkg but by apt itself. We are thus not able to figure
out that this is supposed to be a chroot installation.

I'm am looking for ideas of how to fix this.

Since chroot installation worked well before apt-extracttemplates moved to
"apt" I'm tending to look for a solution which just exits dpkg-preconfigure
early for chrootless installations. But on what condition should this happen?

This is made worse by the fact, that apt is not being told that this is a
chrootless installation -- dpkg is being told via apt options.

Should apt gain support for being told that it's doing a chrootless
installation and then pass the right options or environment variables to the
DPkg::Pre-Install-Pkgs scripts it calls?

Should the caller of apt set an environment variable or touch a special file to
indicate to dpkg-preconfigure that it should please exit early?

Dear debconf & apt developers, do you have ideas?

Thanks!

cheers, josch

[1] https://salsa.debian.org/helmutg/dpkg-root-demo/-/merge_requests/2

Eww.  What a mess.

dpkg-preconfigure arguably shouldn't need to bail out; since debconf
supports DPKG_ROOT, it would be sufficient if DPKG_ROOT were set when
dpkg-preconfigure is called, since that would mean it would use a
debconf database in the right place.  (Well, nearly.  dpkg-preconfigure
currently hardcodes /var/cache/debconf/tmp.ci, but that's easily fixed.)
In general it seems better to me if chrootless installations take more
similar code paths to normal ones.

How viable/evil would it be for something outside dpkg to set DPKG_ROOT?
Perhaps the caller of apt could just set DPKG_ROOT, since it knows the
intended installation path and that the installation is going to be
chrootless?  Then hopefully all we need is something like:

diff --git a/dpkg-preconfigure b/dpkg-preconfigure
index 6239603a..a5f9d37d 100755
--- a/dpkg-preconfigure
+++ b/dpkg-preconfigure
@@ -156,7 +156,8 @@ if (! $have_tty && $frontend->need_tty) {
 	exit 0;
 }

Hi,

Thanks for noticing!

This is a corner of Debian I'm not experienced in, so I am only catching
up now and writing down my limited understanding.

Contrary to its name, dpkg-preconfigure is not part of dpkg, but
debconf. It is run by apt via DPkg::Pre-Install-Pkgs before any package
unpacking and extracts and runs the *.config scripts contained in
control.tar. The benefit here being that the installation does not
scatter debconf prompts to various maintainer scripts and asking (most
of) them ahead of installation in one batch.

(Please now point out how wrong I am.)

Now mmdebstrap goes to great lengths to avoid debconf questions by
setting DEBIAN_FRONTEND=noninteractive and
DEBCONF_NONINTERACTIVE_SEEN=true. It really does not want to answer
those questions interactively. If you really want to answer them, man
mmdebstrap suggests running debconf-set-selections from an
--essential-hook. So really, what is the benefit of running any of these
when run from mmdebstrap? We want the answers to be deterministic in the
interest of reproducibility. Unless I am mistaken, we also want the
.config scripts to not have any effect when being non-interactive as you
cannot rely on them being run (e.g. when installing a package with dpkg
directly).

I question why we run dpkg-preconfigure from mmdebstrap. In what
situation can it have any beneficial effect?

(Not running it also speeds stuff up.)

debconf was required to support DPKG_ROOT, because it is run from
maintainer scripts. I see way less reason to require dpkg-preconfigure
to support this. The DPKG_ROOT variable is meant to be exported by dpkg
for maintainer scripts, but dpkg-preconfigure is not run from a
maintainer script, so the attached semantics do not apply. If we want it
to support on a directory, I argue that it should gain a --root option
instead. I'm not convinced that we should change it at all given the
above though.

I think it would be evil. Presently, we kinda imply that if DPKG_ROOT is
set, we're inside a maintainer script and that property gets violated
here. That's certainly debatable, but it also is a significant departure
from the present design.

Given the above I'd rank things like this:

1. Do not call dpkg-preconfigure from mmdebstrap.
2. Add a --root option to dpkg-preconfigure.
3. Your patch.

This is assuming that my limited understanding is lacking errors. Now
please tell me where I'm wrong.

Helmut

Am Thu, Dec 26, 2024 at 12:41:42PM +0100, schrieb Johannes Schauer Marin Rodrigues:
elsewhere with an assortment of options for sources.list, config,
dpkg.status, lists/, archives/, … so adding yet another option can't
hurt that much, can it?

That said, semantics might be tricky…

I am assuming you use the "chroot" apt config, but the various hooks
tend to use absolute paths ~ in other words paths on the host.
Many of those hook interfaces are invoked via $SHELL and I am certainly
not looking forward to implementing a $SHELL parser so we could mangle
the absolute paths into the "chroot" ……… sometimes?

Random example from my host:
| $ apt-config dump --no-empty | grep -i '::\(pre\|post\)-'
| APT::Update::Post-Invoke-Success:: "if /usr/bin/test -w /var/cache/swcatalog -a -e /usr/bin/appstreamcli; then appstreamcli refresh --source=os > /dev/null || true; fi";
| DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -lt 10";
| DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
| DPkg::post-invoke:: "[ -x /usr/lib/libdvd-pkg/b-i_libdvdcss.sh ] && /usr/lib/libdvd-pkg/b-i_libdvdcss.sh || true";

So, in a sense, I think you are just "lucky" that you ran into this
problem class with dpkg-preconfigure/apt-extracttemplates as depending
on what you install in your chrootless chroots other fun stuff might
happen.
(And that "ignoring" the preconfigure has no consequences for you as the
 config intermixed with the installation is silenced anyhow)

How is this "solved" for maintainer scripts? I mean the ones which don't
use just sed and grep which you can easily provide and use from the host
but lets say texlive or the libdvdcss from above?


The most immediate option here might just be to have a
| $ cat /path/to/chroot/etc/apt/apt.conf.d/99z-disable-hooks.conf
| #clear APT::Update::Auth-Failure;
| #clear APT::Update::Pre-Invoke;
| #clear APT::Update::Post-Invoke;
| #clear APT::Update::Post-Invoke-Stats;
| #clear APT::Update::Post-Invoke-Success;
| #clear APT::Install::Pre-Invoke;
| #clear APT::Install::Post-Invoke-Success;
| #clear DPkg::Pre-Install-Pkgs;
| #clear DPkg::Pre-Invoke;
| #clear DPkg::Post-Invoke;

Those are at least the ones I found with `git grep RunScripts`,
I kinda suspect more to exist… who knows, right? 😉

It /might/ makes sense to deprecate all the various hooks and do
something slightly more controlled aka declarative at least on the
"what on earth are we supposed to run" part of the interface.
I suppose Julian is screaming JSON hook or varlink at me now,
but that might be yet another beast to deal with.


Another more specific option might be to "just" react in
dpkg-preconfigure to DEBIAN_FRONTEND=noninteractive and exit as
I suppose the point of preconfiguring is to front load interactivity,
but if there will be none no extractions have to happen.

(obligatory mention of it being sad that this functionality is in apt to
 begin with as this is not something apt should be doing. It doesn't
 usually poke into deb files…)

A very fine time sink you stumbled over.
Package management is clearly a hard problem, not just for Santa.


Best regards

David Kalnischkies

Hi David,

Quoting David Kalnischkies (2024-12-27 18:48:20)

depends on what makes you hurt I guess. I think it mostly is a matter of how we
design the interface. We may get away without adding another config option to
apt. If mmdebstrap is the only user of such an option I also don't think it
makes much sense in practice, no?

Yes, that would not be a good way forward.

Yes, chrootless installations should only be done in another chroot layer
anyways, which limits the damage. Our CI system is running mmdebstrap in
chrootless mode inside a normal mmdebstrap. This is not only necessary to
prevent damage to the host system but also because the system on the outside
should have the same packages and versions installed as the chrootless system
because we have no way to express maintainer script dependencies by themselves
(yet).

Did you read Helmut's reply to this bug? I'd like to hear your thoughts about
his idea. If we do not need dpkg-preconfigure anyways, because we do not ask
any debconf questions in the first place, then this whole problem becomes
completely moot.

Maintainer scripts have to be manually touched to add support for DPKG_ROOT.
Right now, only Essential:yes, build-essential and init support it. So you can,
for example create a bootable Hurd installation on Linux even though you cannot
run any Hurd binaries on Linux (even QEMU would be of no help here) but to
install packages beyond what is needed to boot a shell you would do that on the
real system. It's also not very interesting to add support for DPKG_ROOT to
many more packages because this is mostly interesting for bootstrapping
purposes.

Thank you, I had forgotten about the #clear directive.

Unless apt changes the way it calls these scripts, for example by adding an
environment variable, I think I should just disable all of these in chrootless
mode.

This is only tangential to this bug. I think what should be done is what Helmut
suggests and that is that mmdebstrap unconditionally disables
DPkg::Pre-Install-Pkgs for all installations because it sets
DEBIAN_FRONTEND=noninteractive unconditionally as well so there is no point in
running dpkg-preconfigure, right?

Whether or not dpkg-reconfigure should be doing something about the presence of
DEBIAN_FRONTEND=noninteractive can be handled individually from that.

(obligatory mention that dpkg-reconfigure is called dpkg-* even though it is
not in dpkg :D)

I blame Helmut for having had the idea of chrootless package installations and
now I have a new hobby. :)

Thanks!

cheers, josch

Am Sat, Dec 28, 2024 at 12:01:57PM +0100, schrieb Johannes Schauer Marin Rodrigues:

My point was that packages you install in your chrootless chroot can
contain apt snippets that effect later apt calls including registering
various hooks (the same could happen to dpkg if it supports picking up
config from a different place some day). debconf just happens to be
installed by default and hence brings in the hook by default and debconf
happens also to be installed on the host. That might change with the
other examples if you install them in your chroot.


Other examples of packages modifying also other apt settings might be
apt-file or appstream and its icon-packages that lead to additional
downloads. Is the idea here to play wack-a-mole as specific problems
arise or figure out a "generic" approach?

IF chrootless goal is just to support a very tiny controlled subset
the solution might be different to e.g. reaching prio:standard and
upward (apt-listchanges is prio:standard).
a special snowflake in this regard I would see if we could solve that
issue more centrally for everyone: mmdebstrap isn't the only thing in
existence that runs 'noninteractive' to avoid debconf questions. Very
many things do and even if the hook script doesn't fail for them, it at
least wastes time for them all now that you can't avoid it by just not
installing apt-utils early on. '#clear' works for you now because it is
the only hook in your package selection. If that selection is larger
there are more hooks and some might not be as ignorable.

So, in my view this isn't tangential, but my most preferred solution:
make "dpkg-preconfigure --apt" a conditional noop (if possible) as
that requires no changes elsewhere and solves a problem many might not
have realised yet they have – and all thanks to some strange folks with
fringe use cases nobody cares about. Hurray for fringe use cases! 😉

In a way, "solving" this problem class "also" with '#clear' just for
mmdebstrap is the tangential to this bug for me…


Best regards

David Kalnischkies

P.S.: mmdebstrap is not even the only thing doing a chrootless operation…
apt does it itself in its tests, even before dpkg supported it by
preloading a library that noops chroot() and rewrites execvp() calls.
We don't use 'normal' deb packages through but craft our own to test
e.g. behaviour around failing maintainer scripts. That was the reason
for e.g. talking to myself for years in #824774.

Hi Colin,

Quoting Helmut Grohne (2024-12-26 20:27:29)

you didn't reply anymore to Helmut's mail so I assume that this means that you
do not disagree with his analysis?

In that case I'm reassigning this bug to mmdebstrap and will fix it there.

Thanks!

cheers, josch

We believe that the bug you reported is fixed in the latest version of
mmdebstrap, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1091442@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Johannes Schauer Marin Rodrigues <josch@debian.org> (supplier of updated mmdebstrap package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Fri, 10 Jan 2025 00:40:43 +0100
Source: mmdebstrap
Architecture: source
Version: 1.5.5-1
Distribution: unstable
Urgency: medium
Maintainer: Johannes Schauer Marin Rodrigues <josch@debian.org>
Changed-By: Johannes Schauer Marin Rodrigues <josch@debian.org>
Closes: 1077600 1091442
Changes:
 mmdebstrap (1.5.5-1) unstable; urgency=medium
 .
   * New upstream version 1.5.5
   * debian/control: demote fakeroot and fakechroot to Suggests
   * debian/tests/control: add two more items that make the autopkgtest fail
   * run gpg with --check-trustdb making it compatible with gpg-from-sq
     (closes: #1077600)
   * do not run dpkg-preconfigure via DPkg::Pre-Install-Pkgs hook (Closes:
     #1091442)
Checksums-Sha1:
 250302bc65558a4487c5c896093e9cac1a34509d 2553 mmdebstrap_1.5.5-1.dsc
 86590417d08a56c58aff02aee0ac9393b59fcfba 162224 mmdebstrap_1.5.5.orig.tar.gz
 2aeebd1f1022584d414f986c8fdac04dd6ce432b 18968 mmdebstrap_1.5.5-1.debian.tar.xz
Checksums-Sha256:
 ff75429a2a1906b45fd60f8bfe67511b0d1c75326fc1aba170adbc6cf5eeaf9a 2553 mmdebstrap_1.5.5-1.dsc
 863bfbc037b4d26d6088974613d52fe6d3b25052c26680baee5b1ec4412edd53 162224 mmdebstrap_1.5.5.orig.tar.gz
 753a6a336867150841dd643ba82ae25345d51b44b531b2f09c18c38be8c68167 18968 mmdebstrap_1.5.5-1.debian.tar.xz
Files:
 1f6453aedf63084d38b6eb9c250e7a5f 2553 admin optional mmdebstrap_1.5.5-1.dsc
 b10fb4b437401c989293c2bec6eb92d6 162224 admin optional mmdebstrap_1.5.5.orig.tar.gz
 1711b4f4e27efc257c243adbf86027da 18968 admin optional mmdebstrap_1.5.5-1.debian.tar.xz
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmeAYUoACgkQ8sulx4+9
g+GRuA//TKXkst4LSiqlGlZzYfl4nThyPDqNrl3IT+LKolyAXnQjYm3vch4yILBN
UI8ig9mlfbx6WJg9AMOzujsH3XO2W+89T9+s1SwTsc7xufrpQuO8RV/xp23Y9o3r
4o4YjEGFhdQpymzyjEWTpVr1EoT4eLlNC4Od66S4FREj4pIaw3pZuPi8NHn3wEe+
bdehyaGQtFf9l1JIhnYqMbDqOBwxfhvzfWzkexQReujF9VVSN4LdCW27bNCVIkPO
uR6k45b1hKv2PSzFAapSqpK3svuDaF3Igb5sQh/7RNs97yYfeB5WAoMvUzRzCdyu
ANyh5sHckyddumzABCtQgYNseAYoatqNcXH14yiDYKwGfilx3EYZQlc61y/45DVt
nb7hAyaCMya/GFY+aw9B25kH14W7F2vie6SO2SuOtOHBWHixkvIX43T0x/lcUV3N
L7zMJEDAi6cF0b3ViN0lqFLe/v/iEK8x0NsJPzr7RYGfF9Y6ncgHdoOX/xRlg9G/
p7A/CBGo9tJyEfupVbgyiiHJQ972x2DxDY5h+ePGGOCGGM6lMOSRX86p4yHa/0Q/
vUi4JiSca7Ts9marvAa84CG87rKE90Yj4p0ZDKXjYKJjl6dNzwpZ11zYQNEoAKrq
DUKQ6piqMjzn7fD+1hKDDAbhSd80A7XKP7u/ont7gIXTYFus6Eo=
=YowE
-----END PGP SIGNATURE-----

Hi debconf & apt maintainers,

Quoting Johannes Schauer Marin Rodrigues (2025-01-09 18:45:32)

it seems that the test that I wrote to check that this works was faulty and the
bug persists. I talked again with Helmut Grohne and Julian Klode on IRC and we
there are now the following ideas to fix this:

1. mmdebstrap fixes this
------------------------

The problem with my workaround in mmdebstrap was that setting

    #clear DPkg::Pre-Install-Pkgs;

in a config passed to apt via APT_CONFIG is rendered useless as
/etc/apt/apt.conf.d/70debconf will still take priority. So one possible
solution to this is: let mmdebstrap write yet another config file which is able
to override 70debconf:

@@ -3108,7 +3108,11 @@ sub run_setup() {
     # dpkg-preconfigure should not be needed as we also have set
     # DEBIAN_FRONTEND=noninteractive and DEBCONF_NONINTERACTIVE_SEEN=true and
     # should thus never see debconf prompts. See #1091442 for details.
-    print $conf "#clear DPkg::Pre-Install-Pkgs;\n";
+    {
+        open my $tmp, '>', "$options->{root}/etc/apt/apt.conf.d/99debconf"
+          or error "cannot open /etc/apt/apt.conf.d/99debconf: $!";
+        print $tmp "#clear DPkg::Pre-Install-Pkgs;\n";
+    }

     close $conf;


2. apt fixes this
-----------------

In #debian-apt Julian argued that apt should really not be running *any*
scripts if "Dir" is not "/". The reason for that is that apt is not telling
scripts about the setting of "Dir" at all, so it's dangerous to call them if
apt is not operating on the root directory but a different path. According to
Julian (and I agree) it is weird if somebody runs:

    apt update -o Dir=foo

and then hooks from the system outside of foo are run. So we could also fix
this by doing (minimal untested patch):

--- a/apt-pkg/deb/dpkgpm.cc
+++ b/apt-pkg/deb/dpkgpm.cc
@@ -1545,11 +1545,13 @@ bool pkgDPkgPM::Go(APT::Progress::PackageManager *progress)
    unsigned int const MaxArgBytes = _config->FindI("Dpkg::MaxArgBytes", OSArgMax);
    bool const NoTriggers = _config->FindB("DPkg::NoTriggers", true);

+   if (_config->FindDir("Dir") == "/") { // only run hooks if we operate on the root
    if (RunScripts("DPkg::Pre-Invoke") == false)
       return false;

    if (RunScriptsWithPkgs("DPkg::Pre-Install-Pkgs") == false)
       return false;
+   }

    auto const noopDPkgInvocation = _config->FindB("Debug::pkgDPkgPM",false);
    // store auto-bits as they are supposed to be after dpkg is run
@@ -2216,8 +2218,10 @@ bool pkgDPkgPM::Go(APT::Progress::PackageManager *progress)

    d->progress->Stop();

+   if (_config->FindDir("Dir") == "/") { // only run hooks if we operate on the root
    if (RunScripts("DPkg::Post-Invoke") == false)
       return false;
+   }

    return d->dpkg_error.empty();
 }

3. debconf fixes this
---------------------

Thirdly, and lastly, one could argue that it does not make sense for
dpkg-preconfigure to do anything in a bootstrapping scenario where the user is
not asked debconf questions in the first place. Instead of inventing a way how
to communicate "we are in a bootstrapping scenario" to dpkg-preconfigure, could
dpkg-reconfigure just exit early and do nothing if
DEBIAN_FRONTEND=noninteractive? Because in that case, there will not be any
debconf questions asked, so why run dpkg-preconfigure at all?

--- a/dpkg-preconfigure
+++ b/dpkg-preconfigure
@@ -113,6 +113,11 @@ qq{Usage: dpkg-preconfigure [options] [debs]
        "apt"                   => \$apt,
 );

+if (exists $ENV{DEBIAN_FRONTEND} and $ENV{DEBIAN_FRONTEND} eq 'noninteractive') {
+    print STDERR sprintf("dpkg-preconfigure: ".gettext("Skipping dpkg-preconfigure with DEBIAN_FRONTEND set to 'noninteractive'")), "\n";
+    exit 0;
+}
+
 # In apt mode, need unbuffered output. Let's just enable it.
 $|=1;


What do you think?

Thanks!

cheers, josch

Hi,

Quoting Johannes Schauer Marin Rodrigues (2025-01-10 15:53:00)

I have added this workaround in mmdebstrap 1.5.6 because a couple of packages like tzdata and perltidy are waiting for it to succeed its autopkgtest again and transition to testing. I'd like to drop this workaround again soon.

while apt developers agree that it is not optimal for hooks to run if Dir is not /, a good patch would be much more complex than the POC I posted, so they would prefer debconf to change for this.

I implemented that in https://salsa.debian.org/pkg-debconf/debconf/-/merge_requests/20

Thanks!

cheers, josch