Dear maintainer:

During a rebuild of all packages in unstable using the nocheck build profile,
all the above packages failed to build. The root cause is probably the same,
so I'm filing a single bug for all of them, which is easy to reassign,
initially against dbus-system-bus-common.

This is the way it fails:

Setting up dbus-system-bus-common (1.16.2-2) ...
fatal: The user `messagebus' already exists and has a password. Exiting.
dpkg: error processing package dbus-system-bus-common (--configure):
  old dbus-system-bus-common package postinst maintainer script subprocess failed with exit status 13

I've put full build logs for all the packages here:

https://people.debian.org/~sanvila/build-logs/202601/

Maybe this is related to this other bug in the shadow package:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1124795

but I'm not really sure. If the bugs are really the same and they have to be merged,
please keep the affects.

(I've put Chris in x-debbugs-cc).

Thanks.

Control: affects -1 + shadow adduser
Control: tags -1 + moreinfo help

I suspect this to be a regression in a related package (adduser, useradd
or something in that general ecosystem) or a regression on the host
system where you are doing the builds, rather than a problem with
dbus-system-bus-common specifically, because dbus-system-bus-common
isn't doing anything particularly fancy with adduser. Here is the code
that is failing:

     if command -v systemd-sysusers >/dev/null; then
         systemd-sysusers ${DPKG_ROOT:+--root="$DPKG_ROOT"} dbus.conf
     else
         in_sysroot adduser --system --quiet --group "$MESSAGEUSER"
     fi

I think you are seeing this in dbus simply because it is a
widely-depended-on package that calls adduser, and in particular, lots
of packages (usually incorrectly) build-depend on dbus (see recent MBF
<https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=dbus@packages.debian.org&tag=direct-dbus-dependency>,
resolving more of those bugs would mitigate the impact of what you've
reported here).

codesearch says the error message you quoted comes from adduser, so we
must be in the code path where dbus-system-bus-common has pulled in
adduser to satisfy the adduser|systemd-sysusers dependency. I'd expect
that in a chroot or minimal container like the ones used in sbuild
(although not in a full bootable system with systemd, where
dbus-system-bus-common would use systemd-sysusers).

I was unable to reproduce this with just

     $ podman run --rm -it debian:sid
     # apt update && apt upgrade && apt install dbus-system-bus-common

(which installs adduser), so there must be something in the build
environment that is causing a more complicated setup than just that. The
error message also suggests that the prior contents of /etc/passwd (on
entry to dbus' maintainer scripts) could be significant.

 From the build logs for the packages you marked as affected, am I
correct to think that you're using sbuild in its schroot mode (like
older official buildds), as opposed to unshare mode (like newer official
buildds) or any other backend? I *think* the build logs imply that by
mentioning var/run/schroot/mount, but it isn't 100% obvious.

schroot usually (configuration-dependent!) copies the /etc/passwd,
/etc/shadow and similar files from the host system into the container.
Is your schroot configured to do that? I don't think we can tell from
the log. Please check /etc/schroot/*/nssdatabases, replacing * by
whichever profile your build chroot is configured to use (normally I
would expect "buildd" for this purpose).

What is the host system that these builds run on, and how/when was it
installed? (A cloud image perhaps?) If the regression is being triggered
by /etc/passwd and friends being copied from the host into the chroot,
then the root cause might be a regression in the packages or
configuration used on the host, rather than in the packages inside the
chroot.

What is the output of these on the host system? (run at least the last
two as root, e.g. via sudo)

getent passwd messagebus
getent group messagebus
getent shadow messagebus
getent gshadow messagebus

For comparison, what I get on a desktop machine that was installed as
trixie is:

messagebus:x:101:108::/nonexistent:/usr/sbin/nologin
messagebus:x:108:
messagebus:!:19814::::::
messagebus:!::

Also, if you do a similar build but with the unshare backend instead of
schroot, does that alter the behaviour? The specific package shouldn't
matter, just choose anything smallish that Build-Depends on dbus
(perhaps telepathy-glib).

It doesn't look closely related to any of the quoted errors at first
glance, but it could be a similar regression in useradd or adduser
(stricter validation or a user being created in a different state), perhaps.

I would be surprised if the presence or absence of nocheck matters here.
If you build one of the packages in the same environment (one should be
enough, choose whichever one is quick to build) but without nocheck,
does it fail in the same way?

If a build failure doesn't appear to have anything to do with nocheck,
then reporting it as "FTBFS with nocheck" without knowing whether
nocheck is a key factor seems likely to send maintainers off on a wrong
track.

     smcv

Control: retitle -1 dbus-system-bus-common: fatal: The user `messagebus' already exists and has a password

Setting a new title that is more indicative than "weird dbus/adduser error".

     smcv

severity 1126595 important
thanks

Thanks a lot for your quick reply.

Yes, I was using the schroot backend, and yes, this had nothing to do
with the nocheck profile (so your retitle was more than appropriate).

I would still expect that to work, but I'm downgrading the bug for the
time being, and I will try to answer all the good questions you made.

Thanks.

That's very likely. The reason I reported this initially against
dbus-system-bus-common is that it's the same package which is failing
all the time.

There is not a single "host system" where I'm doing the builds.

The machines are virtual machines from AWS. I create a bunch of them
when I have to build a lot of packages, and they are created from
scratch from the official trixie images. No special customization of
/etc/passwd is made to those machines, other than creating a
user called buildd which is the one which calls sbuild.

Well, I can also reproduce the error in a directory-based chroot
using schroot, doing this as root from a trixie machine:

# schroot -c sid
# apt-get update
# apt-get build-dep gcp

where "sid" is a directory-based chroot created with sbuild-createchroot.

Yes, you guessed right.

Apparently, yes. My build profile is based on the one called "sbuild",
so /etc/schroot/sbuild/nssdatabases is being used.

Yes, virtual machines from the AWS cloud, using the images for trixie.

Well, that would be a problem indeed, because I am not aware that my
trixie machines are "misconfigured" or anything like that.

I get this from another system which is also a trixie VM from AWS:

messagebus:x:996:996:System Message Bus:/nonexistent:/usr/sbin/nologin
messagebus:x:996:
messagebus:!*:20342::::::
messagebus:!*::

Well, there are several differences here.

It is a bug in trixie that those entries have a "*"?

(My desktop system, which is running trixie as well, also has "*").

It is a bug in trixie that messagebugs user has a UID like 996?

(My desktop system has 990).

It is a bug in schroot at all that it copies some files via the
nssdatabases mechanism?

I agree there seems to be some kind of incompatility here, but I don't
think that "this is a regression in the trixie images" is a good
summary of the problem. This has worked flawlessly for many many
years, and I'm inclined to think that the problem is in one
of the packages currently in unstable.

Yes, building with the unshare backend works ok.

Yes, sorry for that. When I reported this, nocheck was the difference, but
only because I would not expect the schroot backend to fail in this
completely unexpected way.

You already retitled the bug and hopefully we are on the good track again.

Thanks.

Hi.

If edit /etc/shadow by hand and remove the "*", like this:

messagebus:!:20294::::::

then I can "apt-get build-dep gcp" again inside the chroot.

Simple questions:

- It is correct or incorrect for trixie images to have "!*" in the
messagebus user?

- It is correct or incorrect for adduser/useradd to say that the user
has a password when it has "!*"?

Note that regardless of the supported/unsupported status of schroot
backend of sbuild, the above issues may have implications regarding
upgrades from trixie to forky.

Thanks.

Hi. In the code you quoted:

  if command -v systemd-sysusers >/dev/null; then
    systemd-sysusers ${DPKG_ROOT:+--root="$DPKG_ROOT"} dbus.conf
  else
    in_sysroot adduser --system --quiet --group "$MESSAGEUSER"
  fi

we are inside a build chroot, which has only essential and
build-essential packages, so the command which works is the second one,
and this boils down to this command not working:

adduser --system --quiet --group "messagebus"

When the messagebus user has "!*", this used to work in trixie but it
does not work anymore in forky/sid. That seems to be the regression.

Would it be ok to reassign this to the adduser package?

Thanks.

For the adduser maintainers, the minimal steps to reproduce this issue
in an expendable chroot/container/thing are:

# apt update && apt upgrade
# apt install adduser
# adduser --system _star
# adduser --system _bang
# adduser --system _bangstar
# echo '_star:*' | chpasswd -e
# echo '_bang:!' | chpasswd -e
# echo '_bangstar:!*' | chpasswd -e
# adduser --system _bang
# adduser --system _star
# adduser --system _bangstar

Expected result: all commands succeed, as they did (I think) in trixie.

Actual result: on sid, the last `adduser --system _bangstar` (after
setting the password) fails with the same error message shown in the
subject line.

Alternative steps to get to the same error, which I think are a closer
match for what actually happened on Santiago's systems, are to do this
in a chroot/container/thing that does not already have a _bangstar user:

# apt update && apt upgrade
# apt install adduser systemd-standalone-sysusers
# systemd-sysusers - <<EOF
u _bangstar
EOF
# adduser --system _bangstar

although in some versions of systemd you might have to use

u! _bangstar

as input to get the same result.
at least as correct as "!", and possibly more so.

According to shadow(5), an encrypted-password field starting with "!"
indicates a user whose account is locked, and everything after the "!"
is what their encrypted password was before the account was locked.

Meanwhile "*" is not a valid result of crypt(3), meaning that "*"
indicates a user who cannot authenticate using a password (but they
might be able to authenticate with non-password mechanisms like ssh
public keys).

Combining the two, "!*" ought to mean that messagebus's account is
locked, *and* even if it wasn't locked, messagebus would still not be
able to authenticate using a password. This seems like what we want for
the system uids that exist to run daemons, like messagebus.

Meanwhile "!" indicates a user whose account is locked, but if it was
somehow unlocked for whatever reason, they would be able to log in (for
example via getty/login) by using a blank password, or perhaps without a
password prompt at all. I think we never actually want that behaviour
for system users like messagebus.

In newer versions of systemd and sysusers.d(5), a line starting with "u
messagebus" would create a user with encrypted password "*", but a line
starting with "u! messagebus" would create a user with encrypted
password "!*". With my dbus upstream hat on, I've had a merge request
to change its sysusers.d snippet to use "u!", which seems like a
positive change: you shouldn't be able to log in as messagebus, under
any circumstances.

However, it seems that this behaviour has evolved over time: in some
versions of systemd, including trixie and sid, plain "u" creates a
locked account with encrypted password "!*".

My reading of shadow(5) is that it's incorrect, because neither "!*" nor
"*" is a possible output of crypt(3), therefore neither can be said to
be a password. But I could be wrong, I'm not an expert.

No, these are dynamically-allocated uids in the 100-999 range and any
uid in that range is as good as any other.

Typically uids close to the low end (like my 108) indicate that the
account was created with adduser, and the high end (like your 990 or
996) indicates that the account was created with systemd-sysusers.

Yes and no. It's schroot working as designed, but for the use-case where
you are using sbuild to get a clean, predictable, reproducible
environment for doing builds, it could be argued that it's a bug in
sbuild and it shouldn't be letting this implementation detail of the
host leak into the chroot/container. The unshare backend is the
solution for this maybe-bug.

     smcv

Well, the unshare backend is indeed my default mode of building
packages these days, and I only use the old schroot backend
occasionally, as it helps me to catch things that otherwise would
remain hidden.

For example, when I build packages in the "future" (to ensure that they
will build in the next stable at least for 3 years), I still use the
old backend, because this way packages trying to access Internet using
https are more likely to fail when they see that the certificates are
expired. I could also use unshare + allowing internet access,
but have not had time to change my scripts for that.

Regarding nocheck build profile, this bug shows that it would be
better indeed to use the unshare backend, so I'll add that to my todo list.

Thanks a lot.

Control: tags -1 = forky sid ftbfs pending
…

This appears to have been fixed in adduser git already, by commit
debian/3.154-13-gd5820ae "rework EXISTING_ and existing_user_status", so
the next adduser upload can presumably close this bug report.

For the FTBFS seen by Santiago as a result of this, it seems that the
problem scenario is:

1. install dbus-system-bus-common on a system that has
    systemd-sysusers(8), so that the messagebus user is created by that;
2. remove systemd-sysusers(8) or copy /etc/shadow to a system that does
    not have it;
3. install dbus-system-bus-common again, so that the messagebus user is
    (re)created by adduser (which is meant to be idempotent, but this bug
    makes it not idempotent in all cases)

For autobuilders, that can be avoided by using sbuild's unshare backend
in preference to its schroot backend, because it is the schroot backend
that copies /etc/shadow from the host system to the chroot.

Regarding Santiago's concerns about upgrades from one Debian release to
another, the only problematic scenario for dbus would be if
systemd-sysusers(8) was previously installed, but is subsequently
removed. This seems like it would be an unusual thing to do: bootable
systems (even with sysvinit) normally have udev, which has a hard
dependency on systemd-sysusers(8), and minimal chroots/containers
normally add packages as required but do not remove them (they might
install systemd-standalone-sysusers or systemd, but if they do, they
will most likely keep it installed).

If I switched dbus-system-bus-common to exclusively use
systemd-sysusers(8) and no longer have an adduser fallback (making it
more similar to for example udev, openssh-server, dhcpcd-base and
polkitd) then that would prevent this from affecting dbus. Given that
openssh-server now has this dependency, I think we're closer to
consensus on it being a normal thing to do than we were last time I
looked at this.

     smcv

Did it get fixed?

In theory, packages should no longer need scaffolding around any adduser
calls. Please let me know in which cases scaffolding is still needed.

otoh, things are probably moot already since the systemd people are
working hard to have systemd-sysusers will squeeze out adduser soon.

Is there anything adduser can still do here?

Greetings
Marc

Did it get fixed?

In theory, packages should no longer need scaffolding around any adduser
calls. Please let me know in which cases scaffolding is still needed.

otoh, things are probably moot already since the systemd people are
working hard to have systemd-sysusers will squeeze out adduser soon.

Is there anything adduser can still do here?

Greetings
Marc

Version: 3.155

I did provide steps to reproduce that don't require specific dependent
packages (like dbus-system-bus-common in Santiago's original report). I
confirm that they now work.

The bug was to do with systemd-sysusers creating a user with encrypted
password "!*", and older adduser --system not accepting that as a valid
locked system user:

$ podman run --rm -it debian:forky-slim
# apt update && apt upgrade
# apt install adduser
...
Setting up adduser (3.155) ...
# adduser --system _star
# adduser --system _bang
# adduser --system _bangstar
# echo '_star:*' | chpasswd -e
# echo '_bang:!' | chpasswd -e
# echo '_bangstar:!*' | chpasswd -e
# adduser --system _bang
# adduser --system _star
# adduser --system _bangstar

The last line (second `adduser --system _bangstar`) is the one that
previously failed.

Alternatively,

$ podman run --rm -it debian:forky-slim
# apt update && apt upgrade
# apt install adduser systemd-standalone-sysusers
# systemd-sysusers - <<EOF
u _bangstar
EOF
# adduser --system _bangstar

also reproduced this bug in the past, and also now works.

In both cases `getent shadow _bangstar` should start with:

     _bangstar:!*:...

Thanks,
     smcv