#1107744 libnss-ldapd,debian-edu-config: flaky autopkgtest for debian-edu-config: newly-created group cannot be resolved

#1107744#5
Date:
2025-06-13 15:44:14 UTC
From:
To:
As previously discussed in the thread starting at
<https://lists.debian.org/debian-powerpc/2025/06/msg00002.html>,
the autopkgtest for debian-edu-config does not seem to be entirely
stable, especially on ppc64el. I'm attempting to summarize the thread in
this bug report:

This test installs debian-edu metapackages, which in particular include
libnss-ldapd and polkitd. As far as I can see, there is no LDAP
configuration and no LDAP server provided: the nss(5) configuration
allows user and group name resolution via LDAP, but in practice it will
fail to resolve anything, and hopefully it should gracefully fail over
to looking up users and groups in the passwd(5) and group(5) flat files.

The symptom of the failure is that sometimes, polkitd.postinst will
successfully invoke systemd-sysusers to create the polkitd user and
group in passwd(5) and group(5):

but then immediately after that, an operation that involves looking up
the newly-created polkitd group will fail, instead of falling back to
group(5) as it should:

This is not specific to polkitd. Other postinsts also fail to look up a
newly-created user:
an error synthesized internally by systemd utility code to represent
"the lookup failed" rather than specifically referring to a process not
being found.

For whatever reason, this seems to fail more often on ppc64el:
https://ci.debian.net/packages/d/debian-edu-config/testing/ppc64el/61391819/
https://ci.debian.net/packages/d/debian-edu-config/testing/ppc64el/61390539/
https://ci.debian.net/packages/d/debian-edu-config/testing/ppc64el/61390462/
https://ci.debian.net/packages/d/debian-edu-config/testing/ppc64el/61388736/
https://ci.debian.net/packages/d/debian-edu-config/testing/ppc64el/61387577/
https://ci.debian.net/packages/d/debian-edu-config/testing/ppc64el/61164352/

However, it has failed on amd64 and arm64 on at least one occasion each:
https://ci.debian.net/packages/d/debian-edu-config/testing/amd64/60137393/
https://ci.debian.net/packages/d/debian-edu-config/testing/arm64/61120346/

(search for "root:polkitd" to find the error)

I think this could point to a bug in either the libnss-ldapd package, or
the nslcd service that it depends on, or something about how debian-edu
configures these packages. I think it's unlikely to be a bug in polkitd
or systemd-sysusers.

    smcv

#1107744#10
Date:
2025-06-13 16:26:20 UTC
From:
To:
Hi Simon,

thanks for sending this bug report.

Normally, if nslcd and libnss-ldapd get installed they are inert until
and admin adjust /etc/nsswitch.conf and adds ldap ad nss provider.

In Debian Edu, we do this via cf3/cf.ldap-client in debian-edu-config
after all packages have been installed.

Is nscd also installed in the testbed? If so, an nscd -i group might
work around that issue (for debugging things and hunting down the root
cause of this).

Hmmm, this is very strange. It would be interesting if this error also
occurs if libnss-ldapd/nscd were not installed.

hmmm, this is peculiar...

ok.

I agree that this is unrelated to polkitd or any of those other
services. I will check how to intercept a shell session in an
autopkgtest testbedband try to narrow down the cause of the issue.

Mike

#1107744#15
Date:
2025-06-13 16:26:20 UTC
From:
To:
Hi Simon,

thanks for sending this bug report.

Normally, if nslcd and libnss-ldapd get installed they are inert until
and admin adjust /etc/nsswitch.conf and adds ldap ad nss provider.

In Debian Edu, we do this via cf3/cf.ldap-client in debian-edu-config
after all packages have been installed.

Is nscd also installed in the testbed? If so, an nscd -i group might
work around that issue (for debugging things and hunting down the root
cause of this).

Hmmm, this is very strange. It would be interesting if this error also
occurs if libnss-ldapd/nscd were not installed.

hmmm, this is peculiar...

ok.

I agree that this is unrelated to polkitd or any of those other
services. I will check how to intercept a shell session in an
autopkgtest testbedband try to narrow down the cause of the issue.

Mike

#1107744#20
Date:
2025-06-13 16:41:56 UTC
From:
To:
OK, so it doesn't automatically add itself to the configuration like
e.g. libnss-systemd or libnss-mdns do?

That does make sense - libnss-ldapd is presumably not going to be useful
until the sysadmin has had an opportunity to configure an LDAP server.
If true, that would seem to point to libnss-ldapd not being the problem,
because I think the failure to install polkitd is happening long before
any debian-edu-specific setup gets a chance to run.

 From the log, looks like yes, but only because debian-edu-config
installed it. If nscd is automatically used when installed, then that's
another possible root cause - I know it has had a chequered history and
lots of old bugs remain open.

I don't have interactive access to the testbed. I only reported this
because Ted ran into this as a blocker for an e2fsprogs upload,
initially suspected a polkitd bug and asked the polkitd uploaders for
help; I'm afraid I don't have any more access, knowledge, or ability to
reproduce the issue than you do.

You could try uploading a version of debian-edu-config without those
packages to experimental, or testing it locally?

Unfortunately the error is intermittent, so it'll be hard to know
whether you've successfully worked around it or whether you were just
lucky.

You might find `autopkgtest --shell-fail` useful.

     smcv

#1107744#25
Date:
2025-06-13 20:26:10 UTC
From:
To:
Hi Simon,

Ah, ooh... maybe it does since trixie. I hasn't been doing this until
bookworm (at least not when installing libnss-ldapd manually).

Checked this a little more thoroughly. When manually installing
libnss-ldapd, it offers nsswitch.conf updating via debconf, no service
is pre-selected, so nothing in nsswitch.conf gets changed. Matches
with my observation from previous Debian releases.

However, in debian-edu-install src:pkg, we preseed this paramter (see
preseed-values/defaults.networked in debian-edu-install):

```
# Settings for libnss-ldapd
libnss-ldapd    libnss-ldapd/nsswitch   multiselect     group,
netgroup, networks, passwd, shadow
```

So, in Debian Edu, it is assumed that LDAP is always available when
installing networked clients. The "networked" profile applies to
machines that get installed on-site where and when the LDAP-server
(TJENER) is already running.

However, I wonder if we should change this. We have the preseed values
_and_ we have the cfengine3 postinstall configuration of
nsswitch.conf. IMHO, we should stop applying the preseedings during
installation (fixing the here-discussed problem) and apply those same
preseeding values via cfengine3 post-install.

Normally, nscd and nslcd are not needed on the same host. In fact,
nscd masks/delays changes applied to LDAP being available on the
client. So, in our customer deployments, we actually remove nscd (or
disable it for most services, need to recheck this, irrelevant here).

Ouch. I think I will go for the removal of the libnss-ldapd/nsswitch
preseeding in debian-edu-install. This should resolved this issue
sustainably.

Thanks, will try...

Mike

#1107744#30
Date:
2025-07-20 05:51:01 UTC
From:
To:
Hi,

I ran into this for the current unblock request for glibc.

On Fri, 13 Jun 2025 20:26:10 +0000 Mike Gabriel  <mike.gabriel@das-netzwerkteam.de> wrote:
 >>> I will check how to intercept a shell session in an autopkgtest testbed

 >> You might find `autopkgtest --shell-fail` useful.

 > Thanks, will try...


Antonio and I can give you access to a testbed where the test just ran.
Ping us on IRC on otfc #debci. We'll need to be on-line while debugging
is ongoing so it needs some alignment on the moment.

Paul

#1107744#35
Date:
2025-07-20 10:02:41 UTC
From:
To:
Hi Elbrus,

I can be available tomorrow (2025-07-21) during day time (after 11am
CEST). Will that work?

Mike

#1107744#40
Date:
2025-07-20 11:43:04 UTC
From:
To:
Hi,


I do all my Debian work outside of my work hours. During day time I can
support on Thursdays. In the evening I'm regularly available, more so if
planned.

Paul

#1107744#45
Date:
2025-07-24 15:51:49 UTC
From:
To:
Hi Simon,

I have looked into this a bit close with help from elbrus to intercept
a testbed after autopkgtest run.

Only thing I could spot is that /etc/nsswitch.conf refers to the
systemd libnss plugin. This requires systemd-userdbd.service to be
started which is not the case in the testbed it seems.

passwd:         files systemd
group:          files systemd
shadow:         files systemd
gshadow:        files systemd

Could a non-started systemd-userdbd be the cause of the problem?

Furthermore, the recommended setup is:

passwd:         files systemd
group:          files [SUCCESS=merge] systemd
shadow:         files systemd
gshadow:        files systemd

Greets,
Mike