#1055529 pdns-server: Spams journal after installation because it is configured to automatically restart

Package:
pdns-server
Source:
pdns-server
Description:
extremely powerful and versatile nameserver
Submitter:
Uwe Kleine-König
Date:
2023-11-16 15:15:05 UTC
Severity:
normal
Tags:
#1055529#5
Date:
2023-11-07 20:54:03 UTC
From:
To:
Hello,

on installation of pdns-server the pdns.service is automatically
started. However in my case port 53 is already bound and so it fails to
start. (That might also happen if port 53 isn't blocked because the
default config isn't suitable to successfully run pdns? I didn't check.)

Because of

	Restart=on-failure
	RestartSec=1
	StartLimitInterval=0

in pdns.service the systemd tries to start pdns once per second and for
each try logs something like:

	Nov 07 21:41:55 algol systemd[1]: Starting PowerDNS Authoritative Server...
	Nov 07 21:41:55 algol pdns_server[2329737]: Loading '/usr/lib/x86_64-linux-gnu/pdns/libbindbackend.so'
	Nov 07 21:41:55 algol pdns_server[2329737]: This is a standalone pdns
	Nov 07 21:41:55 algol pdns_server[2329737]: Listening on controlsocket in '/run/pdns/pdns.controlsocket'
	Nov 07 21:41:55 algol pdns_server[2329737]: UDP server bound to 0.0.0.0:53
	Nov 07 21:41:55 algol pdns_server[2329737]: UDP server bound to [::]:53
	Nov 07 21:41:55 algol pdns_server[2329737]: Unable to bind to TCP socket 0.0.0.0:53: Address already in use
	Nov 07 21:41:55 algol pdns_server[2329737]: Fatal error: Unable to bind to TCP socket
	Nov 07 21:41:55 algol systemd[1]: pdns.service: Main process exited, code=exited, status=1/FAILURE
	Nov 07 21:41:55 algol systemd[1]: pdns.service: Failed with result 'exit-code'.
	Nov 07 21:41:55 algol systemd[1]: Failed to start PowerDNS Authoritative Server.
	Nov 07 21:41:56 algol systemd[1]: pdns.service: Scheduled restart job, restart counter is at 23.
	Nov 07 21:41:56 algol systemd[1]: Stopped PowerDNS Authoritative Server.

to the journal. If you don't notice this immediately and stop the
service this effectively spams your journal in a very short time.

IMHO the above mentioned settings are not suitable as a default for a
distribution's package even if the default configuration worked. It
should be an administrator's choice to configure such a behaviour.

Best regards
Uwe

#1055529#10
Date:
2023-11-11 00:20:12 UTC
From:
To:
Hello Uwe,

* Uwe Kleine-König <ukleinek@debian.org> [231107 22:06]:
[..]

I respectfully disagree. If users have pdns-server installed and
running, they want it restarted ASAP. This is the correct behaviour.

Now, I believe Debian's default of "start various daemons on
install" is just wrong nowadays.

For a long time there was nothing we could do in pdns-server (and
pdns-recursor) to change this without breaking existing installs,
but I'll think about passing --no-enable to dh_installsystemd.
I think this should be safe for upgrades, and new installs then need
to systemctl enable pdns-server.service explicitly.

Chris

#1055529#15
Date:
2023-11-11 23:04:35 UTC
From:
To:
Hello Chris,

That's a subjective assumption. At least I don't want that pdns (or any
other service) is restarted once per second without rate limit and
spamming my machine's journal.

I personally prefer a problematic service to die (which I notice by
proper monitoring).

I assume the systemd developers on my side as the default for Restart is
"no", and the default for StartLimitInterval is 10s.

In my experience high-frequency automatic restart has very little
benefits. If there is a problem that makes a certain service fail to
start, you have only problems with such a rogue service (journal
spamming; maybe high load; if the problem is too little system memory,
you might make it hard for an admin to login; ...) If the problem is
that a remote user can trigger a crash, it might be annoying to not have
the service running, but maybe the next remote user can trigger a RCE if
you automatically give unlimited tries for such remote users? So not
restarting might be safer. And if it's only an occasional problem, at
least some rate limiting doesn't hurt.

Having said that, I still think that the restart behaviour should be the
administrator's choice with the package defaulting to no special
configuration.

Looking at pdns's fellow contenders and how they configure automatic
restart:

  - knot:
    Restart=on-abort
    no burst settings
  - bind9:
    Restart=on-failure
    no burst settings
  - unbound:
    Restart=on-failure
    no burst settings

So while these consider themself important enough, too, to Restart on
problems, at least they don't disable systemd's ratelimiting.

This at least makes the situation better directly after package
installation, so that's very welcome. (But IMHO that's only third best
compared to dropping Restart=on-failure and at least not modifying
StartLimitInterval. I'm sure you still disagree.)

Best regards
Uwe

#1055529#20
Date:
2023-11-15 11:29:18 UTC
From:
To:
* Uwe Kleine-König <ukleinek@debian.org> [231112 00:04]:
case. However, pdns-server has a certain behaviour and history that
does not match the other DNS daemons you listed or 'the general
case'.

If you look closely, the service file we ship is upstream's, and,
you'll also see it says --guardian=no in ExecStart.

In case that doesn't mean anything to you: long before systemd
existed, pdns grew its own 'Restart=on-failure'-style thing, called
"guardian". It did that out of a need - TTBOMK because people expect
their DNS servers to be up, and databases (libraries) tend(ed) to
have problem states that were not solvable without the process
exiting. So to get better availability, pdns shipped its own "auto
restart".

When systemd entered the picture, all the auto-restarting was
delegated to systemd, with the (AFAIK) same delays etc. as with the
home-grown guardian.

It certainly is possible that nowadays the autorestart/guardian
thing is obsolete, or the settings could be improved, etc.
I don't want to patch the settings out in Debian. A lot of people
running the pdns packages tend to be confused enough without any
differences from upstream behaviour. -- I know you are
active/present in the upstream channels, could I ask you to raise
the question there?

Best,
Chris