Dear Maintainer,
After upgrading dhcpcd-base/dhcpcd from 1:10.3.2-6 to 1:10.5.2-1, the next
shutdown/reboot takes about three minutes longer than before.
networking.service's ExecStop (ifdown -a) hangs and is eventually SIGKILLed
after two consecutive 90 s systemd timeouts:
systemd[1]: Stopping networking.service - Raise network interfaces...
ifdown[9640]: sending commands to dhcpcd process
dhcpcd[453]: control command: dhcpcd -k enp1s0
dhcpcd[453]: enp1s0: releasing lease of 172.31.0.122
[90 s pass]
ifdown[9621]: Got signal Terminated, terminating...
systemd[1]: networking.service: Control process exited, code=exited, status=1/FAILURE
[90 s pass]
systemd[1]: networking.service: State 'final-sigterm' timed out. Killing.
systemd[1]: networking.service: Killing process 9640 (dhcpcd) with signal SIGKILL.
systemd[1]: networking.service: Failed with result 'timeout'.
CAUSE
Upgrading the package does not restart dhcpcd.service. dhcpcd.postinst
contains only the dh_installsystemd *enable* snippet, and
dhcpcd-base.postinst only runs systemd-sysusers; neither invokes
deb-systemd-invoke restart. So after the upgrade the long-running manager is
still the old (10.3.2) binary, while ifup/ifdown now exec the new (10.5.2)
dhcpcd client.
The 10.5.2 client writes its command to the AF_UNIX control socket and then
blocks in read() forever. The 10.3.2 manager receives and fully executes the
command -- it removes the interface, releases the lease and deletes the routes
-- but never sends the reply the new client waits for. The client has no
timeout, so it hangs until something kills it.
The hung process, with the manager still at 10.3.2 and the client at 10.5.2:
PID PPID STAT WCHAN COMMAND
460 1 Ss do_epoll_wait dhcpcd: [manager] [ip4] [ip6]
848 846 S do_wait ifdown enp1s0
854 848 S do_wait /bin/sh -c dhcpcd -k enp1s0
855 854 S unix_stream_read_generic dhcpcd -k enp1s0
# cat /proc/855/stack
[<0>] unix_stream_read_generic+0x6f9/0xcb0
[<0>] unix_stream_recvmsg+0x87/0x90
[<0>] sock_recvmsg+0x9e/0xa0
[<0>] sock_read_iter+0x97/0x100
[<0>] vfs_read+0x358/0x390
[<0>] ksys_read+0xbf/0xf0
[<0>] do_syscall_64+0xe1/0x610
[<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e
# cat /proc/855/syscall
0 0x5 0x7ffc9bf4dd2c 0x4 0x0 0x0 0x0 0x7ffc9bf4dcb0 0x7f5dff69bca2
(syscall 0 = read, on fd 5, the control socket)
REPRODUCER
1. Boot a system with dhcpcd-base 1:10.3.2-6, dhcpcd.service enabled, and
/etc/network/interfaces containing:
allow-hotplug enp1s0
iface enp1s0 inet dhcp
2. apt install dhcpcd=1:10.5.2-1 dhcpcd-base=1:10.5.2-1
(do not restart dhcpcd.service -- the package does not do it either)
3. ifdown enp1s0
Expected: returns promptly.
Actual: prints "sending commands to dhcpcd process" and hangs forever.
At shutdown the same call is made by networking.service's ExecStop, which is
why the first reboot after this upgrade costs about three extra minutes.
BISECTION
Same VM, same kernel (7.1.12+deb14-cloud-amd64), same ifupdown (0.8.45+nmu1,
whose only change over 0.8.45 is packaging: DPKG_ROOT support and sysusers.d),
same /etc/network/interfaces. Only dhcpcd was varied:
manager 10.3.2 + client 10.3.2 -> ifdown returns in <1 s, clean shutdown
manager 10.5.2 + client 10.5.2 -> ifdown returns in <1 s, clean shutdown
manager 10.5.2 + client 10.3.2 -> ifdown returns in <1 s, clean shutdown
manager 10.3.2 + client 10.5.2 -> ifdown hangs indefinitely
Only the mismatched pair produced by upgrading without restarting the daemon
fails, and only in that direction. Reproduced on two independent VMs, and the
hang disappears on every subsequent reboot, once the manager is 10.5.2 too.
That is what makes it easy to miss: it bites exactly once per upgrade.
SUGGESTED FIX
Restart dhcpcd.service on upgrade (deb-systemd-invoke restart), so that the
manager and the client binaries cannot diverge.
Separately, and worth forwarding upstream: a client that blocks in read() on
the control socket with no timeout and no version handshake is fragile. A
manager that does not understand a command should still respond or close the
socket, and the client should not wait forever. I did not find an existing
upstream issue for this; the closest, NetworkConfiguration/dhcpcd#560, is a
different case in 10.3.0 and is closed. I have filed the upstream half as
NetworkConfiguration/dhcpcd#723:
https://github.com/NetworkConfiguration/dhcpcd/issues/723
WORKAROUND
systemctl restart dhcpcd.service
after upgrading dhcpcd-base, before the next reboot.
Attached is a debdiff against 1:10.5.2-1 that fixes this.
The reason the daemon is never restarted is explicit in debian/rules:
override_dh_installinit:
dh_installinit --name=dhcpcd --no-start --no-stop-on-upgrade --no-restart-after-upgrade
override_dh_installsystemd:
dh_installsystemd --name=dhcpcd --no-start --no-stop-on-upgrade --no-restart-after-upgrade
dh_installsystemd --name=dhcpcd@ --no-start --no-stop-on-upgrade --no-restart-after-upgrade
dh_installsystemd(1) notes that --no-stop-on-upgrade "has the side-effect of
not restarting the service as a part of the upgrade", and --no-start
suppresses the start/restart snippet outright, so the generated postinst
contains only the enable logic.
I assume those flags are deliberate -- a DHCP client is exactly the sort of
daemon you do not want bounced in the middle of an unattended upgrade -- so
rather than change debian/rules I added a debian/dhcpcd.postinst that restarts
the service only when it is already running. Fresh installs, and systems
where the admin has stopped or masked the unit, are unaffected, so the intent
of --no-start is preserved. It follows the same #DEBHELPER# pattern as the
existing debian/dhcpcd.preinst.
If you would rather have debhelper handle it, dropping the three flags in
favour of plain --restart-after-upgrade gives the same result with less code,
at the cost of also starting the daemon on fresh installs. I went with the
postinst because it changes strictly less behaviour, but I have no attachment
to either form.
TESTING
Built the package with and without the change and ran both from the same
starting state: a VM running the 1:10.3.2-6 manager, interface configured by
ifupdown (iface enp1s0 inet dhcp), on Debian forky/sid amd64.
Case 1, stock 1:10.5.2-1
before upgrade: manager=dhcpcd-10.3.2 pkg=1:10.3.2-6
after upgrade: manager=dhcpcd-10.3.2 pkg=1:10.5.2-1
control command: HUNG, SIGKILLed after 20s (exit 137)
Case 2, patched 1:10.5.2-2
before upgrade: manager=dhcpcd-10.3.2 pkg=1:10.3.2-6
after upgrade: manager=dhcpcd-10.5.2 pkg=1:10.5.2-2
control command: returned in 0s (exit 0)
network after the restart: enp1s0 UP 172.31.0.122/24 (unchanged)
Case 3, patched package but dhcpcd.service stopped beforehand
before upgrade: dhcpcd.service is-active=inactive
after upgrade: dhcpcd.service is-active=inactive
The restart does not disturb the running configuration: /etc/dhcpcd.conf ships
"persistent", so the addresses and routes stay up across it, as Case 2 shows.
Finally, the user-visible symptom, upgrading from 1:10.3.2-6 with the patched
package and then rebooting:
Sep 06 10:23:49 testy systemd-logind[443]: System is rebooting.
Sep 06 10:23:49 testy systemd[1]: Stopping networking.service...
Sep 06 10:23:49 testy ifdown[5826]: sending commands to dhcpcd process
Sep 06 10:23:49 testy systemd[1]: networking.service: Deactivated successfully.
Sep 06 10:23:49 testy systemd[1]: Shutting down.
All in the same second, against the roughly three minutes and a SIGKILL
reported originally.
ONE THING I DID NOT COVER
debian/rules passes the same flags for dhcpcd@.service. A running
dhcpcd@<iface>.service instance has the identical mismatch after an upgrade,
but restarting per-interface instances is more disruptive than restarting the
manager, and it is not the case I hit, so I left it alone. You may want to
handle it too.
UPSTREAM
I also reported the other half of this -- that the client blocks in read(2)
forever instead of timing out -- upstream, with a patch:
https://github.com/NetworkConfiguration/dhcpcd/issues/723
https://github.com/NetworkConfiguration/dhcpcd/pull/724
That change makes a mismatched pair recover after 5 seconds with a warning
instead of hanging, but it only helps once the client side is new enough to
contain it; restarting the daemon on upgrade is what actually prevents the
mismatch from arising on Debian.
bin:dhcpcd-base doesn't ship any systemd unit or or init.d script;
bin:dhcpcd does. Reassigning accordingly.
This being said, we indeed don't currently restart after an upgrade.
Would the following change do what you need?
diff --git a/debian/rules b/debian/rules
index 196aab72..7810aff9 100755
--- a/debian/rules
+++ b/debian/rules
@@ -18,11 +18,11 @@ override_dh_install:
dh_install
override_dh_installinit:
- dh_installinit --name=dhcpcd --no-start --no-stop-on-upgrade
--no-restart-after-upgrade
+ dh_installinit --name=dhcpcd --no-stop-on-upgrade
--restart-after-upgrade
override_dh_installsystemd:
- dh_installsystemd --name=dhcpcd --no-start
--no-stop-on-upgrade --no-restart-after-upgrade
- dh_installsystemd --name=dhcpcd@ --no-start
--no-stop-on-upgrade --no-restart-after-upgrade
+ dh_installsystemd --name=dhcpcd --no-stop-on-upgrade
--restart-after-upgrade
+ dh_installsystemd --name=dhcpcd@ --no-stop-on-upgrade
--restart-after-upgrade
-----
Martin-Éric
Understood, and agreed -- I filed it against the wrong binary package.
Thanks for reassigning.
Test: a forky/sid amd64 VM running the 1:10.3.2-6 manager, interface
configured by ifupdown (iface enp1s0 inet dhcp), upgraded to a package
built with your change.
before upgrade: manager=dhcpcd-10.3.2 pkg=1:10.3.2-6
after upgrade: manager=dhcpcd-10.5.2 pkg=1:10.5.2-2~varA
control command (dhcpcd -n enp1s0): returned in 0s, exit 0
network across the restart: enp1s0 UP 172.31.0.122/24 (unchanged)
and the symptom I originally reported, on the next reboot:
Sep 06 11:01:23 testy systemd-logind[431]: System is rebooting.
Sep 06 11:01:23 testy systemd[1]: Stopping networking.service...
Sep 06 11:01:23 testy ifdown[5282]: sending commands to dhcpcd process
Sep 06 11:01:23 testy systemd[1]: networking.service: Deactivated successfully.
Sep 06 11:01:24 testy systemd[1]: Shutting down.
All within a second, against the ~3 minutes and a SIGKILL before.
The restart is safe for the running configuration because /etc/dhcpcd.conf
ships "persistent", so addresses and routes survive it, as the unchanged
address above shows.
TWO SIDE EFFECTS OF DROPPING --no-start, IN CASE THEY ARE UNINTENDED
Your diff drops --no-start as well as flipping --no-restart-after-upgrade.
That selects debhelper's postinst-systemd-restart snippet, which does
if [ -n "$2" ]; then _dh_action=restart; else _dh_action=start; fi
deb-systemd-invoke $_dh_action 'dhcpcd.service'
so besides the fix it also changes two things I measured:
* fresh install now starts the daemon
after purge + install: is-enabled=enabled is-active=active
(previously it was enabled but left inactive until the next boot)
* upgrading while the admin has dhcpcd.service stopped now starts it
before upgrade: is-active=inactive
after upgrade: is-active=active
If you want those, your diff is exactly right and I am happy with it.
If you would rather keep --no-start, then keeping it *and* passing
--restart-after-upgrade explicitly also fixes the bug:
dh_installsystemd --name=dhcpcd --no-start --no-stop-on-upgrade --restart-after-upgrade
An explicitly passed --restart-after-upgrade survives the -r side effect
(dh_installsystemd lines 183-184), and combined with --no-start it selects
postinst-systemd-restartnostart, which emits
if [ -n "$2" ]; then
deb-systemd-invoke try-restart 'dhcpcd.service'
fi
try-restart restarts only a service that is already running, and only on
upgrade. I tested this variant too:
your diff with --no-start kept
upgrade, service running restarted restarted
upgrade, service stopped started left stopped
fresh install started left stopped
Both fix the reported hang. The difference is only in the two rows that are
not the bug.
One thing that argues for your version: with --no-start kept, dh_installinit
emits no invoke-rc.d call at all, so on a non-systemd system the daemon is
not restarted and the bug persists there. Your diff does cover that case.
Conversely, your version's init snippet is
invoke-rc.d --skip-systemd-native dhcpcd $_dh_action || exit 1
where a failed start leaves the package unconfigured; that is debhelper's
normal behaviour, just worth knowing.
I have no strong preference between the two -- you know the package's
history better than I do. Either resolves what I reported.
(I had also prepared a debian/dhcpcd.postinst doing a conditional restart,
which may have crossed with your mail. Your debian/rules change is simpler
and I would drop mine in favour of it.)
TWO SMALL NOTES
The dhcpcd@ line in your diff is a no-op: template units cannot be started
or enabled directly, and dh_installsystemd emits nothing for dhcpcd@ in
postinst, prerm or postrm with either flag set. So a running
dhcpcd@<iface>.service instance still keeps the old binary across an
upgrade. Not my case, and probably rare, but it is not covered by either
variant.
I also reported the other half upstream -- that the client blocks in read(2)
forever rather than timing out -- with a patch:
https://github.com/NetworkConfiguration/dhcpcd/issues/723
https://github.com/NetworkConfiguration/dhcpcd/pull/724
That makes a mismatched pair recover after 5s with a warning instead of
hanging, but it only helps once the client side is new enough to contain it.
Restarting the daemon on upgrade is what actually prevents the mismatch on
Debian, so the two are complementary.
Thanks for the quick turnaround.
Cheers,
Thomas
su 6.9.2026 klo 12.08 Thomas Glanzmann (thomas@glanzmann.de) kirjoitti: Thanks. Good to know. Noted. Let's see how Roy reacts to this one. Martin-Éric
We believe that the bug you reported is fixed in the latest version of
dhcpcd, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 1146835@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Martin-Éric Racine <martin-eric.racine@iki.fi> (supplier of updated dhcpcd package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Sun, 06 Sep 2026 12:05:01 +0300
Source: dhcpcd
Architecture: source
Version: 1:10.5.2-2
Distribution: unstable
Urgency: medium
Maintainer: Martin-Éric Racine <martin-eric.racine@iki.fi>
Changed-By: Martin-Éric Racine <martin-eric.racine@iki.fi>
Closes: 1146692 1146835
Changes:
dhcpcd (1:10.5.2-2) unstable; urgency=medium
.
* [systemd]
- Remove After/Before references to networking.service (Closes: #1146692).
* [rules]
- Ensure that monitor mode restarts after an upgrade (Closes: #1146835).
Checksums-Sha1:
912bebde8030a9d3736d266fffe86bfab4a72328 2449 dhcpcd_10.5.2-2.dsc
6d72d7ffa1dc6c08cd92978f92863402d44f6fd7 21476 dhcpcd_10.5.2-2.debian.tar.xz
7efa99b3fba75c4955577ef9515f5893b8f9269d 5322 dhcpcd_10.5.2-2_source.buildinfo
Checksums-Sha256:
84bd570b5a0485bcd575fb3ab5b077f053351ca2d15a735fb729827908e5ee8e 2449 dhcpcd_10.5.2-2.dsc
02354d8184fad966abfc062ded35c63004339d7c80f527671f18ba3beff63634 21476 dhcpcd_10.5.2-2.debian.tar.xz
2c64cb9c4f370865c0e6e2e576f6b9105b82bf46942ca01da01b6edb3e4d4872 5322 dhcpcd_10.5.2-2_source.buildinfo
Files:
f32ae8834f2057fc6aba851eaf9ea94c 2449 net optional dhcpcd_10.5.2-2.dsc
57e6028a2d0f7523dd2af2fc0e5c2653 21476 net optional dhcpcd_10.5.2-2.debian.tar.xz
69fe6b28a58e86c2a4e845671fafceff 5322 net optional dhcpcd_10.5.2-2_source.buildinfo
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEyJACx3qL7GpObXOQrh+Cd8S017YFAmqdLzkACgkQrh+Cd8S0
17YBIg//fxxCnOFHzXmkTBacL3fnnP//79oPn++ToT5rGTBh3QdpE23O3wflhAu8
JuzNEdhB4MzPluZpDyvk17zQ22XXF62EmSYGbWo5U0A5MSJ+0LfrgJHCacCl/JJu
WvrRFi1wVcsdej1RkkmeXyY6UDT1LyNAngtGhKrXg5g5u1pk/2WFvmVJvQPQYDzh
Gj9wE5c57FbnQ6TFgLDEy31CeESM4YtVFeAKLvuveJ0FqNVLIm3W3W5Uj2uD3nPs
If+mCDWxlb/V8oaEtwyjmcCzUBkysucQrp9BQpKOwIWcYk4ITSCVej1e16ty12bA
x3uEvRm0wTkrHNTtgGc7MHXzsP7DkcUQkMoEQFvPnhO6aU1Nnnhi8moLV3Sho4o7
2HUTSVXfKMSp6Vnh7cEEhCZ3VOii3NayGtWvkzoqH1kQurALmqcYu9UzcXoPZU57
LbVxTVGcxnMJJV+VdbYalq9ZlInFDSBvuwadY39+9PlmLjK+jBbdTCEP1FdX+bIb
YAvvrxG+D6BB/uwItIac1uFYRrQHoopT5emBlzobGbgVJN9kWki0+b+gFOqOn8os
T+AYPfPQSrxd4X4QyUdSTz7QO7n52xfLv3erruaR4ZGlX4OMScayQ0NjDxCRIadZ
royhpqZWYOW5snhgG3tCmDNgfPQlPw5bQA1feAk3jPjXLDWzTVg=
=qd/6
-----END PGP SIGNATURE-----