#1144866 runit-run: shutdown ordering under systemd

Package:
runit-run
Source:
runit-run
Submitter:
Jan Braun
Date:
2026-09-06 12:11:01 UTC
Severity:
normal
Tags:
#1144866#5
Date:
2026-08-19 14:52:34 UTC
From:
To:
Dear Maintainer,

I'm using runit under systemd via runit-run. Some of the
runit-supervised services depend on systemd-supervised ones, and it's
easy to add a file in /etc/systemd/system/runit.service.d/ specifying
e.g.
| [Unit]
| After=postgresql.service valkey-server.service
| Wants=postgresql.service valkey-server.service

This works properly for startup. However, there's a race condition on
shutdown:

The runit.service is defined with
| KillSignal=SIGHUP
| KillMode=process
which causes systemd to send SIGHUP to the runsvdir. Quoting
runsvdir(8):
| If runsvdir receives a HUP signal, it sends a TERM signal to each
| runsv(8) process it is monitoring and then exits with 111.
...and that is exactly what it does. In particular, runsvdir does NOT
wait for the runsv processes to exit first, hence the runit-supervised
services may not have terminated (or even gotten SIGTERM) yet.

But systemd (in KillMode=process) assumes that runsvdir exiting means
the service is down, and proceeds to stop the dependencies. Depending on
timing, this breaks my still-running (or still-shutting-down) services.

I wish for this race to be fixed. :)

I can see 2.5 ways to do that:
1) It should be possible to teach runsvdir to wait for its runsv
   children to exit before exiting itself. And since runsv in turn does
   wait for its children to exit, that would ensure the whole
   supervision tree has exited when runsvdir does, as systemd expects.
2) It is possible to use systemd's ExecStop= configuration to delay
   systemd from considering the service stopped until the process
   specified there has exited. Hence, specify a process that waits for
   the supervision tree to die.
2a) You could do so by interacting with
    /etc/runit/runsvdir/svmanaged/*/supervise/ to find the processes in
    question, or
2b) since systemd puts each service in its own cgroup, you can wait for
    that cgroup to become empty.

While I think 1) would be cleaner, I'm unsure about the compatibility
implications (Change SIGHUP behaviour? Use a new signal? Would Debian
want to patch runsvdir like that?) and the reason for the current
runsvdir behaviour (When/why is not waiting ever useful?).
If you think some version of this would be a good feature, please say
so, and I'll try to come up with a patch.

So meanwhile, I implemented 2b), the resulting ExecStop= script is
attached. It has been working well for me. I hope it's useful, or even
acceptable for inclusion in Debian.

Please tell me if I can do anything else to help.
Thank you for maintaining runit!
cheers,
    Jan

#1144866#10
Date:
2026-08-21 14:45:55 UTC
From:
To:
Hi Jan,

thanks for reporting this. I don't use runit-run (and systemd) so the
package relies on report from users

On Wed, 19 Aug 2026 16:52:34 +0200 Jan Braun <janbraun@gmx.de> wrote:

[..]

OK

now that upstream is active again I prefer not to carry additional
Debian specific patches, so this would be better discussed upstream.

let's first try to address this without patching runit

I'm ok with this patch, but first I wonder if there is a way to address
this with systemd directives (without extra scripts);

I don't know why we use KillMode=process , by looking at

https://www.freedesktop.org/software/systemd/man/latest/systemd.kill.html?__goaway_challenge=meta-refresh&__goaway_id=f51c50d8745b3678afa63ccfdd4956af

something like

KillMode=control-group
TimeoutStopSec=2

can be used instead: did you already tried something like the above
(and it does not work)?

maybe unrelated, but we may also benefit from

SuccessExitStatus=111


Cheers,
Lorenzo

#1144866#17
Date:
2026-09-01 20:02:48 UTC
From:
To:
Hello Jan,

I plan to release runit 2.3.1-5 this weekend or maybe early next week;

if you can confirm whether my idea (below) works or not I'm going to
include either this or your patch to the next runit upload.

Best,
Lorenzo

#1144866#22
Date:
2026-09-03 20:53:55 UTC
From:
To:
Hi,

Lorenzo schrob:
to *all* processes in the control group, including children of
supervised processes, which should not be killed in this way, but
instead be left alone, to be handled by the supervised processes
themselves.

One case where that might conceivably make a difference is dhcpcd, where
the supervised master process might still use its privsep children to do
additional teardown work before it kills them. I haven't tested that
hypothesis, however.

(Of course, it would always be possible to work around
KillMode=control-group for such services by making their ./run file move
them to their own cgroup, and runsv(8) would still DTRT by virtue of
staying in the runit.cgroup and getting the TERM signal.
I'd consider that a hack, tho.)

That seems appropriate.

cheers,
    Jan

#1144866#27
Date:
2026-09-04 15:15:35 UTC
From:
To:
Hi,

ok, then I'm applying your patch. Only thing to be careful is that I
think the service can remain stuck in exectop status for long time if
there is some misbehaving runit service that does not stop at all.
I think that will result in a systemd timeout of 60 (or 180?) seconds,
but I'm not sure...

Best,
Lorenzo

#1144866#30
Date:
2026-09-04 22:09:43 UTC
From:
To:
Hello,

Bug #1144866 in runit reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/debian/runit/-/commit/98ef77c9c935d82715beaf96be0eb85149bc8047
------------------------------------------------------------------------
runit-run: fix race at systemd shutdown

Use ExecStop in runit.service to fix a race at systemd shutdown:
use systemd's cgroup to track childs of the service and wait for
the cgroup to be empty before exec-stop script returns (wait for
all runit services to stop before runit.service is considered as
down by systemd)

Closes: #1144866
Thanks:  Jan Braun <janbraun@gmx.de>
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1144866

#1144866#37
Date:
2026-09-06 12:09:54 UTC
From:
To:
Lorenzo schrob:

Yes. TimeoutStopSec ?= DefaultTimeoutStopSec ?= 90s, according to the
manpage.

And I think that waiting a long time is indeed the only reasonable
default for a supervisor, because a misbehaving service that does not
stop at all is indistinguishable from a properly behaving service that
just takes very long to stop. Noticing the former case and fixing the
underlying bug should be left as an execise for the humans.

Thanks again for maintaining runit. :)
cheers,
    Jan