#987046 autopkgtest: VirtSubproc's execute_timeout leaves sudoify'd processes behind

#987046#5
Date:
2021-04-16 11:23:03 UTC
From:
To:
If the execution duration exceeds the specified timeout,
`execute_timeout` tries to kill the process it started. This will
systematically fail if the command was wrapped in `sudoify`, because
`execute_timeout` will of course not be able to terminate a process
owned by UID 0.

Here's an actual example, where an lxc test container took too long to
get its networking available (the corresponding call is in
autopkgtest-virt-lxc's `wait_booted` at line 131):

```
autopkgtest-virt-lxc [06:18:24]: ERROR: WARNING: Cannot kill timed out process ['sudo', 'lxc-attach', '--name', 'ci-106-c7628f8c', '--', 'sh', '-ec', 'if [ -d /run/systemd/system ]; then systemctl start network-online.target; else while ps -ef | grep -q "/etc/init\\.d/rc"; do sleep 1; done; fi']: [Errno 1] Operation not permitted
```

I used the following pytest code to isolate and reproduce the error:

```python
import pytest
import re
import sys
sys.path.append('lib')

import VirtSubproc


@pytest.mark.parametrize("cmd", (["sleep", "10"], ["sudo", "sleep", "10"]))
def test_timeout_one_second(cmd, capfd):
    with pytest.raises(VirtSubproc.Timeout):
        VirtSubproc.execute_timeout(None, 1, cmd)

    out, err = capfd.readouterr()
    assert not re.search(r'WARNING.*Cannot.*timed out process', err)
```

I don't see an easy solution to this problem; I am able to work around
the issue by using the attached patch, combined with giving the `debci` user
(that runs our autopkgtests) additional sudo permissions for the
commands that timeout for us, but that's suboptimal.

#987046#8
Date:
2026-09-17 16:49:39 UTC
From:
To:
Hello,

Bug #987046 in autopkgtest reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/ci-team/autopkgtest/-/commit/25b7e3c43259a3b3e53ad7675ba35aba200f94d8
In execute_timeout(), when the timeout is reached(), send a SIGTERM
instead of a SIGKILL.

Sending a SIGKILL is problematic when the command uses sudo, as sudo
cannot catch the signal and forward it to its privileged child. This was
worked around in 63eb7e60c by adding an extra timeout layer, making sudo
invocations like:

  check_exec(sudoify(argv, 300), outp=True, timeout=310)

Note the two timeouts: one for check_exec(), one for the inner
timeout(1). Other than being ugly, this layering caused #1011509, which
778f0aac6df fixed by running "timeout sudo" instead of "sudo timeout".
It also left a gap: 63eb7e60c only wrapped lxc-start and friends, so
sudoify() calls without a timeout still rely on the SIGKILL that sudo
cannot forward. That is #987046, hit via wait_booted().

A better solution is: on timeout, send a SIGTERM first so sudo can
forward it, wait up to 10 seconds, and only then fall back to SIGKILL.
This also makes the extra timeout(1) layer redundant, so drop it, and
also drop the sudoify() timeout parameter.

Closes: #987046
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/987046

#987046#13
Date:
2026-09-17 16:50:05 UTC
From:
To:
Hello,

Bug #987046 in autopkgtest reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/ci-team/autopkgtest/-/commit/9c0edea17ecce8462be7d9222825586ed8eaf525
In execute_timeout(), when the timeout is reached, send a SIGTERM
instead of a SIGKILL.

Sending a SIGKILL is problematic when the command uses sudo, as sudo
cannot catch the signal and forward it to its privileged child. This was
worked around in 63eb7e60c by adding an extra timeout layer, making sudo
invocations like:

  check_exec(sudoify(argv, 300), outp=True, timeout=310)

Note the two timeouts: one for check_exec(), one for the inner
timeout(1). Other than being ugly, this layering caused #1011509, which
778f0aac6df fixed by running "timeout sudo" instead of "sudo timeout".
It also left a gap: 63eb7e60c only wrapped lxc-start and friends, so
sudoify() calls without a timeout still rely on the SIGKILL that sudo
cannot forward. That is #987046, hit via wait_booted().

A better solution is: on timeout, send a SIGTERM first so sudo can
forward it, wait up to 10 seconds, and only then fall back to SIGKILL.
This also makes the extra timeout(1) layer redundant, so drop it, and
also drop the sudoify() timeout parameter.

Closes: #987046
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/987046

#987046#16
Date:
2026-09-17 20:05:37 UTC
From:
To:
Hello,

Bug #987046 in autopkgtest reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/ci-team/autopkgtest/-/commit/9c0edea17ecce8462be7d9222825586ed8eaf525
In execute_timeout(), when the timeout is reached, send a SIGTERM
instead of a SIGKILL.

Sending a SIGKILL is problematic when the command uses sudo, as sudo
cannot catch the signal and forward it to its privileged child. This was
worked around in 63eb7e60c by adding an extra timeout layer, making sudo
invocations like:

  check_exec(sudoify(argv, 300), outp=True, timeout=310)

Note the two timeouts: one for check_exec(), one for the inner
timeout(1). Other than being ugly, this layering caused #1011509, which
778f0aac6df fixed by running "timeout sudo" instead of "sudo timeout".
It also left a gap: 63eb7e60c only wrapped lxc-start and friends, so
sudoify() calls without a timeout still rely on the SIGKILL that sudo
cannot forward. That is #987046, hit via wait_booted().

A better solution is: on timeout, send a SIGTERM first so sudo can
forward it, wait up to 10 seconds, and only then fall back to SIGKILL.
This also makes the extra timeout(1) layer redundant, so drop it, and
also drop the sudoify() timeout parameter.

Closes: #987046
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/987046