#772826 watchdog fails to upgrade

Package:
watchdog
Source:
watchdog
Description:
system health checker and software/hardware watchdog handler
Submitter:
Uwe Storbeck
Date:
2014-12-17 11:39:08 UTC
Severity:
important
#772826#5
Date:
2014-12-11 13:58:29 UTC
From:
To:
Dear Maintainer,

the watchdog package fails to upgrade (from version 5.14-2):

  Setting up watchdog (5.14-3) ...
  /run/udev or .udevdb or .udev presence implies active udev.  Aborting MAKEDEV invocation.
  Job for watchdog.service canceled.
  invoke-rc.d: initscript watchdog, action "restart" failed.
  dpkg: error processing package watchdog (--configure):
   subprocess installed post-installation script returned error exit status 1

Trying to recover at the end of the aptitude run fails too:

  Errors were encountered while processing:
   watchdog
  E: Sub-process /usr/bin/dpkg returned an error code (1)
  Failed to perform requested operation on package.  Trying to recover:
  Setting up watchdog (5.14-3) ...
  /run/udev or .udevdb or .udev presence implies active udev.  Aborting MAKEDEV invocation.

Also when I try to restart watchdog manually I end up with no
watchdog process running:

  # systemctl status watchdog.service
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: active (running) since Thu 2014-12-11 14:27:42 CET; 22s ago
    Process: 29018 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (code=exited, status=1/FAILURE)
    Process: 29042 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
    Process: 29040 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
   Main PID: 29044 (watchdog)
     CGroup: /system.slice/watchdog.service
	     └─29044 /usr/sbin/watchdog

  # systemctl restart watchdog.service
  Job for watchdog.service canceled.
  # systemctl status -l watchdog.service
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: inactive (dead) since Thu 2014-12-11 14:30:15 CET; 33s ago
    Process: 29639 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (code=exited, status=1/FAILURE)
    Process: 29042 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
    Process: 29040 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
   Main PID: 29044 (code=exited, status=0/SUCCESS)

  Dec 11 14:30:10 grappa watchdog[29044]: stopping daemon (5.14)
  Dec 11 14:30:15 grappa systemd[1]: watchdog.service: control process exited, code=exited status=1
  Dec 11 14:30:15 grappa systemd[1]: Unit watchdog.service entered failed state.

Using only the start command works, but ends up with status
1/FAILURE for the service:

  # systemctl start watchdog.service
  # systemctl status -l watchdog.service
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: active (running) since Thu 2014-12-11 14:31:35 CET; 15s ago
    Process: 29639 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (code=exited, status=1/FAILURE)
    Process: 29675 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
    Process: 29673 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
   Main PID: 29678 (watchdog)
     CGroup: /system.slice/watchdog.service
	     └─29678 /usr/sbin/watchdog

  Dec 11 14:31:35 grappa watchdog[29678]: starting daemon (5.14):
  Dec 11 14:31:35 grappa watchdog[29678]: int=1s realtime=yes sync=no soft=no mla=0 mem=0
  Dec 11 14:31:35 grappa watchdog[29678]: ping: no machine to check
  Dec 11 14:31:35 grappa watchdog[29678]: file: no file to check
  Dec 11 14:31:35 grappa watchdog[29678]: pidfile: no server process to check
  Dec 11 14:31:35 grappa watchdog[29678]: interface: no interface to check
  Dec 11 14:31:35 grappa watchdog[29678]: temperature: no sensors to check
  Dec 11 14:31:35 grappa watchdog[29678]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=root no_act=no force=no
  Dec 11 14:31:35 grappa watchdog[29678]: cannot set timeout 60 (errno = 22 = 'Invalid argument')
  Dec 11 14:31:35 grappa watchdog[29678]: hardware watchdog identity: INTCAMT

I'm not sure what the correct severity for this bug should be
(I would consider it RC, so "serious"?). Please adjust the
severity as needed.

Regards

Uwe


PS: The old config file /etc/default/watchdog seems not to be
properly removed from the package, dpkg -s still lists it (as
obsolete). See also the "Configuration Files" section below.

#772826#10
Date:
2014-12-16 07:51:51 UTC
From:
To:
Please leave it at important for now. I can see your reasoning for a higher
level, but I haven't heard of anyone else having the same problem and I
cannot reproduce it on my system either, at least not yet, so it may be a
local problem.

What happens if you stop watchdog, does the process go away? Or if you happen to reboot, does that change things? I'm wondering if the old process stays in the way and thus may need to be killed manually. I have it that before, that processes do not disappear despite being stopped via systemctl.

Again, works for me. This one could be related to the configure not running
correctly.

Michael

#772826#15
Date:
2014-12-16 12:50:03 UTC
From:
To:
No, the process is stopped properly. systemctl start and stop
work as expected, only systemctl restart fails.


Stop / start:

  # systemctl status watchdog
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: active (running) since Thu 2014-12-11 14:44:09 CET; 4 days ago
   Main PID: 30954 (watchdog)
     CGroup: /system.slice/watchdog.service
	     └─30954 /usr/sbin/watchdog

  Dec 11 14:44:09 grappa watchdog[30954]: starting daemon (5.14):
  Dec 11 14:44:09 grappa watchdog[30954]: int=1s realtime=yes sync=no soft=no mla=0 mem=0
  Dec 11 14:44:09 grappa watchdog[30954]: ping: no machine to check
  Dec 11 14:44:09 grappa watchdog[30954]: file: no file to check
  Dec 11 14:44:09 grappa watchdog[30954]: pidfile: no server process to check
  Dec 11 14:44:09 grappa watchdog[30954]: interface: no interface to check
  Dec 11 14:44:09 grappa watchdog[30954]: temperature: no sensors to check
  Dec 11 14:44:09 grappa watchdog[30954]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=root no_act=no force=no
  Dec 11 14:44:09 grappa watchdog[30954]: cannot set timeout 60 (errno = 22 = 'Invalid argument')
  Dec 11 14:44:09 grappa watchdog[30954]: hardware watchdog identity: INTCAMT
  # ps ax|grep watchdog
     10 ?        S      0:01 [watchdog/0]
     11 ?        S      0:02 [watchdog/1]
  28719 pts/4    S+     0:00 grep watchdog
  30954 ?        SLs    0:17 /usr/sbin/watchdog

  # systemctl stop watchdog
  # systemctl status watchdog
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: inactive (dead) since Tue 2014-12-16 13:32:38 CET; 7s ago
    Process: 28726 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (code=exited, status=1/FAILURE)
   Main PID: 30954 (code=exited, status=0/SUCCESS)

  Dec 11 14:44:09 grappa watchdog[30954]: file: no file to check
  Dec 11 14:44:09 grappa watchdog[30954]: pidfile: no server process to check
  Dec 11 14:44:09 grappa watchdog[30954]: interface: no interface to check
  Dec 11 14:44:09 grappa watchdog[30954]: temperature: no sensors to check
  Dec 11 14:44:09 grappa watchdog[30954]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=root no_act=no force=no
  Dec 11 14:44:09 grappa watchdog[30954]: cannot set timeout 60 (errno = 22 = 'Invalid argument')
  Dec 11 14:44:09 grappa watchdog[30954]: hardware watchdog identity: INTCAMT
  Dec 16 13:32:33 grappa watchdog[30954]: stopping daemon (5.14)
  Dec 16 13:32:38 grappa systemd[1]: watchdog.service: control process exited, code=exited status=1
  Dec 16 13:32:38 grappa systemd[1]: Unit watchdog.service entered failed state.
  # ps ax|grep watchdog
     10 ?        S      0:01 [watchdog/0]
     11 ?        S      0:02 [watchdog/1]
  28746 pts/4    S+     0:00 grep watchdog

  # systemctl start watchdog
  # systemctl status watchdog
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: active (running) since Tue 2014-12-16 13:32:59 CET; 20s ago
    Process: 28726 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (code=exited, status=1/FAILURE)
    Process: 28756 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
    Process: 28753 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
   Main PID: 28758 (watchdog)
     CGroup: /system.slice/watchdog.service
	     └─28758 /usr/sbin/watchdog

  Dec 16 13:32:59 grappa watchdog[28758]: starting daemon (5.14):
  Dec 16 13:32:59 grappa watchdog[28758]: int=1s realtime=yes sync=no soft=no mla=0 mem=0
  Dec 16 13:32:59 grappa watchdog[28758]: ping: no machine to check
  Dec 16 13:32:59 grappa watchdog[28758]: file: no file to check
  Dec 16 13:32:59 grappa watchdog[28758]: pidfile: no server process to check
  Dec 16 13:32:59 grappa watchdog[28758]: interface: no interface to check
  Dec 16 13:32:59 grappa watchdog[28758]: temperature: no sensors to check
  Dec 16 13:32:59 grappa watchdog[28758]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=root no_act=no force=no
  Dec 16 13:32:59 grappa watchdog[28758]: cannot set timeout 60 (errno = 22 = 'Invalid argument')
  Dec 16 13:32:59 grappa watchdog[28758]: hardware watchdog identity: INTCAMT
  # ps ax|grep watchdog
     10 ?        S      0:01 [watchdog/0]
     11 ?        S      0:02 [watchdog/1]
  28758 ?        SLs    0:00 /usr/sbin/watchdog
  28762 pts/4    S+     0:00 grep watchdog


Restart:

  # systemctl restart watchdog
  Job for watchdog.service canceled.
  # systemctl status watchdog
  ● watchdog.service - watchdog daemon
     Loaded: loaded (/lib/systemd/system/watchdog.service; enabled)
     Active: inactive (dead) since Tue 2014-12-16 13:34:04 CET; 6s ago
    Process: 28766 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (code=exited, status=1/FAILURE)
    Process: 28756 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
    Process: 28753 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
   Main PID: 28758 (code=exited, status=0/SUCCESS)

  Dec 16 13:32:59 grappa watchdog[28758]: file: no file to check
  Dec 16 13:32:59 grappa watchdog[28758]: pidfile: no server process to check
  Dec 16 13:32:59 grappa watchdog[28758]: interface: no interface to check
  Dec 16 13:32:59 grappa watchdog[28758]: temperature: no sensors to check
  Dec 16 13:32:59 grappa watchdog[28758]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=root no_act=no force=no
  Dec 16 13:32:59 grappa watchdog[28758]: cannot set timeout 60 (errno = 22 = 'Invalid argument')
  Dec 16 13:32:59 grappa watchdog[28758]: hardware watchdog identity: INTCAMT
  Dec 16 13:33:59 grappa watchdog[28758]: stopping daemon (5.14)
  Dec 16 13:34:04 grappa systemd[1]: watchdog.service: control process exited, code=exited status=1
  Dec 16 13:34:04 grappa systemd[1]: Unit watchdog.service entered failed state.
  # ps ax|grep watchdog
     10 ?        S      0:01 [watchdog/0]
     11 ?        S      0:02 [watchdog/1]
  28783 pts/4    S+     0:00 grep watchdog

The config file is still referenced in the package database:

  # dpkg -S /etc/default/watchdog
  watchdog: /etc/default/watchdog

  # dpkg -s watchdog
  Package: watchdog
  Status: install ok installed
  Priority: extra
  Section: admin
  Installed-Size: 196
  Maintainer: Michael Meskes <meskes@debian.org>
  Architecture: i386
  Version: 5.14-3
  Depends: libc6 (>= 2.7), debconf (>= 0.5) | debconf-2.0, init-system-helpers (>= 1.18~), makedev (>= 2.3.1-24) | udev, lsb-base (>= 3.2-14)
  Conffiles:
   /etc/init.d/watchdog 350ab37d9e081970c491adb488c29d17
   /etc/init.d/wd_keepalive d2f8e84bd91ffd2c5b5ac07774fb751b
   /etc/watchdog.conf bd071fbb51fb2f9ee807af043111ba27
   /etc/default/watchdog f6517ce4745ab05fd8cd4da1a39e5906 obsolete
  Description: system health checker and software/hardware watchdog handler

There's a mechanism to properly remove a config file from the
package database on a package upgrade, but I don't know the
details.

I'm going to purge and reinstall the package, that should at
least solve the problem of the obsolete config file locally.

Regards

Uwe

#772826#20
Date:
2014-12-16 13:46:23 UTC
From:
To:
Hi Michael,

here are my results from purging and reinstalling watchdog.

There are some messages during the installation:

  /run/udev or .udevdb or .udev presence implies active udev.  Aborting MAKEDEV invocation.
  update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults

Not sure if that means anything, but to me it sounds like an
obsolete mechanism is used somewhere.

I don't know what the intended behavior for /etc/default/watchdog
is now. The changelog states:

  Changed rules file to make sure default file doesn't come back.

But the package installation still references /etc/default/watchdog:

  Configuring watchdog
  Please specify whether stopping watchdog should start wd_keepalive to
  keep on triggering the watchdog device. This can be changed later by
  editing /etc/default/watchdog.
  Start wd_keepalive after stopping watchdog?

And the file is recreated on package installation.

At least the package database is cleaned up as expected by
purging and reinstalling watchdog.

I also have found out what causes systemctl restart watchdog to
fail. When I disable run_wd_keepalive in /etc/default/watchdog
anything works as expected (at least as far as I can see). But
when I enable run_wd_keepalive the restart fails as reported
before.

Is restarting watchdog with wd_keepalive as a temporary
replacement not supported?

Regards

Uwe


PS: Sorry for not testing the reboot now, it's my desktop system
and I have too many windows open at the moment.

#772826#25
Date:
2014-12-17 11:33:23 UTC
From:
To:
Ah, I missed that bit in your original report. It seems I can reproduce that
on my system.

I assume you refer to the rm_conffile bits, but that function is about
removing the file from disk. However, we need the file and will recreate it
in postinst. It's just the package database that appears to be wrong. I'll
try reproducing.

Michael

#772826#30
Date:
2014-12-17 11:36:57 UTC
From:
To:
Partly right. :)

I don't like touching these during freeze, though.

Right, this is the intended behaviour. The file should be there.

Ok, good.

It is, but starting one service while stopping another is not really
supported by systemd, sigh! So I went with a workaround/hack to get things
working, but apparently they don't for restart.

No worries, you found the problem without rebooting anyway.

Michael