Once a day, at exactly the same time (+/- a couple of minutes), I get the
following errors in my logs:

  Oct  6 07:39:49 akranes dunst[28243]: CRITICAL: Cannot open X11 display.
  Oct  6 07:39:49 akranes systemd[28227]: dunst.service: Main process exited, code=exited, status=1/FAILURE
  Oct  6 07:39:49 akranes systemd[28227]: dunst.service: Failed with result 'exit-code'.
  Oct  6 07:39:49 akranes systemd[28227]: Failed to start Dunst notification daemon.
  Oct  6 07:40:23 akranes dunst[28332]: CRITICAL: Cannot open X11 display.
  Oct  6 07:40:23 akranes systemd[28315]: dunst.service: Main process exited, code=exited, status=1/FAILURE
  Oct  6 07:40:23 akranes systemd[28315]: dunst.service: Failed with result 'exit-code'.
  Oct  6 07:40:23 akranes systemd[28315]: Failed to start Dunst notification daemon.

Here's how I start dunst when I start my Xorg session:

  /usr/bin/systemctl status --user dunst

The service does seem to be running fine despite the errors in the logs:

$ systemctl --user status dunst.service
● dunst.service - Dunst notification daemon
   Loaded: loaded (/usr/lib/systemd/user/dunst.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-10-04 11:05:36 PDT; 1 day 22h ago
     Docs: man:dunst(1)
 Main PID: 5046 (dunst)
   Memory: 3.9M
   CGroup: /user.slice/user-1000.slice/user@1000.service/dunst.service
           └─5046 /usr/bin/dunst

Francois

Hi,

This is usually caused by not having exported the DISPLAY variable into the
systemd environment, there has been extensive discussion about this in #347[1]
upstream.
How are you starting X11, are you using a custom xinitrc?

However it is weird that systemd reports that dunst is running even though it
obviously fails to start. I'm not sure what is going on there.

[1] https://github.com/dunst-project/dunst/issues/347

I've got this line in my ~/.config/i3/config:

  exec --no-startup-id /home/francois/devel/remote/user-scripts/startup

and that corresponds to a script [1] with these lines:

  /usr/bin/systemctl --user import-environment DISPLAY
  /usr/bin/systemctl status --user dunst

I looked at both of the following bugs before switching [2] to the systemd
service:

https://github.com/dunst-project/dunst/issues/347
https://github.com/dunst-project/dunst/issues/611

I don't think it fails to start because it works fine and it looks like
this:

  $ systemctl status --user dunst.service
  ● dunst.service - Dunst notification daemon
     Loaded: loaded (/usr/lib/systemd/user/dunst.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2019-10-11 19:21:31 PDT; 15h ago
       Docs: man:dunst(1)
   Main PID: 5330 (dunst)
     Memory: 4.0M
     CGroup: /user.slice/user-1000.slice/user@1000.service/dunst.service
             └─5330 /usr/bin/dunst

  $ pgrep -a dunst
  5330 /usr/bin/dunst

The part that confuses me is that once a day (always almost exactly at the
same time) it tries to start or restart (and fails) even though it's already
running in my user session.

Is there a cron-like job that runs every morning?

Francois

[1] https://github.com/fmarier/user-scripts/blob/master/startup
[2] https://github.com/fmarier/user-scripts/commit/e8be9777cb7af012962408ba8f66ff19b13a485e

Huh, why `status` and not `start`? Is that a typo?

Perhaps the import-environment fails, could you add `echo DISPLAY=$DISPLAY` under that
and see if it appears in .xsession_errors?

Indeed that looks like it's running - however the error you showed is always
fatal and the logs do show dunst service failed a few times, so something
happens at that point in time daily that prevents access to X11 and dunst can't
start? Bizarre.

No, there's nothing that's scheduled to run at a specific time. However, dbus
_will_ try to start dunst via systemd if a notification comes through and it
detects that it isn't running. Perhaps you have a cron scheduled at that time?

Yes, typo indeed! I've now fixed that.

I added a line to that startup script to email me the value of $DISPLAY when
it runs and I got the following:

  DISPLAY=:1

Oct 20 07:04:58 hostname dunst[9811]: CRITICAL: Cannot open X11 display.
Oct 20 07:04:58 hostname systemd[9794]: dunst.service: Main process exited, code=exited, status=1/FAILURE
Oct 20 07:04:58 hostname systemd[9794]: dunst.service: Failed with result 'exit-code'.
Oct 20 07:04:58 hostname systemd[9794]: Failed to start Dunst notification daemon.
Oct 20 07:09:28 hostname dunst[24191]: CRITICAL: Cannot open X11 display.
Oct 20 07:09:28 hostname systemd[24177]: dunst.service: Main process exited, code=exited, status=1/FAILURE
Oct 20 07:09:28 hostname systemd[24177]: dunst.service: Failed with result 'exit-code'.
Oct 20 07:09:28 hostname systemd[24177]: Failed to start Dunst notification daemon.
Oct 20 07:42:24 hostname dunst[18236]: CRITICAL: Cannot open X11 display.
Oct 20 07:42:24 hostname systemd[18223]: dunst.service: Main process exited, code=exited, status=1/FAILURE
Oct 20 07:42:24 hostname systemd[18223]: dunst.service: Failed with result 'exit-code'.
Oct 20 07:42:24 hostname systemd[18223]: Failed to start Dunst notification daemon.
Oct 20 07:42:50 hostname dunst[18303]: CRITICAL: Cannot open X11 display.
Oct 20 07:42:50 hostname systemd[18289]: dunst.service: Main process exited, code=exited, status=1/FAILURE
Oct 20 07:42:50 hostname systemd[18289]: dunst.service: Failed with result 'exit-code'.
Oct 20 07:42:50 hostname systemd[18289]: Failed to start Dunst notification daemon.

I looked at the notification log (Ctrl+`) this morning after seeing these
errors in the logs and there was nothing matching that timestamp. Not
surprising since these are complaints about failing to start.

The only cronjob I have that is set to run specifically at 7am is:

  /etc/cron.d/anacron

but then maybe the /etc/cron.daily/ jobs also run at around that time.

Francois

I apologise for the delay in replying but I don't have many clues as to what's
happening here or any easy ways to debug it.

Curious why it's :1 and not :0, are you running a second X11 session? It could
have something to do with that if that's the case.

Well, when anacron runs it basically also calls all global crons in the system
so that doesn't help much.

From the logs you provided it seems that for your own user session dunst seems
to be running normally even when these failure logs appears, so my working
hypotheses now is that there's an automated job trying to send a notification as
a different user - systemd tries to start dunst which fails as there's no X11
running as that user.

Does that sound something that might be happening - do you have any such
customizations to your systems?

I'm not aware of any packages that use libnotify notifications as part of
cron/automated jobs so it's most likely something added later.